Similar PCA but I want every element of the first eigenvector to be positive / non-negative matrix factorization?

External question — Cross Validated Stack Exchange Author: CuriousMind Original post: https://stats.stackexchange.com/questions/665028 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. I am familiar with the PCA algorithm for dimension reduction. But I would like every element of the first principal component to have positive sign. So when I try to use my principal component, it's a bit easier to interpret. I am aware that this will lose some optimality, but I would like to trade for some interpretability. I am aware of " non-negative matrix factorization " that seemingly does the job. But it requires input data to be positive. That doesn't fit my requirement though ... my input data can be negative. It is just that features tend to be positively correlated. I can also write out the optimization problem: maximizing $w' S w$ subject to $w \geq 0$ and $||w|| = 1$ . But that's not a convex problem? At least my solver can't solve this... Are there other solutions out there?

External answer — Cross Validated Stack Exchange Author: Cryo Original post: https://stats.stackexchange.com/a/665029 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. Sounds to me like it should be solvable with simple coordinate transformation. Let's say your original basis is: $\{\mathbf{e}_1,\dots \mathbf{e}_n\}$ . You have your matrix $\mathbf{M}$ , with components: $$ M_{i,j}=\mathbf{e}_i^T.\mathbf{M}.\mathbf{e}_j $$ You then do eigen-vector decomposition, part of PCA, and find a vector: $\mathbf{v}^{(\lambda)}$ such that: $$ \mathbf{M}.\mathbf{v}^{(\lambda)}=\lambda\cdot \mathbf{v}^{(\lambda)} $$ i.e. an eigenvector with eigen-value $\lambda$ . Lets say that's the eigenvector you want to focus on (e.g. because this $\lambda$ is the largest-magnitude eigenvalue). You then want to guarantee that the components of the eigen-vector are positive. To do this, let me define a new basis: $\{\mathbf{q}_{1}\dots \mathbf{q}_n\}$ . I can choose $$ \mathbf{q}_i^T.\mathbf{v}^{(\lambda)}=\alpha,\quad 0<\alpha<\left|\mathbf{v}^{(\lambda)}\right|/\sqrt{n} $$ This will give me: $$ \mathbf{q}_i^T.\mathbf{v}^{(\lambda)}=v^{(\lambda)}_1\cdot\left(\mathbf{q}_i^T.\mathbf{e}_1\right)+\dots + v^{(\lambda)}_n\cdot\left(\mathbf{q}_i^T.\mathbf{e}_n\right)=\alpha $$ Where $v^{(\lambda)}_i=\mathbf{q}_i^T.\mathbf{e}_i$ are the components of the eigen-vector in the original basis. All you need to do now is to construct remaining $n-1$ components $\mathbf{q}_i$ , maintaining orthogonality to prior ones, and you will have a new basis set. In that basis, by design, the representation of your eigen-vector of choice will have positive components. The representation of matrix, i.e. the matrix components in that set will be: $$ M'_{i,j}=\mathbf{q}_i^T.\mathbf{M}.\mathbf{q}_j=\sum_{s,p}\left(\mathbf{q}_i^T.\mathbf{e}_s\right).M_{s,p}.\left(\mathbf{e}_p^T\mathbf{q}_j\right) $$