Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Matrix decomposition (PCA, NMF, ICA)

solow-decomposition covers PCA variants (kernel, incremental, sparse), NMF (batch and mini-batch), FastICA, truncated SVD (LSA), dictionary learning, latent Dirichlet allocation, and random projections.

PCA family

EstimatorWhat it does
KernelPcaNon-linear PCA in an RKHS via Gram-matrix centering. Linear, RBF, polynomial kernels.
IncrementalPCAStreaming PCA for out-of-core data.
SparsePCAPCA with L1 penalty on components for sparse loadings.

For classical (full-covariance) PCA, factor analysis, and canonical correlation, see Multivariate analysis.

Independent and non-negative

EstimatorWhat it does
FastIcaHyvärinen (1999) symmetric decorrelation with LogCosh or Exp non-Gaussianity contrasts.
NmfLee-Seung (2001) multiplicative updates under the Frobenius objective.
MiniBatchNmfOnline mini-batch NMF for large data.

Truncated SVD and dictionary learning

EstimatorWhat it does
TruncatedSVDTop-k singular value decomposition, ideal for latent semantic analysis.
DictionaryLearningSparse coding via alternating dictionary and code updates.
MiniBatchDictionaryLearningOnline variant for streaming or large data.

Topic modeling

EstimatorWhat it does
LatentDirichletAllocationBatch variational Bayes for topic modeling.

Random projection

EstimatorWhat it does
GaussianRandomProjectionJohnson-Lindenstrauss projection with a random dense Gaussian matrix.
SparseRandomProjectionAchlioptas sparse projection with {-1, 0, +1} entries.
johnson_lindenstrauss_min_dimAnalytic lower bound on the target dimension.

Example

#![allow(unused)]
fn main() {
use solow_decomposition::{TruncatedSVD, Nmf};

let svd = TruncatedSVD::new(50).fit(&x)?;
let x_reduced = svd.transform(&x)?;

let nmf = Nmf::new(10).max_iter(300).seed(42).fit(&x)?;
Ok::<_, solow_core::Error>(())
}

References

  • Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks.
  • Lee, D. D., & Seung, H. S. (2001). Algorithms for non-negative matrix factorization. NIPS.
  • Schölkopf, B., Smola, A., & Müller, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation.