API and Modules#

Dimensionality Reduction sklearn Compatible Estimators#

TorchDR provides a set of classes that are compatible with the sklearn API. For example, running TSNE can be done in the exact same way as running sklearn.manifold.TSNE with the same parameters. Note that the TorchDR classes work seamlessly with both Numpy and PyTorch tensors.

For all methods, TorchDR provides the ability to use GPU acceleration using device='cuda' as well as LazyTensor objects that allows to fit large scale models directly on the GPU memory without overflows using keops=True.

TorchDR supports a variety of dimensionality reduction methods. They are presented in the following sections.

Spectral Embedding#

PCA([n_components, device, verbose, ...])

Principal Component Analysis module.

IncrementalPCA([n_components, copy, ...])

Incremental Principal Components Analysis (IPCA) leveraging PyTorch for GPU acceleration.

KernelPCA(affinity, n_components, device, ...)

Kernel Principal Component Analysis module.

PHATE([k, n_components, t, alpha, ...])

Implementation of PHATE introduced in [Moon et al., 2019].

Neighbor Embedding#

UMAP([n_neighbors, n_components, min_dist, ...])

UMAP introduced in [McInnes et al., 2018] and further studied in [Damrich and Hamprecht, 2021].

LargeVis([perplexity, n_components, lr, ...])

LargeVis algorithm introduced in [Tang et al., 2016].

PACMAP([n_neighbors, n_components, lr, ...])

PACMAP algorithm introduced in [Wang et al., 2021].

InfoTSNE([perplexity, n_components, lr, ...])

InfoTSNE algorithm introduced in [Damrich et al., 2022].

SNE([perplexity, n_components, lr, ...])

Stochastic Neighbor Embedding (SNE) introduced in [Hinton and Roweis, 2002].

TSNE([perplexity, n_components, lr, ...])

t-Stochastic Neighbor Embedding (t-SNE) introduced in [Van der Maaten and Hinton, 2008].

TSNEkhorn([perplexity, n_components, lr, ...])

TSNEkhorn algorithm introduced in [Van Assel et al., 2024].

COSNE([perplexity, lambda1, gamma, ...])

Implementation of the CO-Stochastic Neighbor Embedding (CO-SNE) introduced in [Guo et al., 2022].

Advanced Dimensionality Reduction with TorchDR#

TorchDR provides a set of generic classes that can be used to implement new dimensionality reduction methods. These classes provide a modular and extensible framework that allows you to focus on the core components of your method.

Base Classes#

The torchdr.DRModule class is the base class for a dimensionality reduction estimator. It is the base class for all the DR classes in TorchDR.

torchdr.AffinityMatcher is the base class for all the DR methods that use gradient-based optimization to minimize a loss function constructed from two affinities in input and embedding spaces.

DRModule([n_components, device, backend, ...])

Base class for DR methods.

AffinityMatcher(affinity_in[, affinity_out, ...])

Perform dimensionality reduction by matching two affinity matrices.

Base Neighbor Embedding Modules#

Neighbor embedding base modules inherit from the torchdr.AffinityMatcher class. torchdr.SparseNeighborEmbedding relies on the sparsity of the input affinity to compute the attractive term in linear time. torchdr.SampledNeighborEmbedding inherits from this class and adds the possibility to approximate the repulsive term of the loss via negative samples.

NeighborEmbedding(affinity_in[, ...])

Solves the neighbor embedding problem.

SparseNeighborEmbedding(affinity_in[, ...])

Solves the neighbor embedding problem with a sparse input affinity matrix.

SampledNeighborEmbedding(affinity_in[, ...])

Solves the neighbor embedding problem with both sparsity and sampling.

Affinity Classes#

Simple Affinities#

GaussianAffinity([sigma, metric, zero_diag, ...])

Compute the Gaussian affinity matrix.

StudentAffinity([degrees_of_freedom, ...])

Compute the Student affinity matrix based on the Student-t distribution.

ScalarProductAffinity([device, backend, verbose])

Compute the scalar product affinity matrix.

NormalizedGaussianAffinity([sigma, metric, ...])

Compute the Gaussian affinity matrix which can be normalized along a dimension.

NormalizedStudentAffinity([...])

Compute the Student affinity matrix which can be normalized along a dimension.

Affinities Normalized by kNN Distances#

SelfTuningAffinity([K, normalization_dim, ...])

Self-tuning affinity introduced in [Zelnik-Manor and Perona, 2004].

MAGICAffinity([K, metric, zero_diag, ...])

Compute the MAGIC affinity with alpha-decay kernel introduced in [Van Dijk et al., 2018].

PHATEAffinity([metric, device, backend, ...])

Compute the potential affinity used in PHATE [Moon et al., 2019].

PACMAPAffinity([n_neighbors, metric, ...])

Compute the input affinity used in PACMAP [Wang et al., 2021].

UMAPAffinityIn([n_neighbors, tol, max_iter, ...])

Compute the input affinity used in UMAP [McInnes et al., 2018].

Entropic Affinities#

SinkhornAffinity([eps, tol, max_iter, ...])

Compute the symmetric doubly stochastic affinity matrix.

EntropicAffinity([perplexity, tol, ...])

Solve the directed entropic affinity problem introduced in [Hinton and Roweis, 2002].

SymmetricEntropicAffinity([perplexity, lr, ...])

Compute the symmetric entropic affinity (SEA) introduced in [Van Assel et al., 2024].

Other Affinities#

DoublyStochasticQuadraticAffinity([eps, ...])

Compute the symmetric doubly stochastic affinity.

UMAPAffinityOut([min_dist, spread, a, b, ...])

Compute the affinity used in embedding space in UMAP [McInnes et al., 2018].

Scores#

silhouette_score(X, labels[, weights, ...])

Compute the Silhouette score as the mean of silhouette coefficients.

Utils#

pairwise_distances(X[, Y, metric, backend, ...])

Compute pairwise distances matrix between points in two datasets.

binary_search(f, n[, begin, end, max_iter, ...])

Implement the binary search root finding method.

false_position(f, n[, begin, end, max_iter, ...])

Implement the false position root finding method.