UMAP

class torchdr.UMAP(n_neighbors: float = 30, n_components: int = 2, min_dist: float = 0.1, spread: float = 1.0, a: float | None = None, b: float | None = None, lr: float = 0.1, optimizer: str = 'SGD', optimizer_kwargs: dict | None = None, scheduler: str = 'constant', scheduler_kwargs: dict | None = None, init: str = 'pca', init_scaling: float = 0.0001, tol: float = 1e-07, max_iter: int = 2000, tolog: bool = False, device: str | None = None, keops: bool = False, verbose: bool = False, random_state: float = 0, early_exaggeration: float = 1.0, coeff_repulsion: float = 1.0, early_exaggeration_iter: int = 0, tol_affinity: float = 0.001, max_iter_affinity: int = 100, metric_in: str = 'sqeuclidean', metric_out: str = 'sqeuclidean', n_negatives: int = 5)

Bases: SampledNeighborEmbedding

Implementation of UMAP introduced in [M18] and further studied in [D21].

It involves selecting a UMAPAffinityIn as input affinity \(\mathbf{P}\) and a UMAPAffinityOut as output affinity \(\mathbf{Q}\).

The loss function is defined as:

\[-\sum_{ij} P_{ij} \log Q_{ij} + \sum_{i,j \in N(i)} \log (1 - Q_{ij})\]

where \(N(i)\) is the set of negatives samples for point \(i\).

Parameters:

• n_neighbors (int) – Number of nearest neighbors.

• n_components (int, optional) – Dimension of the embedding space.

• min_dist (float, optional) – Minimum distance between points in the embedding space.

• spread (float, optional) – Initial spread of the embedding space.

• a (float, optional) – Parameter for the Student t-distribution.

• b (float, optional) – Parameter for the Student t-distribution.

• lr (float, optional) – Learning rate for the algorithm, by default 1e-1.

• optimizer ({'SGD', 'Adam', 'NAdam'}, optional) – Which pytorch optimizer to use, by default ‘SGD’.

• optimizer_kwargs (dict, optional) – Arguments for the optimizer, by default None.

• scheduler ({'constant', 'linear'}, optional) – Learning rate scheduler.

• scheduler_kwargs (dict, optional) – Arguments for the scheduler, by default None.

• init ({'normal', 'pca'} or torch.Tensor of shape (n_samples, output_dim), optional) – Initialization for the embedding Z, default ‘pca’.

• init_scaling (float, optional) – Scaling factor for the initialization, by default 1e-4.

• tol (float, optional) – Precision threshold at which the algorithm stops, by default 1e-7.

• max_iter (int, optional) – Number of maximum iterations for the descent algorithm. by default 2000.

• tolog (bool, optional) – Whether to store intermediate results in a dictionary, by default False.

• device (str, optional) – Device to use, by default “auto”.

• keops (bool, optional) – Whether to use KeOps, by default False.

• verbose (bool, optional) – Verbosity, by default False.

• random_state (float, optional) – Random seed for reproducibility, by default 0.

• early_exaggeration (float, optional) – Coefficient for the attraction term during the early exaggeration phase. By default 1.0.

• coeff_repulsion (float, optional) – Coefficient for the repulsion term, by default 1.0.

• early_exaggeration_iter (int, optional) – Number of iterations for early exaggeration, by default 250.

• tol_affinity (float, optional) – Precision threshold for the input affinity computation.

• max_iter_affinity (int, optional) – Number of maximum iterations for the input affinity computation.

• metric_in ({'euclidean', 'manhattan'}, optional) – Metric to use for the input affinity, by default ‘euclidean’.

• metric_out ({'euclidean', 'manhattan'}, optional) – Metric to use for the output affinity, by default ‘euclidean’.

• n_negatives (int, optional) – Number of negative samples for the noise-contrastive loss, by default 5.

References

[M18]

Leland McInnes, John Healy, James Melville (2018). UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.

[D21]

Sebastian Damrich, Fred Hamprecht (2021). On UMAP’s True Loss Function. Advances in Neural Information Processing Systems 34 (NeurIPS).

Examples using UMAP:

Neighbor Embedding on genomics & equivalent affinity matcher formulation

Neighbor Embedding on genomics & equivalent affinity matcher formulation