.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/affinities/demo_ea_adaptivity.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_affinities_demo_ea_adaptivity.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_affinities_demo_ea_adaptivity.py:


Entropic Affinities can adapt to varying noise levels
=====================================================

We show the adaptivity property of entropic affinities on a toy
simulated dataset with heteroscedastic noise.

.. GENERATED FROM PYTHON SOURCE LINES 9-20

.. code-block:: Python


    # Author: Hugues Van Assel <vanasselhugues@gmail.com>
    #
    # License: BSD 3-Clause License

    import matplotlib.pyplot as plt
    import torch
    from matplotlib import cm

    from torchdr import EntropicAffinity, NormalizedGaussianAffinity








.. GENERATED FROM PYTHON SOURCE LINES 21-23

We generate three Gaussian clusters of points with different standard deviations
(respectively std=1, 2 and 3).

.. GENERATED FROM PYTHON SOURCE LINES 23-52

.. code-block:: Python


    torch.manual_seed(0)
    n_cluster = 20  # number of points per cluster

    X1 = torch.Tensor([-10, -10])[None, :] + torch.normal(
        0, 1, size=(n_cluster, 2), dtype=torch.double
    )
    X2 = torch.Tensor([10, -10])[None, :] + torch.normal(
        0, 2, size=(n_cluster, 2), dtype=torch.double
    )
    X3 = torch.Tensor([0, 10])[None, :] + torch.normal(
        0, 3, size=(n_cluster, 2), dtype=torch.double
    )

    X = torch.cat([X1, X2, X3], 0)


    def plot_affinity_graph(G):
        m = G.max().item()
        for i in range(3 * n_cluster):
            for j in range(i):
                plt.plot(
                    [X[i, 0], X[j, 0]],
                    [X[i, 1], X[j, 1]],
                    color="black",
                    alpha=G[i, j].item() / m,
                )









.. GENERATED FROM PYTHON SOURCE LINES 53-65

Row-normalised Gaussian affinity with constant bandwidth
--------------------------------------------------------

We first consider a Gaussian affinity, normalized by row,
with a **constant bandwidth**.
Such a global bandwidth **only controls the global
entropy** of the affinity.

In ``TorchDR``, we can easily implement it using
:class:`torchdr.NormalizedGaussianAffinity` and setting the
parameter ``normalization_dim=1``.


.. GENERATED FROM PYTHON SOURCE LINES 65-83

.. code-block:: Python


    aff = NormalizedGaussianAffinity(
        sigma=1, normalization_dim=1, backend=None, zero_diag=False
    )
    K = aff(X)

    plt.figure(1, (6, 3))

    plt.subplot(1, 2, 1)
    plt.imshow(K, cmap=cm.Blues)
    plt.title("Gaussian Affinity Matrix")

    plt.subplot(1, 2, 2)
    plot_affinity_graph(K)
    plt.scatter(X[:, 0], X[:, 1], alpha=0.5)
    plt.title("Gaussian Affinity Graph")
    plt.show()




.. image-sg:: /auto_examples/affinities/images/sphx_glr_demo_ea_adaptivity_001.png
   :alt: Gaussian Affinity Matrix, Gaussian Affinity Graph
   :srcset: /auto_examples/affinities/images/sphx_glr_demo_ea_adaptivity_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 84-88

We can observe a remarkable **heterogeneity in the density of connections**.
This occurs because it is less costly to create connections in high-density regions.
As a consequence, points in sparse regions create very few connections with
their neighbors.

.. GENERATED FROM PYTHON SOURCE LINES 90-101

Entropic affinity (adaptive bandwidth)
--------------------------------------

To remedy this issue, we can use an **entropic affinity**. The entropic affinity
employs an **adaptive bandwidth** that depends on the local density of points.
By controlling the entropy of each row of the affinity matrix, it ensures that
**each point has the same number of effective neighbors** (given by
the ``perplexity`` parameter) regardless of the local density around it.

In ``TorchDR``, this object is available here :
:class:`torchdr.EntropicAffinity`.

.. GENERATED FROM PYTHON SOURCE LINES 101-120

.. code-block:: Python


    aff_ea = EntropicAffinity(
        perplexity=5, backend=None, verbose=False, zero_diag=False, sparsity=False
    )
    EA = aff_ea(X)

    plt.figure(1, (6, 3))

    plt.subplot(1, 2, 1)
    plt.imshow(EA, cmap=cm.Blues)
    plt.title("Entropic Affinity Matrix")

    plt.subplot(1, 2, 2)
    plot_affinity_graph(EA)
    plt.scatter(X[:, 0], X[:, 1], alpha=0.5)
    plt.title("Entropic Affinity Graph")
    plt.show()





.. image-sg:: /auto_examples/affinities/images/sphx_glr_demo_ea_adaptivity_002.png
   :alt: Entropic Affinity Matrix, Entropic Affinity Graph
   :srcset: /auto_examples/affinities/images/sphx_glr_demo_ea_adaptivity_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 121-126

We can now observe a **homogeneous density of connections** across clusters.
Thus, the entropic affinity effectively filters out the various noise levels.

This affinity is an important component of the **TSNE** algorithm.



.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 2.738 seconds)


.. _sphx_glr_download_auto_examples_affinities_demo_ea_adaptivity.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: demo_ea_adaptivity.ipynb <demo_ea_adaptivity.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: demo_ea_adaptivity.py <demo_ea_adaptivity.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: demo_ea_adaptivity.zip <demo_ea_adaptivity.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_