Exploring non-negativity for improved manifold embedding: Application to t-SNE

Drawing inspiration from Non-negative Matrix Factorization (NMF), this paper explores the potential of incorporating non-negativity constraints into embedding techniques, with a focus on t-SNE as an application. Specifically, we investigate the following questions: Can enforcing non-negativity in the embedding space enhance interpretability and improve the quality of embedded data? By prioritizing non-negativity, can embedding methods achieve better performance and more meaningful representations? Additionally, does enforcing non-negativity in the embedded space help preserve both the local and global structure of data in the manifold, leading to more accurate and interpretable embeddings? In this work, we could show both objectively and subjectively how enforcing t-SNE to leverage the non-negativity of the data addresses the raised questions. To achieve this, we introduced a novel approach to transforming the additive update rule of the gradient descent used by t-SNE to a multiplicative counterpart to enforce the non-negativity in the embedded space. However, grappling with full non-negativity in the gradient descent formula presents challenges, prompting our focus solely on the (yi-y) term, resulting in a semi-non-negative t-SNE algorithm, shortly named SN-tSNE. Nevertheless, experimental findings substantiate the significant impact of the proposed update rule on the performance and efficacy of the SN-tSNE algorithm. Furthermore, additional experiments are performed to compare SN-tSNE with its precursor t-SNE, as well as the competitive embedding technique UMAP, alongside other relevant embedding and dimensionality reduction models like NMF. The source code of SN-tSNE is available on GitHub (https://github.com/M-Allaoui/SN-tSNE.git).