t-SNE — Module 5.3

How This Lesson Fits the Module

PCA gives linear projections for modeling. t-SNE (t-Distributed Stochastic Neighbor Embedding) prioritizes local neighborhoods—producing striking 2D plots where similar points cluster visually. It is an exploration tool, not a production feature transformer.

Use t-SNE to sanity-check clusters from K-Means or DBSCAN, compare with UMAP in the capstone, and never treat axis distances as meaningful.

Learning Objectives

By the end of this lesson, students should be able to:

Explain why t-SNE preserves local structure better than global distances.
Run TSNE from sklearn with sensible perplexity settings.
Interpret t-SNE plots without over-reading cluster separation.
List limitations: non-determinism, no transform for new points, cost.
Color embeddings by cluster labels or metadata for validation.
Choose t-SNE vs PCA vs UMAP for a given exploratory goal.

How t-SNE Works (Intuition)

t-SNE converts high-dimensional similarities into probabilities that neighbors stay close. It then optimizes low-dimensional coordinates so similar points remain near and dissimilar points repel. The t-distribution in the low-D space reduces the “crowding problem” that hurts simpler SNE.

Global geometry is not preserved: distances between far-apart islands in a t-SNE plot are meaningless. Focus on local cohesion and separation.

Hyperparameter	Role	Starting guidance
`perplexity`	Effective number of neighbors	5–50; try 30 on medium datasets
`learning_rate`	Step size in optimization	`"auto"` or 200–1000
`n_iter`	Optimization iterations	1000+; increase for large n
`random_state`	Reproducibility	Always set; layouts still vary slightly

sklearn Implementation

Reduce dimensionality with PCA first when p is large (e.g., 50 components) to speed t-SNE and reduce noise.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
from sklearn.preprocessing import StandardScaler

X_scaled = StandardScaler().fit_transform(X)
X_pca = PCA(n_components=50, random_state=42).fit_transform(X_scaled)

# Subsample for interactive exploration on large n
rng = np.random.default_rng(42)
idx = rng.choice(len(X_pca), size=min(5000, len(X_pca)), replace=False)

tsne = TSNE(
    n_components=2,
    perplexity=30,
    learning_rate="auto",
    init="pca",
    random_state=42,
)
coords = tsne.fit_transform(X_pca[idx])

plt.scatter(coords[:, 0], coords[:, 1], c=labels[idx], s=8, cmap="tab10")
plt.title("t-SNE colored by K-Means segment")
plt.show()

Engineering Habit — Plot Metadata, Not Just Labels

Color t-SNE by cluster ID, true class, churn flag, or time cohort. If structure only appears for the label you already used to train, you may be seeing supervision—not discovery.

When to Use t-SNE

Good Uses

Presentations and EDA slide decks
Validating whether clusters look separated
Comparing embedding models qualitatively
Medium n after PCA preprocessing

Poor Uses

Production feature generation
Measuring inter-cluster distances
Scoring brand-new points without refit
Million-row datasets without heavy subsampling

Method	Linear?	New-point transform?	Best for
PCA	Yes	Yes	Pipelines, variance compression
t-SNE	No	No (standard API)	Local-structure visualization
UMAP	No	Yes (with library support)	Faster global-aware plots + optional transform

Critical Mistake — Clustering in t-SNE Space

Running K-Means on 2D t-SNE coordinates and calling it segmentation confuses visualization with inference. t-SNE distortions change with perplexity and seed. Cluster in the original (or PCA) feature space; use t-SNE only to inspect results.

t-SNE vs UMAP Preview

UMAP often runs faster, better preserves global structure, and can embed new points. t-SNE still wins for fine local detail in some image and embedding workflows. The capstone lesson compares both on the same dataset with the same color scheme.

Knowledge Check

Short Answer: What does perplexity control? Answer: Balance between local and global neighbor information (effective neighborhood size).
True/False: Distance between two separated blobs in t-SNE reflects their true high-D distance. Answer: False—global distances are unreliable.
Multiple Choice: Safe production dimensionality reduction: (a) t-SNE, (b) PCA, (c) both equally. Answer: (b).
Short Answer: Why PCA before t-SNE on images? Answer: Reduces noise and compute time while keeping signal.
Short Answer: How validate K-Means with t-SNE? Answer: Color t-SNE points by cluster label and check visual cohesion.
True/False: t-SNE is stochastic; different seeds can change the plot. Answer: True.
Multiple Choice: Perplexity is typically: (a) 1, (b) about 5–50, (c) equal to n. Answer: (b).
Short Answer: Why subsample before t-SNE on huge datasets? Answer: Compute and memory cost grow quickly; plots remain exploratory.
True/False: You should train a production classifier on t-SNE (x,y) coordinates. Answer: False—they are not stable deployment features.
Multiple Choice: init="pca" in t-SNE: (a) often improves stability, (b) removes the need to scale, (c) sets perplexity. Answer: (a).

Key Takeaways

t-SNE reveals local neighborhoods in 2D—not global distances.
Preprocess with scaling and often PCA; subsample large datasets.
Use for EDA and cluster validation, not deployment features.
Never cluster directly on t-SNE coordinates for production labels.
Next: UMAP—module capstone comparing all unsupervised tools.

Trainer’s Guide

Hands-on idea: Same subsample, three perplexity values (5, 30, 80). Students discuss how island shapes change and why only local patterns are trustworthy.

Discussion prompt: Stakeholders want a t-SNE dashboard refreshed nightly with new users. What technical and interpretability problems arise?

Recap: t-SNE visualizes local neighborhoods in 2D—great for EDA, not for deployment features. Continue with UMAP.