K-Means — Module 5.3

How This Lesson Fits the Module

Module 5.3 opens with K-Means—the baseline clustering algorithm most teams reach for first. It partitions data into a fixed number of spherical groups by minimizing within-cluster variance, and it scales well when you have a rough idea of how many segments exist.

K-Means assumes clusters are roughly equal in size and density. Later lectures (DBSCAN, Hierarchical Clustering) handle irregular shapes and unknown cluster counts. Start here to learn the vocabulary: centroids, inertia, and the elbow method.

Learning Objectives

By the end of this lesson, students should be able to:

Explain how K-Means alternates between assignment and centroid updates.
Choose n_clusters using the elbow method and silhouette score.
Recognize when scaling is mandatory before clustering.
Implement K-Means with sklearn and interpret cluster labels.
List failure modes: non-spherical clusters, uneven sizes, outliers.
Decide when K-Means is appropriate vs density-based or hierarchical methods.

How K-Means Works

K-Means groups n observations into k clusters. Each point belongs to the cluster whose centroid (mean vector) is closest. The algorithm repeats two steps until assignments stabilize:

Assign each point to the nearest centroid (usually Euclidean distance).
Update each centroid to the mean of its assigned points.

Initialization matters: random starting centroids can land in poor local minima. sklearn’s KMeans runs multiple restarts (n_init) and keeps the best result by inertia—total within-cluster sum of squared distances.

Hyperparameter	What it controls	Typical starting point
`n_clusters`	Number of segments k	Domain guess or elbow/silhouette sweep
`n_init`	Random restarts	10–20 for stability
`max_iter`	Iterations per restart	300 (default is usually fine)
`random_state`	Reproducibility	Always set in teaching and production

sklearn Implementation

Distance-based clustering requires comparable feature scales. Fit a scaler on training data, then cluster the transformed matrix.

import numpy as np
import pandas as pd
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.metrics import silhouette_score

# X: numeric feature matrix (e.g., RFM customer features)
cluster_pipe = Pipeline([
    ("scale", StandardScaler()),
    ("kmeans", KMeans(n_clusters=4, n_init=20, random_state=42)),
])

labels = cluster_pipe.fit_predict(X)
centroids = cluster_pipe.named_steps["kmeans"].cluster_centers_

df["segment"] = labels
df.groupby("segment")[["recency", "frequency", "monetary"]].mean()

X_scaled = StandardScaler().fit_transform(X)

# Compare k values
for k in range(2, 9):
    km = KMeans(n_clusters=k, n_init=20, random_state=42)
    km.fit(X_scaled)
    print(k, km.inertia_, silhouette_score(X_scaled, km.labels_))

Engineering Habit — Name Segments by Centroids

Raw labels 0–3 mean nothing to stakeholders. Profile each cluster’s feature means, size, and business metrics, then assign names like “High-value loyal” or “At-risk churn.” Document the clustering date and feature set for reproducibility.

Choosing the Number of Clusters

There is no single correct k. Use multiple signals and domain input together.

Elbow Method

Plot inertia vs k
Look for diminishing returns (“elbow”)
Subjective but fast
Works when clusters are well separated

Silhouette Score

Range −1 to +1 per point
Higher = tighter, well-separated clusters
Compare average across candidate k
Penalizes overlapping groups

When to Use K-Means

Use K-Means when…	Prefer another method when…
Clusters are roughly spherical and similar size	Clusters are crescent-shaped or nested rings
You have a business reason for a fixed k	Cluster count is unknown and exploratory
Dataset is large (scales with Lloyd’s algorithm)	You need explicit outlier/noise labels
Features are numeric and scaled	Mixed categorical data without encoding

Critical Mistake — Clustering Without Scaling

If one feature is in dollars (0–1,000,000) and another is age (0–100), distance is dominated by the large-scale column. Always scale numeric inputs—or use algorithms less sensitive to scale, like tree-based approaches on engineered features.

K-Means vs Other Module Methods

K-Means is a partitioning method: every point gets a label, and k is fixed upfront. DBSCAN discovers density-connected regions and marks sparse points as noise. Hierarchical clustering builds a dendrogram so you can cut at different levels. PCA and visualization methods (t-SNE, UMAP) reduce dimensions—often used before or alongside clustering to inspect structure.

Knowledge Check

Short Answer: What two steps does K-Means repeat? Answer: Assign points to nearest centroids, then recompute centroids as cluster means.
True/False: K-Means always finds the globally optimal clustering. Answer: False—it converges to a local minimum depending on initialization.
Multiple Choice: Best first step before K-Means on mixed-scale features: (a) drop outliers only, (b) StandardScaler, (c) increase k. Answer: (b).
Short Answer: What does inertia measure? Answer: Total within-cluster sum of squared distances to centroids.
Short Answer: Why set n_init > 1? Answer: Multiple random starts reduce the chance of a poor local optimum.
True/False: K-Means assumes roughly spherical, similar-sized clusters. Answer: True.
Multiple Choice: Silhouette near 1 means: (a) overlapping clusters, (b) well-separated clusters, (c) noise only. Answer: (b).
Short Answer: What is k-means++? Answer: A smarter centroid initialization that spreads starting points.
True/False: Elbow plots give a unique mathematically correct k. Answer: False—they are a heuristic plus domain judgment.
Multiple Choice: After fitting, new rows are assigned by: (a) rebuilding a dendrogram, (b) nearest centroid, (c) DBSCAN eps. Answer: (b).

Key Takeaways

K-Means partitions data into k spherical groups via centroid iteration.
Scale features first; profile and name clusters for stakeholders.
Use elbow plots and silhouette scores to guide k, not replace domain judgment.
Switch to DBSCAN or hierarchical methods for irregular shapes or unknown k.
Next: DBSCAN—density-based clustering with noise detection.

Trainer’s Guide

Hands-on idea: Cluster the Iris or a retail RFM dataset with k = 2–6. Students plot inertia and silhouette, then present segment profiles to the class.

Discussion prompt: Marketing wants exactly five segments for campaign slots. Does that justify k = 5 even if the elbow suggests three?

Recap: K-Means partitions data into k spherical clusters via centroid iteration—scale first and justify k. Continue with DBSCAN.