K-Means asks you to choose k before fitting. DBSCAN discovers dense regions and noise. Hierarchical clustering gives a third lens: it builds a tree of nested groups so you can inspect structure at multiple levels.
This lesson is useful when stakeholders want an explainable segmentation path, not just final labels. The dendrogram shows which observations merge, when they merge, and how far apart groups are.
Learning Objectives
By the end of this lesson, students should be able to:
- Explain agglomerative hierarchical clustering as a bottom-up merge process.
- Interpret a dendrogram and choose a cut height for cluster labels.
- Compare linkage strategies: single, complete, average, and Ward.
- Implement hierarchical clustering with sklearn and scipy.
- Recognize scale, distance metric, and computational constraints.
- Decide when hierarchical clustering is more useful than K-Means or DBSCAN.
How Hierarchical Clustering Works
The most common version is agglomerative clustering. It starts with every observation as its own cluster, then repeatedly merges the two closest clusters until everything belongs to one tree. The sequence of merges is displayed as a dendrogram.
The distance between individual points is controlled by the metric, while the distance between groups is controlled by linkage. Linkage choice changes the shape and stability of the resulting clusters.
| Linkage | How distance is measured | Best use |
|---|---|---|
| Single | Closest pair between clusters | Can find chains; sensitive to noise |
| Complete | Farthest pair between clusters | Compact clusters with clear separation |
| Average | Average pairwise distance | Balanced default for many exploratory tasks |
| Ward | Merge that minimizes variance increase | Numeric Euclidean data; K-Means-like compact groups |
sklearn and scipy Implementation
Scale numeric features before distance-based clustering. Use sklearn to create labels and scipy to plot a dendrogram for explanation.
Dendrograms become unreadable with thousands of rows. For large datasets, plot a representative sample, then fit labels on the full scaled matrix if the algorithm is computationally feasible.
Choosing Cluster Labels
A dendrogram does not force a single answer. You choose a horizontal cut height or specify n_clusters. A good cut usually crosses long vertical branches, meaning groups remain separate until a large merge distance.
Choose by n_clusters
- Simple for reporting and dashboards
- Matches business constraints
- Easy to compare against K-Means
- Can hide natural nested structure
Choose by distance_threshold
- Lets the data decide cluster count
- Useful when merge distances have clear gaps
- Supports anomaly-like small groups
- Requires careful visual inspection
Hierarchical clustering is deterministic for a fixed setup, but the tree can change when you scale features, change distance metrics, sample rows, or switch linkage. Validate clusters with profiles, stability checks, and domain review.
When to Use Hierarchical Clustering
| Use it when… | Be careful when… |
|---|---|
| You need explainable nested groups | The dataset is very large |
| The number of clusters is uncertain | Feature scales are inconsistent |
| Stakeholders want to inspect merge history | Noise points create misleading chains |
| You can afford pairwise distance computation | You need real-time prediction for new points |
Knowledge Check
- Short Answer: What does agglomerative clustering start with? Answer: Each observation as its own cluster.
- True/False: A dendrogram can be cut at different heights to produce different cluster counts. Answer: True.
- Multiple Choice: Ward linkage minimizes: (a) variance increase, (b) classification error, (c) entropy. Answer: (a).
- Short Answer: Why scale before hierarchical clustering? Answer: Distance calculations are distorted by unequal feature scales.
- Short Answer: Name one limitation. Answer: It can be expensive for large datasets and does not naturally predict new points.
- True/False: Single linkage is prone to chaining elongated clusters. Answer: True.
- Multiple Choice: Complete linkage uses: (a) min pairwise distance, (b) max pairwise distance between clusters, (c) centroid only. Answer: (b).
- Short Answer: What is a dendrogram? Answer: A tree diagram showing the order and distance of cluster merges.
- True/False: Agglomerative clustering assigns new rows as easily as K-Means centroids. Answer: False—you typically recut or refit; no simple centroid assign.
- Multiple Choice: Average linkage: (a) uses mean pairwise distance, (b) is only Ward, (c) ignores scale. Answer: (a).
Key Takeaways
- Hierarchical clustering builds a tree of nested groups rather than one fixed partition.
- Dendrograms make cluster structure explainable, but cut height is still a modeling decision.
- Linkage choice matters: Ward is compact, single can chain, complete is stricter.
- Scale features and validate clusters with profiles and stability checks.
- Next: PCA—reduce high-dimensional data into principal components.
Hands-on idea: Cluster a small customer or Iris dataset with Ward linkage. Have students draw a horizontal cut on the dendrogram, then compare their chosen cut to n_clusters=3 labels.
Discussion prompt: If a business team wants a two-level taxonomy, how would a dendrogram help them explain both broad groups and subgroups?
Recap: Hierarchical clustering builds a dendrogram of nested groups; cut height and linkage are modeling choices. Continue with PCA.