More Clustering — Hierarchical and DBSCAN
Choose between hierarchical clustering and DBSCAN, interpret their assumptions, and build a scaling-aware workflow.
Clusters are assumptions about shape, distance and density
Before this lesson, understand K-Means, feature scaling, distance, and the difference between labelled and unlabelled learning. K-Means partitions points around a chosen number of centroids, which works well for compact, roughly spherical groups. It is not a universal definition of a cluster. Here you will learn two alternatives: hierarchical clustering, which builds a tree of nested merges, and DBSCAN, which groups dense regions and marks isolated points as noise.
- Explain how linkage changes hierarchical merges.
- Tune DBSCAN with its density assumptions and noise labels in mind.
- Compare cluster stability, shape and practical usefulness rather than relying on one score.
Suppose a learning platform wants to understand usage patterns. Some behaviours may form nested groups: occasional learners inside broader study styles. Other patterns may be irregular: dense activity pockets separated by sparse areas. The method should match the geometry and the decision you need to make, rather than the algorithm you happen to know first.
| Method | Cluster definition | Useful when | Main limitation |
|---|---|---|---|
| K-Means | Near centroid under a distance metric | Compact groups and a defensible K | Sensitive to scale, outliers, and non-spherical shapes |
| Agglomerative hierarchy | Repeatedly merge the nearest clusters | Nested structure or need to inspect multiple cuts | Can be expensive; linkage and distance change results |
| DBSCAN | Connected regions of sufficient local density | Irregular shapes and meaningful noise points | One density setting may not fit varying-density groups |
A dendrogram records how groups combine
Agglomerative clustering starts with each row as its own cluster. At each step it merges the closest pair according to a linkage rule. Single linkage uses the nearest pair of points; complete linkage uses the farthest pair; average linkage averages pairwise distances; Ward linkage merges to minimise the increase in within-cluster variance. The sequence of merges forms a hierarchy.
A dendrogram is a tree diagram of those merges. Cutting it at a selected height yields a number of clusters; the tree lets you inspect several plausible cuts without rerunning the merge procedure. A tall merge joins groups that were relatively far apart, which can suggest a cut—but it is evidence to investigate, not an objective answer to “the correct K”.
The truncated plot keeps the display readable; use the full tree for smaller datasets or inspect selected branches. fcluster returns a flat cut under a specified criterion. Compare a few cuts with profiles and stability rather than choosing a cut solely from the drawing.
Find dense regions without asking for K
DBSCAN uses two controls. eps is the neighbourhood radius; min_samples is the density threshold used to identify core points. A core point has at least that many neighbours in its radius (including itself in scikit-learn's convention). Points reachable from core points join their cluster. Points that cannot be reached are labelled noise with label -1.
Unlike K-Means, DBSCAN can find non-convex shapes and does not require the number of clusters up front. It can also highlight unusual observations. But its density assumption is important: one global radius can merge dense regions or fragment sparse ones when cluster density varies. Scale numeric features first, because a large-unit feature can dominate neighbourhood distance.
The parameter values are starting points for this generated dataset, not reusable defaults. On real data, visualise nearest-neighbour distances, vary plausible settings, inspect cluster sizes, and test whether the resulting groups are stable and useful. For high-dimensional data, distance often becomes less informative, so consider representation, feature selection or dimensionality reduction before interpreting density results.
Preprocessing changes the geometry
Scaling is not a mechanical checkbox; it defines what “near” means. If distance in a feature is not meaningful in your application, rescaling it will not fix the concept. Remove identifiers, consider correlated features that unintentionally count the same information multiple times, and encode categories with an approach appropriate to the distance metric. Fit transformations only on the data being analysed and preserve the same pipeline for future assignments.
| Diagnostic | What to inspect | Why it matters |
|---|---|---|
| Cluster sizes | Counts per label and proportion marked noise | Tiny groups may be fragments or genuinely rare patterns |
| Geometry | Scatter plots or projections, with original features reviewed | A 2-D projection can hide or distort structure |
| Stability | Labels under resampling or modest parameter changes | Fragile clusters may be artifacts of one sample |
| Utility | A downstream action or meaningful profile per group | Clusters without a use case are not automatically valuable |
For usage segmentation, compare the distribution of study frequency, completion, device and time-of-day across clusters. Avoid naming a segment from one noisy feature. If groups are used to target learners, check whether a cluster is a proxy for a sensitive attribute and whether the action benefits the learner.
Use diagnostics to narrow choices, then check stability
For DBSCAN, inspect a sorted k-nearest-neighbour distance plot as one way to find candidate eps values; the bend is a heuristic, not a proof. Change min_samples and compare the number and size of clusters, noise rate and whether important neighborhoods remain together. For hierarchical clustering, compare plausible linkage/cut combinations and inspect the merge structure before naming segments.
Silhouette scores can be useful when the distance assumptions fit, but they favor some compact shapes and are not a universal measure of clustering quality. Use bootstrap or repeated subsamples to test whether group membership is stable. If the clusters will trigger an action, test that action with domain experts and affected users before broad rollout.
Avoid false certainty from an unsupervised result
Choose the method by the structure you expect
- Generate compact blobs and compare K-Means with Ward hierarchical clustering. Compare cluster sizes and profiles, not only label numbers.
- Fit DBSCAN to two-moons data. Vary eps and min_samples; record how many clusters and noise rows result.
- Create a dataset with features in different units. Run DBSCAN before and after scaling and explain why the partition changes.
- Apply a clustering method to a dataset you understand. Write a one-paragraph profile for each group and identify one limitation.
- Propose a stability test: resample the observations or perturb a parameter slightly, then compare the resulting assignments.
Clustering is a modelling choice about geometry. Hierarchical clustering provides a merge tree and flexible cuts; DBSCAN detects dense connected regions and noise without choosing K. Neither is automatically superior to K-Means. Scaling, parameter sensitivity, stability, interpretable profiles and responsible downstream use determine whether the structure is useful.
Learn how to flag unusual observations when labels are scarce, and how to evaluate alerts responsibly.
🎯 Key Takeaways
- ✓Hierarchical clustering builds a sequence of merges; linkage controls the meaning of proximity.
- ✓DBSCAN groups dense, connected regions and marks points that are not density-reachable as noise.
- ✓Scale and feature representation define distance, so preprocessing is part of the clustering model.
- ✓Inspect cluster sizes, profiles, parameter sensitivity and stability; do not infer truth from a plot alone.
- ✓Plan how new observations are assigned and how cluster behavior is monitored before using groups in production.
HELP US IMPROVE
How was this lesson?
Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.
Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.
