K-Means Clustering — Customer Segmentation
Finding hidden groups in data without labels. Inertia, elbow method, silhouette scores, and when clustering is and is not the right approach.
Every algorithm so far required labels. K-Means does not. It finds hidden groups in your data that you never told it to look for.
Amazon has 300 million registered customers. Nobody has manually labelled them as "budget buyer", "premium shopper", "deal hunter", or "occasional visitor." Those labels do not exist anywhere in the database. But the patterns that define those groups are absolutely there — in the purchase history, order frequency, average spend, and browsing behaviour.
Every algorithm we have covered so far was supervised — you provided the correct labels during training and the algorithm learned to reproduce them. K-Means is unsupervised — you provide only the features, no labels, and the algorithm discovers structure on its own. The "structure" it finds is groups of customers who are more similar to each other than to customers in other groups.
Once you have those groups, you can give them meaningful names. The group with high spend and high frequency becomes "premium". The group with many browsing sessions but few purchases becomes "window shoppers". Now every customer has a segment — a label — without anyone ever manually labelling a single customer.
A new teacher receives 30 students on the first day with no prior information. She watches them for a week. Without anyone telling her, she notices three natural groups forming: students who always sit up front and answer questions, students who work quietly at the back, and students who are social and work in groups. She did not impose these categories — she discovered them from the students' natural behaviour.
That is clustering. No labels given. Groups discovered from the data itself. K-Means is the most widely used algorithm for doing this at scale.
How K-Means works — four steps, repeated until convergence
K-Means is one of the simplest ML algorithms to understand. There are no weights to learn, no gradients to compute. Just four steps repeated until nothing changes.
Inertia — the objective K-Means minimises
K-Means minimises inertia — the sum of squared distances from each point to its assigned centroid. Low inertia means points are close to their cluster centres — tight, compact clusters. High inertia means clusters are loose and spread out.
The problem: inertia always decreases as k increases. With k = n (one cluster per point), inertia is exactly zero — every point is its own centroid. But k = n is a useless clustering. You need a way to find the right k — where adding more clusters stops meaningfully improving the structure. That is the elbow method.
Silhouette score — measures cluster quality without labels
The elbow method is visual and subjective — different people see the elbow at different places. The silhouette score is a quantitative metric that measures clustering quality for each individual point and averages them.
For each point, the silhouette score measures: how close is it to points in its own cluster (a) versus how close is it to points in the nearest other cluster (b)? A score near +1 means the point is well inside its cluster and far from all others — perfect assignment. A score near 0 means the point is on the boundary between two clusters. A score near −1 means the point was probably assigned to the wrong cluster.
Amazon customer segmentation — end to end
Customer segmentation is the most common application of K-Means in e-commerce. The output is not just cluster numbers — it is actionable customer groups that the marketing, product, and growth teams can use. Budget buyers get different promotions than premium buyers. Churning customers get re-engagement campaigns. Window shoppers get conversion-focused nudges.
Three situations where K-Means gives wrong answers
K-Means is simple and fast but has three hard limitations. Knowing them saves you from deploying a clustering that looks plausible but is mathematically wrong for your data.
K-Means assumes clusters are round blobs of equal size. It fails completely on ring-shaped clusters (Module 26 showed this for Spectral Clustering), elongated ellipses, or interleaved crescents. K-Means draws Voronoi boundaries (straight lines equidistant between centroids) — these cannot capture curved cluster shapes.
K-Means tends to split large clusters into multiple pieces while merging small adjacent clusters. The centroid of a very large cluster may be pulled toward dense regions, causing boundary misassignment for points at the edges.
The centroid is the mean — outliers pull it toward themselves. One transaction worth $6,000 in a dataset of $60 average transactions will pull the "high-value" centroid toward it, making the cluster definition unstable and unrepresentative.
Day-one task — build DoorDash restaurant delivery zones
DoorDash wants to cluster restaurant locations into delivery zones so each delivery partner is assigned to a compact geographic area. This is a geographic K-Means problem — the features are latitude and longitude. Each cluster becomes one delivery zone.
Every common K-Means error — explained and fixed
Five things people get wrong about K-Means clustering
K-Means does not discover ground-truth groups — it partitions data into k roughly spherical, similarly-sized regions no matter what the actual structure looks like. Feed it three ellipsoidal blobs of very different sizes and it will still draw straight-line (Voronoi) boundaries between them, happily producing three confident-looking clusters even when the real structure is two overlapping groups and one outlier cloud. There is no labeled ground truth to check the answer against, so "the clusters look reasonable" is often the only validation you get — which is exactly why silhouette scores, domain review, and sanity-checking against business logic all matter more here than in supervised learning.
The elbow method is a visual heuristic, not a computation with one right answer. Real inertia curves rarely have a single obvious bend — they often decline smoothly, or show two or three plausible elbows depending on how hard you squint. Two analysts looking at the same curve can reasonably pick different k values. Treat the elbow as one input among several: pair it with silhouette score, Davies-Bouldin, and — most importantly — whether the resulting groups are actually useful for the business question you are trying to answer. If no k produces clusters anyone can act on, the right takeaway is that clustering itself may not be the right tool.
K-Means converges to a local minimum of inertia, not the global one, and which local minimum it lands in depends heavily on where the centroids started. A single unlucky random initialization can produce noticeably worse clusters than a good one, on the exact same data with the exact same k. This is precisely why K-Means++ exists — it spreads the initial centroids apart deliberately instead of picking them uniformly at random — and why scikit-learn runs the whole algorithm multiple times (n_init) and keeps only the lowest-inertia result. Skipping this and running with one random start is one of the most common ways to get an unstable, hard-to-reproduce clustering.
Every extra feature adds another dimension to the Euclidean distance calculation, and in high-dimensional space distances behave strangely — the gap between the nearest and farthest point shrinks toward zero as dimensions grow, so eventually every point looks roughly equidistant from every other point. At that point "closest centroid" stops being a meaningful concept and the clustering degrades toward noise. More features only help if they carry real separating signal; irrelevant or redundant ones dilute the useful dimensions and actively hurt the result. This is why K-Means is often run after PCA on high-dimensional data rather than on the raw feature set.
Classification has an answer key — you can compute accuracy because the true label for every point is known. Clustering has no such thing: there is no ground-truth "correct" cluster assignment to check predictions against, because the groups are not observed, they are invented by the algorithm. Metrics like inertia and silhouette score measure internal consistency (are points close to their own centroid and far from others), which is a different question from "did we find the right groups" — a clustering can score well internally while being practically useless, or vice versa. This is why cluster interpretation always needs a human sanity check against domain knowledge, in a way that classification accuracy usually does not.
K-Means — 5 questions interviewers actually ask
Start by picking k initial centroids (ideally with K-Means++, which spreads them out rather than choosing uniformly at random). Then repeat two steps until nothing changes: assignment, where every point is assigned to its nearest centroid by Euclidean distance; and update, where each centroid moves to the mean of the points currently assigned to it. Convergence happens when assignments stop changing between iterations, which usually takes somewhere between ten and a few hundred iterations. The final output is a set of k centroids and a label for every point saying which centroid it belongs to. I would mention that the objective being minimized throughout is inertia — the sum of squared distances from each point to its assigned centroid — and that this process only guarantees a local minimum of that objective, not a global one.
K-Means clusters purely by Euclidean distance, and Euclidean distance is just a sum of squared differences across features. If one feature ranges into the thousands (like annual spend) and another ranges from zero to one (like a return rate), the large-scale feature contributes orders of magnitude more to every distance calculation, so the clustering ends up driven almost entirely by that one feature and effectively ignores the rest. Standardizing every feature to mean zero and standard deviation one before fitting puts them on equal footing so each one actually contributes to the distance calculation. This is different from tree-based models, where splits are based on per-feature thresholds and scale does not change which split is chosen.
I would not rely on a single method. Start with the elbow method — plot inertia against a range of k values and look for where the curve stops dropping sharply — but treat it as a rough guide since the bend is often ambiguous. Cross-check with silhouette score across the same range of k, which quantifies how well-separated the clusters are rather than just how tight they are. Then, critically, look at the actual clusters that come out for the top two or three candidate k values: do they correspond to groups a stakeholder could name and act on? If k=4 gives an elbow-approved, high-silhouette clustering that nobody can interpret or use, and k=3 gives slightly worse metrics but clean, actionable segments, I would pick k=3. The metrics narrow the search; domain judgment picks the final answer.
Standard K-Means picks its k starting centroids uniformly at random from the data, which means two starting points can land right next to each other by chance, wasting one cluster's worth of representation and biasing the final result toward a bad local minimum. K-Means++ instead picks the first centroid randomly, then picks each subsequent centroid with probability proportional to its squared distance from the nearest centroid already chosen — so points far from existing centroids are much more likely to be picked next. That spreads the initial centroids across the actual spread of the data, which empirically leads to faster convergence and a lower final inertia on average. It is the default initialization in scikit-learn's KMeans for exactly this reason, and even with it, running multiple initializations (n_init) and keeping the best result is still standard practice.
First I would check the basics: are features scaled, is k reasonable given the elbow and silhouette analysis, and did I set n_init high enough to avoid a bad random start. Then I would visualize the clusters, or a PCA projection of them if there are many dimensions, to see if the shapes look like something K-Means could represent at all — since it can only ever produce roughly convex, similarly-sized partitions. If the true structure is ring-shaped, crescent-shaped, or has clusters of wildly different density, no amount of tuning k will fix it, because the algorithm's fundamental assumption about cluster shape is wrong for that data. In that case I would switch to DBSCAN for arbitrary-shaped, density-based clusters, or a Gaussian Mixture Model if clusters have different sizes and orientations but are still roughly elliptical. Outliers pulling centroids off-center would push me toward K-Medoids instead.
K-Means groups data into clusters. PCA compresses data into fewer dimensions.
K-Means answers: which group does this point belong to? PCA (Principal Component Analysis) answers a different question: can I represent this data with fewer features while preserving most of the information? You have a customer with 50 features — PCA finds the 5 most informative directions in that 50-dimensional space and projects the customer onto them. The result: 5 numbers instead of 50, capturing 95% of the original information. PCA and K-Means are often used together — PCA first to reduce dimensions, K-Means after to find groups in the reduced space.
Turn 100 features into 10 without losing most of the information. Explained variance, scree plots, and when PCA helps and when it hurts.
🎯 Key Takeaways
- ✓K-Means is unsupervised — no labels required. It discovers hidden groups by iterating: assign each point to the nearest centroid, move centroids to the mean of their assigned points, repeat until convergence. The objective minimised is inertia — sum of squared distances to cluster centres.
- ✓Always scale features before K-Means. It is distance-based — an unscaled feature with large values completely dominates the clustering. StandardScaler before KMeans is not optional.
- ✓You must choose k in advance. Use the elbow method (plot inertia vs k, find the bend) combined with silhouette score (ranges from -1 to +1, higher is better) to choose k. There is no single correct answer — use domain knowledge to validate.
- ✓Silhouette score measures how well each point fits its cluster vs the nearest other cluster. A mean score above 0.5 indicates good clustering. Scores below 0.2 suggest overlapping or poorly separated clusters.
- ✓K-Means fails on non-spherical clusters (rings, crescents), clusters of very different sizes, and data with significant outliers. Alternatives: DBSCAN for arbitrary shapes, Gaussian Mixture Models for different sizes, K-Medoids for outlier robustness.
- ✓For datasets above 100,000 rows, use MiniBatchKMeans instead of KMeans. It processes random mini-batches rather than the full dataset at each iteration — typically 10× faster with nearly identical clustering quality.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.