← All technical notes

MACHINE LEARNING · CLUSTERING

Understand k-means by Watching the Centroids Move

Assign every point to its nearest centroid. Move each centroid to the group's mean. Watch those two operations turn scattered points into clusters.

Primary sources checked: 31 August 2026

01

Follow the centroids until they settle

Step performs one assignment-and-update cycle. The background shows the region closest to each current centroid.

Iteration
0
Inertia
1,085,095
Points
30

Click the canvas to add points. Switch to the elongated set to expose a shape k-means handles poorly.

02

The objective is the sum of squared distances to assigned centroids

k-means assigns each point to its nearest centroid, then replaces each centroid with the mean of its assigned points. The inertia shown here is the total squared distance from points to those centroids.

Inertia falls as the two steps repeat and eventually the centroids stop moving. The result is still a local optimum: a different initialization can produce a different partition.

03

Good for compact blobs; awkward for stretched or uneven groups

Nearest-centroid boundaries favour convex, similarly sized clusters. Stretched shapes, rings, uneven densities, and outliers can produce a partition that conflicts with the structure a reader sees.

In practice, repeat initializations, inspect scaling, and compare another metric or model such as silhouette analysis, DBSCAN, or a Gaussian mixture. Choosing k is part of the problem, not an input to hide.

04

Write only the core loop

This teaching implementation maps the assignment and mean-update steps directly into Python for 2D points.

Python 3kmeans_from_scratch.py
def kmeans(points, k, iterations=20):
    """A compact teaching implementation for 2D points."""
    centroids = [list(point) for point in points[:k]]

    for _ in range(iterations):
        groups = [[] for _ in range(k)]

        for point in points:
            label = min(
                range(k),
                key=lambda i: squared_distance(point, centroids[i]),
            )
            groups[label].append(point)

        updated = []
        for i, group in enumerate(groups):
            if not group:
                updated.append(centroids[i])
                continue
            updated.append([
                sum(p[0] for p in group) / len(group),
                sum(p[1] for p in group) / len(group),
            ])

        if all(squared_distance(a, b) < 1e-10
               for a, b in zip(centroids, updated)):
            break
        centroids = updated

    return centroids, groups

def squared_distance(a, b):
    return (a[0] - b[0]) ** 2 + (a[1] - b[1]) ** 2

05

Why this is not production k-means

06

Primary sources

  1. scikit-learn User Guide — K-means

    Objective, update loop, initialization sensitivity, and shape assumptions.

  2. scikit-learn API — sklearn.cluster.KMeans

    Current production API parameters and outputs.

  3. Lloyd (1982), Least Squares Quantization in PCM

    Primary publication for the iterative quantization procedure.