K-Means Clustering

By the end of this lesson, you will understand how K-Means clustering partitions data into groups based on distance to centroids and implement it using Python.

What it is

K-Means is an unsupervised machine learning algorithm used to group unlabeled data into k distinct clusters. The core mental model involves placing k points (centroids) in space and iteratively moving them to the center of nearby data points until stability is reached. Related terms include "inertia" (sum of squared distances within clusters), "centroid," and "feature scaling."

Why it matters

• Customer Segmentation: Grouping users by purchasing behavior without predefined labels.
• Anomaly Detection: Identifying data points far from any centroid as potential outliers.
• Image Compression: Reducing color palettes by clustering similar pixel values.
• Document Clustering: Organizing text documents by topic similarity.

Syntax or steps

1. Initialize: Randomly select k initial centroids.
2. Assign: Assign each data point to the nearest centroid.
3. Update: Recalculate centroids as the mean of assigned points.
4. Iterate: Repeat assignment and update until centroids stop moving significantly.

Example

import numpy as np
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

# Generate synthetic data: 300 samples, 2 features
np.random.seed(42)
data = np.random.randn(300, 2) * 10 + [5, 5] # Shifted cluster
data2 = np.random.randn(300, 2) * 10 - [5, 5] # Opposite cluster
X = np.vstack((data, data2))

# Initialize KMeans with k=2
kmeans = KMeans(n_clusters=2, random_state=42, n_init=10)
kmeans.fit(X)

# Predict cluster labels for new data
labels = kmeans.labels_
centroids = kmeans.cluster_centers_

# Visualize results
plt.scatter(X[:, 0], X[:, 1], c=labels, cmap='viridis', alpha=0.6)
plt.scatter(centroids[:, 0], centroids[:, 1], c='red', marker='x', s=200, linewidths=3)
plt.title("K-Means Clustering Result")
plt.show()

This code generates two distinct blobs of data. KMeans(n_clusters=2) creates the model object. fit(X) trains the model by finding optimal centroids. labels_ stores the cluster index for each training point, while cluster_centers_ holds the final coordinates of the centroids.

Common mistakes

• Ignoring Feature Scaling: K-Means uses Euclidean distance. If one feature ranges 0-1000 and another 0-1, the larger range dominates. Always standardize data first.
• Choosing Poor k: Selecting k arbitrarily leads to meaningless groups. Use the Elbow Method or Silhouette Score to determine optimal k.
• Assuming Spherical Clusters: K-Means fails on non-convex shapes (e.g., moons or rings). It assumes clusters are isotropic and convex.
• Local Minima: Random initialization can trap the algorithm in suboptimal solutions. Use n_init > 1 to run multiple times and pick the best result.

When to use it

Use K-Means when you expect compact, well-separated groups and need speed. Use DBSCAN when cluster shapes are irregular or you do not know the number of clusters.

Practice

Guided Exercise: Modify the example above to set n_clusters=3. Observe how the red 'x' markers move and how the colors change.

Challenge: Apply StandardScaler from sklearn.preprocessing to your data before fitting K-Means. Compare the inertia score (kmeans.inertia_) before and after scaling. Why does it change?

Quick check

Question: Why must you choose the number of clusters k before running K-Means?

Answer: K-Means minimizes variance within a fixed number of groups; it cannot automatically discover the natural number of clusters like hierarchical methods or DBSCAN.

Summary

K-Means is a fast, iterative algorithm that partitions data into k spherical clusters by minimizing intra-cluster variance. Success depends heavily on proper feature scaling and selecting an appropriate k value through validation techniques.