K-Means Clustering in Python

K-means is a popular unsupervised machine learning algorithm used for clustering. The goal of K-means is to partition a set of data points into K clusters (groups) such that the points within each cluster are as similar as possible, while the points in different clusters are as different as possible.

Steps in K-Means Algorithm

1. Initialization: Choose K initial cluster centroids randomly.

2. Assign Points: Assign each data point to the nearest centroid.

3. Update Centroids: Recompute the centroids based on the new assignments.

4. Repeat: Repeat steps 2 and 3 until the centroids no longer change or until a stopping criterion is met.

K-Means in Python using Scikit-learn

Python’s scikit-learn library provides an easy-to-use implementation of K-means clustering. Let's go over how to use the KMeans class in scikit-learn.

Step-by-Step Example:

1. Import Required Libraries

import numpy as npimport matplotlib.pyplot as pltfrom sklearn.cluster import KMeans

2. Generate Data

For this example, we'll generate some synthetic data points for clustering.

from sklearn.datasets import make_blobs# Generate synthetic data with 2 features (2D data)X, y = make_blobs(n_samples=300, centers=4, cluster_std=0.60, random_state=0)

• n_samples=300: Number of data points.

• centers=4: Number of clusters to form.

• cluster_std=0.60: Standard deviation of the clusters (spread of the data).

3. Visualize the Data

Let's plot the generated data points.

plt.scatter(X[:, 0], X[:, 1], s=30, cmap='viridis')plt.show()

4. Applying K-Means

Now, we’ll apply the K-means algorithm with K=4 (since we generated data with 4 centers).

kmeans = KMeans(n_clusters=4)kmeans.fit(X)# Get the coordinates of the centroidscentroids = kmeans.cluster_centers_# Get the labels (cluster assignments) for each pointlabels = kmeans.labels_

• n_clusters=4: Specifies the number of clusters.

• .fit(X): Trains the model using the data X.

5. Visualize the Clusters

After applying K-means, we can visualize the data points and their assigned clusters.

# Plot data points and centroidsplt.scatter(X[:, 0], X[:, 1], c=labels, s=30, cmap='viridis')plt.scatter(centroids[:, 0], centroids[:, 1], c='red', s=200, alpha=0.5, marker='X')  # Plot centroidsplt.show()

This will plot the data points colored by their cluster labels, with the centroids shown as red "X"s.

Choosing the Optimal Number of Clusters (K)

One of the challenges in K-means clustering is selecting the number of clusters K. A common method to determine the optimal K is the Elbow Method.

Elbow Method

The idea is to calculate the inertia (the sum of squared distances from each point to its assigned centroid) for different values of K, and look for the "elbow" in the plot where the inertia starts to decrease more slowly.

# Calculate inertia for different values of Kinertia = []k_range = range(1, 11)for k in k_range:    kmeans = KMeans(n_clusters=k)    kmeans.fit(X)    inertia.append(kmeans.inertia_)# Plot the inertia for each Kplt.plot(k_range, inertia, marker='o')plt.xlabel('Number of Clusters (K)')plt.ylabel('Inertia')plt.title('Elbow Method')plt.show()

• Inertia: The sum of squared distances of samples to their closest cluster center.

• The "elbow" point on the graph is typically considered the optimal number of clusters.

Handling K-Means with Scikit-learn

• Initialization of Centroids: The KMeans class in scikit-learn by default uses the k-means++ method for initialization, which helps to spread out the initial centroids.

• Convergence Criteria: The algorithm stops when the centroids do not change significantly between iterations, or when a maximum number of iterations (max_iter) is reached.

Example with Additional Parameters

kmeans = KMeans(n_clusters=4, init='k-means++', max_iter=300, n_init=10, random_state=42)kmeans.fit(X)# Get the cluster labels and centroidslabels = kmeans.labels_centroids = kmeans.cluster_centers_

• init='k-means++': Specifies how the initial centroids are chosen (default is k-means++, which helps avoid poor clustering results).

• max_iter=300: The maximum number of iterations for the algorithm.

• n_init=10: The number of times the algorithm is run with different centroid seeds. The final result is the best output in terms of inertia.

K-Means and Real Data

Let's apply K-means to a real dataset, such as the Iris dataset, which is available in scikit-learn.

from sklearn.datasets import load_irisfrom sklearn.preprocessing import StandardScaler# Load Iris datasetiris = load_iris()X = iris.data# Standardize the featuresscaler = StandardScaler()X_scaled = scaler.fit_transform(X)# Apply KMeans with 3 clusters (since Iris has 3 species)kmeans = KMeans(n_clusters=3)kmeans.fit(X_scaled)# Get labels and centroidslabels = kmeans.labels_centroids = kmeans.cluster_centers_# Visualizing the clusters (only 2 features for simplicity)plt.scatter(X_scaled[:, 0], X_scaled[:, 1], c=labels, cmap='viridis')plt.scatter(centroids[:, 0], centroids[:, 1], c='red', marker='X', s=200)plt.title('K-Means Clustering on Iris Dataset')plt.xlabel('Feature 1')plt.ylabel('Feature 2')plt.show()

Conclusion

K-means clustering is a powerful and widely used algorithm for unsupervised learning, allowing us to group data points into distinct clusters based on their similarity. Python's scikit-learn library provides a simple and efficient way to implement K-means, along with tools to help choose the optimal number of clusters using methods like the Elbow method.

Key points:

• KMeans is the main class for K-means in scikit-learn.

• It’s important to choose the right number of clusters (K), often determined through methods like the Elbow method.

• K-means works best when clusters are spherical and evenly sized, so it may not perform well on non-convex or irregularly shaped clusters.

Let me know if you need more details or further examples!