9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

Clustering

k-means Clustering
https://en.wikipedia.org/wiki/K-means_clustering (https://en.wikipedia.org/wiki/K-means_clustering)

UCI Machine Learning Repository

https://archive.ics.uci.edu/ml/datasets.php
(https://archive.ics.uci.edu/ml/datasets.php)

In [1]: import pandas as pd

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from matplotlib import style

In [2]: style.use('default')

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 1/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [3]: iris_df = sns.load_dataset('iris')

iris_df

Out[3]: sepal_length sepal_width petal_length petal_width species

0 5.1 3.5 1.4 0.2 setosa

1 4.9 3.0 1.4 0.2 setosa

2 4.7 3.2 1.3 0.2 setosa

3 4.6 3.1 1.5 0.2 setosa

4 5.0 3.6 1.4 0.2 setosa

... ... ... ... ... ...

145 6.7 3.0 5.2 2.3 virginica

146 6.3 2.5 5.0 1.9 virginica

147 6.5 3.0 5.2 2.0 virginica

148 6.2 3.4 5.4 2.3 virginica

149 5.9 3.0 5.1 1.8 virginica

150 rows × 5 columns

In [4]: iris_df['species'].value_counts()

Out[4]: virginica 50

setosa 50

versicolor 50

Name: species, dtype: int64

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 2/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [5]: plt.figure(figsize = (7,7))

sns.scatterplot(data = iris_df, x = 'sepal_length', y = 'sepal_width', hue = 'spe

Out[5]: <AxesSubplot:xlabel='sepal_length', ylabel='sepal_width'>

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 3/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [6]: plt.figure(figsize = (7,7))

sns.scatterplot(data = iris_df, x = 'sepal_length', y = 'petal_width', hue = 'spe

Out[6]: <AxesSubplot:xlabel='sepal_length', ylabel='petal_width'>

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 4/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [7]: plt.figure(figsize = (5,5))

sns.scatterplot(data = iris_df, x = 'petal_length', y = 'petal_width', hue = 'spe

Out[7]: <AxesSubplot:xlabel='petal_length', ylabel='petal_width'>

Raw Coding the k Means Clustering Algorithm

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 5/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [8]: from sklearn.metrics import pairwise_distances_argmin

​
def find_clusters(X, n_clusters, rseed=0, num_iter = 100):
# 1. Randomly choose clusters
rng = np.random.RandomState(rseed)
i = rng.permutation(X.shape[0])[:n_clusters]
centers = X[i]

iter = 1
while True:
# 2a. Assign labels based on closest center
labels = pairwise_distances_argmin(X, centers)

# 2b. Find new centers from means of points

new_centers = np.array([X[labels == i].mean(0)
for i in range(n_clusters)])

# 2c. Check for convergence

print(num_iter, iter)
iter +=1
if iter > num_iter:
break

if np.all(centers == new_centers):
break

centers = new_centers

return centers, labels

​
X = iris_df.iloc[:, :-1].to_numpy()
centers = []
labels = []
for i in [1, 2, 5, 10]:
out_center, out_label = find_clusters(X, 3, num_iter = i, rseed=0)
centers.append(out_center)
labels.append(out_label)
​

1 1

2 1

2 2

5 1

5 2

5 3

5 4

5 5

10 1

10 2

10 3

10 4

10 5

10 6

10 7

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 6/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

10 8

10 9

In [9]: iris_df

Out[9]: sepal_length sepal_width petal_length petal_width species

0 5.1 3.5 1.4 0.2 setosa

1 4.9 3.0 1.4 0.2 setosa

2 4.7 3.2 1.3 0.2 setosa

3 4.6 3.1 1.5 0.2 setosa

4 5.0 3.6 1.4 0.2 setosa

... ... ... ... ... ...

145 6.7 3.0 5.2 2.3 virginica

146 6.3 2.5 5.0 1.9 virginica

147 6.5 3.0 5.2 2.0 virginica

148 6.2 3.4 5.4 2.3 virginica

149 5.9 3.0 5.1 1.8 virginica

150 rows × 5 columns

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 7/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [10]: import matplotlib.gridspec as gridspec

fig2 = plt.figure(constrained_layout=True, figsize = (7,7))
spec2 = gridspec.GridSpec(ncols=2, nrows=2, figure=fig2)
f2_ax1 = fig2.add_subplot(spec2[0, 0])
sns.scatterplot(ax = f2_ax1, x = iris_df['sepal_length'], y = iris_df['petal_widt
f2_ax1.scatter(centers[0][:, 0], centers[0][:, -1], marker = '*', color = 'royalb
f2_ax1.set_title('Number of Iterations = 1')
​
f2_ax2 = fig2.add_subplot(spec2[0, 1])
sns.scatterplot(ax = f2_ax2, x = iris_df['sepal_length'], y = iris_df['petal_widt
f2_ax2.scatter(centers[1][:, 0], centers[1][:, -1], marker = '*', color = 'royalb
f2_ax2.set_title('Number of Iterations = 2')
​
f2_ax3 = fig2.add_subplot(spec2[1, 0])
sns.scatterplot(ax = f2_ax3, x = iris_df['sepal_length'], y = iris_df['petal_widt
f2_ax3.scatter(centers[2][:, 0], centers[2][:, -1], marker = '*', color = 'royalb
f2_ax3.set_title('Number of Iterations = 5')
​
f2_ax4 = fig2.add_subplot(spec2[1, 1])
sns.scatterplot(ax = f2_ax4, x = iris_df['sepal_length'], y = iris_df['petal_widt
f2_ax4.scatter(centers[3][:, 0], centers[3][:, -1], marker = '*', color = 'royalb
f2_ax4.set_title('Number of Iterations = 10')
fig2.suptitle('Clustering at various iterations')
plt.show()

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 8/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

Using sklearn

https://scikit-
learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans
(https://scikit-
learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans)

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 9/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [11]: from sklearn.cluster import KMeans

kmc = KMeans(n_clusters=3, max_iter=600, algorithm = 'full')
X = iris_df.iloc[:, :-1]
kmc.fit(X)
​

C:\ProgramData\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:882: User
Warning: KMeans is known to have a memory leak on Windows with MKL, when there
are less chunks than available threads. You can avoid it by setting the environ
ment variable OMP_NUM_THREADS=1.

f"KMeans is known to have a memory leak on Windows "

Out[11]: KMeans(algorithm='full', max_iter=600, n_clusters=3)

In [12]: kmc.cluster_centers_

Out[12]: array([[5.006 , 3.428 , 1.462 , 0.246 ],

[5.9016129 , 2.7483871 , 4.39354839, 1.43387097],

[6.85 , 3.07368421, 5.74210526, 2.07105263]])

In [13]: kmc.labels_

Out[13]: array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,

1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,

1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 2, 2, 2, 1, 2, 2, 2,

2, 2, 2, 1, 1, 2, 2, 2, 2, 1, 2, 1, 2, 1, 2, 2, 1, 1, 2, 2, 2, 2,

2, 1, 2, 2, 2, 2, 1, 2, 2, 2, 1, 2, 2, 2, 1, 2, 2, 1])

In [14]: pd.crosstab(kmc.labels_, iris_df['species'])

Out[14]: species setosa versicolor virginica

row_0

0 50 0 0

1 0 48 14

2 0 2 36

Metrics

https://scikit-learn.org/stable/modules/clustering.html#clustering-evaluation (https://scikit-
learn.org/stable/modules/clustering.html#clustering-evaluation)

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 10/11

9/8/2021 Completed 012 2021-08-23 to 2021-09-10 Part C Clustering - Jupyter Notebook

In [15]: from sklearn.metrics import silhouette_score

​
cluster_df = pd.DataFrame(kmc.labels_, columns = ['Cluster ID'])
# cluster_df
​
full_cluster_df = pd.concat([X.reset_index(drop = True), cluster_df], axis = 1)
full_cluster_df
​
silhouette_score(full_cluster_df, kmc.labels_, metric='euclidean')
​

Out[15]: 0.6128676734836785

In [ ]: ​

localhost:8888/notebooks/Completed 012 2021-08-23 to 2021-09-10 Part C Clustering.ipynb 11/11