classification and clustering

What is classification?
• The term "classification" is usually used when there are exactly two
target classes called binary classification.
• When more than two classes may be predicted, specifically in pattern
recognition problems, this is often referred to as multinomial
classification.
• However, multinomial classification is also used for categorical
response data, where one wants to predict which category amongst
several categories has the instances with the highest probability.
• Classification is one of the most important tasks in data mining.
• It refers to a process of assigning pre-defined class labels to instances
based on their attributes.
• There is a similarity between classification and clustering, it looks
similar, but it is different.
• The major difference between classification and clustering is that
classification includes the levelling of items according to their
membership in pre-defined groups.
• Let's understand this concept with the help of an example;
• suppose you are using a self-organizing map neural network
algorithm for image recognition where there are 10 different kinds of
objects. If you label each image with one of these 10 classes, the
classification task is solved.
• On the other hand, clustering does not involve any labelling.
• Assume that you are given an image database of 10 objects and no
class labels.
• Using a clustering algorithm to find groups of similar-looking images
will result in determining clusters without object labels.
Classification of data mining
• These are given some of the important data mining classification
methods:
• Logistic Regression Method
• The logistic Regression Method is used to predict the response
variable.
• K-Nearest Neighbors Method
• K-Nearest Neighbors Method is used to classify the datasets into what
is known as a K observation. It is used to determine the similarities
between the neighbours.
• Naive Bayes Method
The Naive Bayes method is used to scan the set of data and locate the
records wherein the predictor values are equal.
• Neural Networks Method

The Neural Networks resemble the structure of our brain called the
Neuron. The sets of data pass through these networks and finally come
out as output. This neural network method compares the different
classifications. Errors that occur in the classifications are further
rectified and are fed into the networks. This is a recurring process.
What is clustering?
• Clustering refers to a technique of grouping objects so that objects with the same
functionalities come together and objects with different functionalities go apart.

• In other words, we can say that clustering is a process of portioning a data set into a
set of meaningful subclasses, known as clusters.

• Clustering is the same as classification in which data is grouped. Though, unlike

classification, the groups are not previously defined. Instead, the grouping is
achieved by determining similarities between data according to characteristics found
in the real data. The groups are called Clusters.
 Difference between Classification and
Clustering
Classification Clustering
Classification is a supervised learning approach Clustering is an unsupervised learning approach
where a specific label is provided to the machine to where grouping is done on similarities basis.
classify new observations. Here the machine needs
proper testing and training for the label
verification.

Supervised learning approach. Unsupervised learning approach.

It uses a training dataset. It does not use a training dataset.
It uses algorithms to categorize the new data as per It uses statistical concepts in which the data set is
the observations of the training set. divided into subsets with the same features.
In classification, there are labels for training data. In clustering, there are no labels for training data.
Its objective is to find which class a new object Its objective is to group a set of objects to find
belongs to form the set of predefined classes. whether there is any relationship between them.

It is more complex as compared to clustering. It is less complex as compared to clustering.

Clustering in Machine Learning
• Clustering or cluster analysis is a machine learning technique, which
groups the unlabelled dataset. It can be defined as "A way of grouping
the data points into different clusters, consisting of similar data
points. The objects with the possible similarities remain in a group
that has less or no similarities with another group."
• It does it by finding some similar patterns in the unlabelled dataset
such as shape, size, color, behavior, etc., and divides them as per the
presence and absence of those similar patterns.
• It is an unsupervised learning method, hence no supervision is
provided to the algorithm, and it deals with the unlabeled dataset.
• After applying this clustering technique, each cluster or group is
provided with a cluster-ID. ML system can use this id to simplify the
processing of large and complex datasets.
Example:
• Let's understand the clustering technique with the real-world example
of Mall: When we visit any shopping mall, we can observe that the
things with similar usage are grouped together. Such as the t-shirts are
grouped in one section, and trousers are at other sections, similarly, at
vegetable sections, apples, bananas, Mangoes, etc., are grouped in
separate sections, so that we can easily find out the things. The
clustering technique also works in the same way. Other examples of
clustering are grouping documents according to the topic.
• The clustering technique can be widely used in various tasks. Some
most common uses of this technique are:
• Market Segmentation

• Statistical data analysis

• Social network analysis

• Image segmentation

• Anomaly detection, etc.

Methods of clustering
• Partitioning methods
• Density-based clustering
• Hierarchical clustering
• Fuzzy Clustering
• Model-based clustering
Partitioning Clustering
• It is a type of clustering that divides the data into non-hierarchical
groups. It is also known as the centroid-based method. The most
common example of partitioning clustering is the K-Means
Clustering algorithm.
• In this type, the dataset is divided into a set of k groups, where K is
used to define the number of pre-defined groups. The cluster center is
created in such a way that the distance between the data points of one
cluster is minimum as compared to another cluster centroid.
Partitioning Clustering
Density-Based Clustering
• The density-based clustering method connects the highly-dense areas
into clusters, and the arbitrarily shaped distributions are formed as
long as the dense region can be connected. This algorithm does it by
identifying different clusters in the dataset and connects the areas of
high densities into clusters. The dense areas in data space are divided
from each other by sparser areas.
• These algorithms can face difficulty in clustering the data points if the
dataset has varying densities and high dimensions
Density-Based Clustering
Distribution Model-Based Clustering
• In the distribution model-based clustering method, the data is divided
based on the probability of how a dataset belongs to a particular
distribution. The grouping is done by assuming some distributions
commonly Gaussian Distribution.
Hierarchical Clustering
• Hierarchical clustering can be used as an alternative for the partitioned
clustering as there is no requirement of pre-specifying the number of
clusters to be created. In this technique, the dataset is divided into
clusters to create a tree-like structure, which is also called
a dendrogram. The observations or any number of clusters can be
selected by cutting the tree at the correct level. The most common
example of this method is the Agglomerative Hierarchical algorithm
Fuzzy Clustering
• Fuzzy clustering is a type of soft method in which a data object may
belong to more than one group or cluster. Each dataset has a set of
membership coefficients, which depend on the degree of membership
to be in a cluster. Fuzzy C-means algorithm is the example of this
type of clustering; it is sometimes also known as the Fuzzy k-means
algorithm
Clustering Algorithms
• The Clustering algorithms can be divided based on their models that
are explained above. There are different types of clustering algorithms
published, but only a few are commonly used. The clustering
algorithm is based on the kind of data that we are using. Such as, some
algorithms need to guess the number of clusters in the given dataset,
whereas some are required to find the minimum distance between the
observation of the dataset.
1.K-Means algorithm: The k-means algorithm is one of the most
popular clustering algorithms. It classifies the dataset by dividing the
samples into different clusters of equal variances. The number of
clusters must be specified in this algorithm. It is fast with fewer
computations required, with the linear complexity of O(n).
2.Mean-shift algorithm: Mean-shift algorithm tries to find the dense
areas in the smooth density of data points. It is an example of a
centroid-based model, that works on updating the candidates for
centroid to be the center of the points within a given region.
3.DBSCAN Algorithm: It stands for Density-Based Spatial
Clustering of Applications with Noise. It is an example of a density-
based model similar to the mean-shift, but with some remarkable
advantages. In this algorithm, the areas of high density are separated by
the areas of low density. Because of this, the clusters can be found in
any arbitrary shape.
4. Expectation-Maximization Clustering using GMM: This
algorithm can be used as an alternative for the k-means algorithm or for
those cases where K-means can be failed. In GMM, it is assumed that
the data points are Gaussian distributed.
5.Agglomerative Hierarchical algorithm: The Agglomerative
hierarchical algorithm performs the bottom-up hierarchical clustering.
In this, each data point is treated as a single cluster at the outset and then
successively merged. The cluster hierarchy can be represented as a tree-
structure.
6.Affinity Propagation: It is different from other clustering algorithms
as it does not require to specify the number of clusters. In this, each data
point sends a message between the pair of data points until convergence.
It has O(N2T) time complexity, which is the main drawback of this
algorithm.
Applications of Clustering
• In Identification of Cancer Cells: The clustering algorithms are
widely used for the identification of cancerous cells. It divides the
cancerous and non-cancerous data sets into different groups.
• In Search Engines: Search engines also work on the clustering
technique. The search result appears based on the closest object to the
search query. It does it by grouping similar data objects in one group
that is far from the other dissimilar objects. The accurate result of a
query depends on the quality of the clustering algorithm used.
Applications of Clustering
• Customer Segmentation: It is used in market research to segment the
customers based on their choice and preferences.
• In Biology: It is used in the biology stream to classify different
species of plants and animals using the image recognition technique.
• In Land Use: The clustering technique is used in identifying the area
of similar lands use in the GIS database. This can be very useful to
find that for what purpose the particular land should be used, that
means for which purpose it is more suitable.