Assignment-4

Data analysis case study using R for readily available data set using any one machine learning
Algorithm

Broadly, there are 3 types of Machine Learning Algorithms..

1. Supervised Learning

How it works: This algorithm consist of a target / outcome variable (or dependent variable) which is to
be predicted from a given set of predictors (independent variables). Using these set of variables,
we generate a function that map inputs to desired outputs. The training process continues until the
model achieves a desired level of accuracy on the training data. Examples of Supervised Learning:
Regression, Decision Tree, Random Forest, KNN, Logistic Regression etc.

2. Unsupervised Learning

How it works: In this algorithm, we do not have any target or outcome variable to predict / estimate. It
is used for clustering population in different groups, which is widely used for segmenting customers in
different groups for specific intervention. Examples of Unsupervised Learning: Apriori algorithm, K-
means.

3. Reinforcement Learning:

How it works: Using this algorithm, the machine is trained to make specific decisions. It works this way:
the machine is exposed to an environment where it trains itself continually using trial and error. This
machine learns from past experience and tries to capture the best possible knowledge to make accurate
business decisions. Example of Reinforcement Learning: Markov Decision Process

List of Common Machine Learning Algorithms

Here is the list of commonly used machine learning algorithms. These algorithms can be applied to
almost any data problem:

1. Linear Regression

2. Logistic Regression

3. Decision Tree

4. SVM

5. Naive Bayes

6. KNN

7. K-Means

8. Random Forest

9. Dimensionality Reduction Algorithms

1. Gradient Boosting algorithms GBM XGBoost LightGBM CatBoost

The process of a machine learning project may not be linear, but there are a number of well-known
steps:

 Define Problem.
 Prepare Data.
 Evaluate Algorithms.
 Improve Results.
 Present Result

Hello World of Machine Learning

The best small project to start with on a new tool is the classification of iris flowers (e.g. the iris
dataset).
This is a good project because it is so well understood.
 Attributes are numeric so you have to figure out how to load and handle data.
 It is a classification problem, allowing you to practice with perhaps an easier type of
supervised learning algorithms.
 It is a mutli-class classification problem (multi-nominal) that may require some specialized
handling.
 It only has 4 attribute and 150 rows, meaning it is small and easily fits into memory (and a
screen or A4 page).
 All of the numeric attributes are in the same units and the same scale not requiring any
special scaling or transforms to get started.

Let’s get started with your hello world machine learning project in R.

Here is an overview what we are going to cover:

 Installing the R platform.

 Loading the dataset.
 Summarizing the dataset.
 Visualizing the dataset.
 Evaluating some algorithms.
 Making some predictions.

The caret package provides a consistent interface into hundreds of machine learning algorithms and
provides useful convenience methods for data visualization, data resampling, model tuning and model
comparison, among other features. It’s a must have tool for machine learning projects in R.
2. Load The Data
We are going to use the iris flowers dataset. This dataset is famous because it is used as the “hello
world” dataset in machine learning and statistics by pretty much everyone.
The dataset contains 150 observations of iris flowers. There are four columns of measurements of the
flowers in centimeters. The fifth column is the species of the flower observed. All observed flowers
belong to one of three species.
You can learn more about this dataset on Wikipedia.
Here is what we are going to do in this step:
1. Load the iris data the easy way.
2. Load the iris data from CSV (optional, for purists).
3. Separate the data into a training dataset and a validation dataset.
Choose your preferred way to load data or try both methods.
2.1 Load Data The Easy Way
Fortunately, the R platform provides the iris dataset for us. Load the dataset as follows:

# attach the iris dataset to the environment

> data(iris)
# rename the dataset
> dataset <- iris

You now have the iris data loaded in R and accessible via the dataset variable.
I like to name the loaded data “dataset”. This is helpful if you want to copy-paste code between
projects and the dataset always has the same name.

2.2 Load From CSV

Maybe your a purist and you want to load the data just like you would on your own machine
learning project, from a CSV file.
1. Download the iris dataset from the UCI Machine Learning Repository (here is the direct
link).
2. Save the file as iris.csv your project directory.
Load the dataset from the CSV file as follows:

> dataset <- iris

> filename <- "iris.csv"
> dataset <- read.csv('D:/iris.csv', header=FALSE)
# set the column names in the dataset
> colnames(dataset) <-
c("Sepal.Length","Sepal.Width","Petal.Length","Petal.Width","Species")

You now have the iris data loaded in R and accessible via the dataset variable.

2.3. Create a Validation Dataset

We need to know that the model we created is any good.

Later, we will use statistical methods to estimate the accuracy of the models that we create on
unseen data. We also want a more concrete estimate of the accuracy of the best model on
unseen data by evaluating it on actual unseen data.
That is, we are going to hold back some data that the algorithms will not get to see and we will
use this data to get a second and independent idea of how accurate the best model might
actually be.

We will split the loaded dataset into two, 80% of which we will use to train our models and 20%
that we will hold back as a validation dataset.
# create a list of 80% of the rows in the original dataset we can use for training

> validation_index <- createDataPartition(dataset$Species, p=0.80,

list=FALSE)
# select 20% of the data for validation
> validation <- dataset[-validation_index,]
# use the remaining 80% of data to training and testing the models
> dataset <- dataset[validation_index,]
You now have training data in the dataset variable and a validation set we will use later in
the validation variable.
Note that we replaced our dataset variable with the 80% sample of the dataset. This was an
attempt to keep the rest of the code simpler and readable.

3. Summarize Dataset

Now it is time to take a look at the data.

In this step we are going to take a look at the data a few different ways:

1. Dimensions of the dataset.

2. Types of the attributes.

3. Peek at the data itself.

4. Levels of the class attribute.

5. Breakdown of the instances in each class.

6. Statistical summary of all attributes.

Don’t worry, each look at the data is one command. These are useful commands that you can
use again and again on future projects.

3.1 Dimensions of Dataset

We can get a quick idea of how many instances (rows) and how many attributes (columns) the
data contains with the dim function.

# dimensions of dataset

> dim(dataset)
[1] 120 5

You should see 120 instances and 5 attributes:

3.2 Types of Attributes

It is a good idea to get an idea of the types of the attributes. They could be doubles, integers,
strings, factors and other types.

Knowing the types is important as it will give you an idea of how to better summarize the data
you have and the types of transforms you might need to use to prepare the data before you
model it.

# list types for each attribute

> sapply(dataset, class)

Sepal.Length Sepal.Width Petal.Length Petal.Width Species
"numeric" "numeric" "numeric" "numeric" "factor"
You should see that all of the inputs are double and that the class value is a factor:

3.3 Peek at the Data

It is also always a good idea to actually eyeball your data.

# take a peek at the first 5 rows of the data

> head(dataset)
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 Iris-setosa
2 4.9 3.0 1.4 0.2 Iris-setosa
5 5.0 3.6 1.4 0.2 Iris-setosa
6 5.4 3.9 1.7 0.4 Iris-setosa
9 4.4 2.9 1.4 0.2 Iris-setosa
10 4.9 3.1 1.5 0.1 Iris-setosa
3.4 Levels of the Class

The class variable is a factor. A factor is a class that has multiple class labels or levels.
Let’s look at the levels:

# list the levels for the class

> levels(dataset$Species)

[1] "Iris-setosa" "Iris-versicolor" "Iris-virginica"

Notice above how we can refer to an attribute by name as a property of the dataset. In the
results we can see that the class has 3 different labels:

This is a multi-class or a multinomial classification problem. If there were two levels, it would
be a binary classification problem.

3.5 Class Distribution

Let’s now take a look at the number of instances (rows) that belong to each class. We can
view this as an absolute count and as a percentage.

# summarize the class distribution

> percentage <- prop.table(table(dataset$Species)) * 100

> cbind(freq=table(dataset$Species), percentage=percentage)
freq percentage
Iris-setosa 40 33.33333
Iris-versicolor 40 33.33333
Iris-virginica 40 33.33333
We can see that each class has the same number of instances (40 or 33% of the dataset)

3.6 Statistical Summary

Now finally, we can take a look at a summary of each attribute.

 This includes the mean, the min and max values as well as some percentiles (25th, 50th or
media and 75th e.g. values at this points if we ordered all the values for an attribute).

# summarize attribute distributions

> summary(dataset)
Sepal.Length Sepal.Width Petal.Length Petal.Width
Min. :4.400 Min. :2.000 Min. :1.000 Min. :0.100
1st Qu.:5.100 1st Qu.:2.800 1st Qu.:1.600 1st Qu.:0.300
Median :5.800 Median :3.000 Median :4.350 Median :1.300
Mean :5.854 Mean :3.047 Mean :3.768 Mean :1.192
3rd Qu.:6.425 3rd Qu.:3.300 3rd Qu.:5.100 3rd Qu.:1.800
Max. :7.900 Max. :4.200 Max. :6.900 Max. :2.500
Species
Iris-setosa :40
Iris-versicolor:40
Iris-virginica :40
We can see that all of the numerical values have the same scale (centimeters) and similar
ranges [0,8] centimeters.

4. Visualize Dataset
We now have a basic idea about the data. We need to extend that with some visualizations.

We are going to look at two types of plots:

1. Univariate plots to better understand each attribute.

2. Multivariate plots to better understand the relationships between attributes.
4.1 Univariate Plots

We start with some univariate plots, that is, plots of each individual variable.

It is helpful with visualization to have a way to refer to just the input attributes and just the
output attributes. Let’s set that up and call the inputs attributes x and the output attribute (or
class) y.

# split input and output

> x <- dataset[,1:4]

> y <- dataset[,5]
Given that the input variables are numeric, we can create box and whisker plots of each.

Box plot for each Attribute:-

> par(mfrow=c(1,4))
> for(i in 1:4) {
+ boxplot(x[,i], main=names(iris)[i])
+ }
This gives us a much clearer idea of the distribution of the input attributes:
# barplot for class breakdown

> plot(y)
4.2 Multivariate Plots

Now we can look at the interactions between the variables.

First let’s look at scatterplots of all pairs of attributes and color the points by class. In
addition, because the scatterplots show that points for each class are generally separate,
we can draw ellipses around them.

# scatterplot matrix

> featurePlot(x=x, y=y, plot="ellipse")

We can see some clear relationships between the input attributes (trends) and between
attributes and the class values (ellipses):

We can also look at box and whisker plots of each input variable again, but this time broken
down into separate plots for each class. This can help to tease out obvious linear
separations between the classes.

# box and whisker plots for each attribute

> featurePlot(x=x, y=y, plot="box")

This is useful to see that there are clearly different distributions of the attributes for each
class value.

Next we can get an idea of the distribution of each attribute, again like the box and whisker
plots, broken down by class value. Sometimes histograms are good for this, but in this case
we will use some probability density plots to give nice smooth lines for each distribution.

This confirms what we learned in the last section, that the instances are evenly distributed
across the three class:

Density Plot:-

> scales <- list(x=list(relation="free"), y=list(relation="free"))

> featurePlot(x=x, y=y, plot="density", scales=scales)
Like he boxplots, we can see the difference in distribution of each attribute by class
value. We can also see the Gaussian-like distribution (bell curve) of each attribute.

5. Evaluate Some Algorithms

Now it is time to create some models of the data and estimate their accuracy on unseen
data.
Here is what we are going to cover in this step:

1. Set-up the test harness to use 10-fold cross validation.

2. Build 5 different models to predict species from flower measurements
3. Select the best model.

5.1 Test Harness

We will 10-fold crossvalidation to estimate accuracy.

This will split our dataset into 10 parts, train in 9 and test on 1 and release for all
combinations of train-test splits. We will also repeat the process 3 times for each algorithm
with different splits of the data into 10 groups, in an effort to get a more accurate estimate.

# run the Algorithm 10-fold crossvalidation

> control <- trainControl(method="cv", number=10)

> metric <- "Accuracy"

We are using the metric of “Accuracy” to evaluate models. This is a ratio of the number of
correctly predicted instances in divided by the total number of instances in the dataset
multiplied by 100 to give a percentage (e.g. 95% accurate). We will be using
the metricvariable when we run build and evaluate each model next.

5.2 Build Models

We don’t know which algorithms would be good on this problem or what configurations to
use. We get an idea from the plots that some of the classes are partially linearly separable
in some dimensions, so we are expecting generally good results.

Let’s evaluate 5 different algorithms:

 Linear Discriminant Analysis (LDA)

 Classification and Regression Trees (CART).
 k-Nearest Neighbors (kNN).
 Support Vector Machines (SVM) with a linear kernel.
 Random Forest (RF)

This is a good mixture of simple linear (LDA), nonlinear (CART, kNN) and complex
nonlinear methods (SVM, RF). We reset the random number seed before reach run to
ensure that the evaluation of each algorithm is performed using exactly the same data
splits. It ensures the results are directly comparable.

Let’s build our five models:

# a) linear algorithms

> set.seed(7)

> fit.lda <- train(Species~., data=dataset, method="lda", metric=metric,

trControl=control)
# b) nonlinear algorithms

> set.seed(7)
> fit.cart <- train(Species~., data=dataset, method="rpart", metric=metric,
trControl=control)

# kNN
> set.seed(7)
> fit.knn <- train(Species~., data=dataset, method="knn", metric=metric,
trControl=control)

# c) advanced algorithms
# SVM
> set.seed(7)
> fit.svm <- train(Species~., data=dataset, method="svmRadial",
metric=metric, trControl=control)

Attaching package: ‘kernlab’

The following object is masked from ‘package:ggplot2’:

alpha

#Random Forest
> set.seed(7)
> fit.rf <- train(Species~., data=dataset, method="rf", metric=metric,
trControl=control)

randomForest 4.6-12
Type rfNews() to see new features/changes/bug fixes.

Attaching package: ‘randomForest’

The following object is masked from ‘package:ggplot2’:

margin

5.3 Select Best Model

We now have 5 models and accuracy estimations for each. We need to compare the
models to each other and select the most accurate.

We can report on the accuracy of each model by first creating a list of the created models
and using the summary function

#Summarize the Accuracy of Model.

> results <- resamples(list(lda=fit.lda, cart=fit.cart, knn=fit.knn,

svm=fit.svm, rf=fit.rf))
> summary(results)

Call:
summary.resamples(object = results)

Models: lda, cart, knn, svm, rf

Number of resamples: 10

Accuracy
Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
lda 0.8333333 1.0000000 1.0000000 0.9750000 1.0000000 1 0
cart 0.8333333 0.8541667 0.9166667 0.9166667 0.9791667 1 0
knn 0.8333333 0.9166667 1.0000000 0.9583333 1.0000000 1 0
svm 0.7500000 0.9166667 0.9583333 0.9333333 1.0000000 1 0
rf 0.8333333 0.9166667 0.9583333 0.9416667 1.0000000 1 0

Kappa
Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
lda 0.750 1.00000 1.0000 0.9625 1.00000 1 0
cart 0.750 0.78125 0.8750 0.8750 0.96875 1 0
knn 0.750 0.87500 1.0000 0.9375 1.00000 1 0
svm 0.625 0.87500 0.9375 0.9000 1.00000 1 0
rf 0.750 0.87500 0.9375 0.9125 1.00000 1 0
We can see the accuracy of each classifier and also other metrics like Kappa:

We can also create a plot of the model evaluation results and compare the spread and the
mean accuracy of each model. There is a population of accuracy measures for each
algorithm because each algorithm was evaluated 10 times (10 fold cross validation).

# compare accuracy of models

> dotplot(results)

We can see that the most accurate model in this case was LDA:

The results for just the LDA model can be summarized.

# summarize Best Model

> print(fit.lda)
Linear Discriminant Analysis

120 samples
4 predictor
3 classes: 'Iris-setosa', 'Iris-versicolor', 'Iris-virginica'

No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 108, 108, 108, 108, 108, 108, ...
Resampling results:

Accuracy Kappa
0.975 0.9625
This gives a nice summary of what was used to train the model and the mean and standard
deviation (SD) accuracy achieved, specifically 97.5% accuracy +/- 4%

6. Make Predictions
The LDA was the most accurate model. Now we want to get an idea of the accuracy of the
model on our validation set.

This will give us an independent final check on the accuracy of the best model. It is valuable
to keep a validation set just in case you made a slip during such as overfitting to the training
set or a data leak. Both will result in an overly optimistic result.

We can run the LDA model directly on the validation set and summarize the results in a
confusion matrix.
# estimate Skill of LDA on validation of dataset

> predictions <- predict(fit.lda, validation)

> confusionMatrix(predictions, validation$Species)
Confusion Matrix and Statistics

Reference
Prediction Iris-setosa Iris-versicolor Iris-virginica
Iris-setosa 10 0 0
Iris-versicolor 0 10 0
Iris-virginica 0 0 10

Overall Statistics

Accuracy : 1
95% CI : (0.8843, 1)
No Information Rate : 0.3333
P-Value [Acc > NIR] : 4.857e-15

Kappa : 1
Mcnemar's Test P-Value : NA

Statistics by Class:

Class: Iris-setosa Class: Iris-versicolor

Sensitivity 1.0000 1.0000
Specificity 1.0000 1.0000
Pos Pred Value 1.0000 1.0000
Neg Pred Value 1.0000 1.0000
Prevalence 0.3333 0.3333
Detection Rate 0.3333 0.3333
Detection Prevalence 0.3333 0.3333
Balanced Accuracy 1.0000 1.0000
Class: Iris-virginica
Sensitivity 1.0000
Specificity 1.0000
Pos Pred Value 1.0000
Neg Pred Value 1.0000
Prevalence 0.3333
Detection Rate 0.3333
Detection Prevalence 0.3333
Balanced Accuracy 1.0000
Finally We can see that the accuracy is 100%. It was a small validation dataset (20%), but
this result is within our expected margin of 97% +/-4% suggesting we may have an accurate
and a reliably accurate model.

References:- https://machinelearningmastery.com/machine-learning-in-r-step-by-step/

Please read following Article related different machine learning Algorithm

1. Support Vector Machine(SVM)

https://www.analyticsvidhya.com/blog/2017/09/understaing-support-vector-machine-
example-code/

2. https://www.analyticsvidhya.com/blog/2017/09/common-machine-learning-algorithms/

**************************************THE END************************************************