D.N.R.

COLLEGE OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

REG No : ………………………….

LAB NAME : MACHINE LEARNING WITH PYTHON PROGRAMMING LAB

YEAR : I Year II Semester

STUDENT NAME : ………………………………………..……

D.N.R. COLLEGE OF ENGINEERING AND TECHNOLOGY

(APPROVED BY AICTE, AFFILIATED TO JNTU KAKINADA & GOVT. OF A.P)

Balusumudi, Bhimavaram – 534 202, W. G. Dist, Andhra Pradesh

Phone : 08816-221237, Fax : 08816-221236

Website : www.dnrcet.org
 D.N.R. COLLEGE OF ENGINEERING AND TECHNOLOGY
(APPROVED BY AICTE, AFFILIATED TO JNTU KAKINADA & GOVT. OF A.P.)

Balusumudi, Bhimavaram – 534 202, W. G. Dist, Andhra Pradesh

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

CERTIFICATE

This is to certify that the practical work done by Mr./Mrs. ……………….……

…………..….…..….… bearing the Reg No: …..…..….……………. is a bonafide student of
D. N. R. College of Engineering & Technology, and has successfully completed the Advanced
Data Structures & Algorithms Lab in partial fulfillment of I Year, II Semester of CSE
Department during the Year 2019-2020.

Number of Experiments Conducted :

Number of Experiments Completed :

External Lab Examination Held on : ….../....../..….

INTERNAL EXAMINER EXTERNAL EXAMINER

HEAD OF THE DEPARTMENT

 INDEX

S Exp PAGE
NAME OF THE EXPERIMENT
No No NO
Experiment-1

Exercises to solve the real-world problems using the following

machine learning methods:
1 1
a) Linear Regression
b) Logistic Regression.

Experiment-2

2 2 Write a program to Implement Support Vector Machines.

Experiment-3

3 3 Write a program to simulate a perception network for pattern

classification and function approximation.

Experiment-4

Write a program to demonstrate the working of the decision tree based

4 4 ID3 algorithm. Use an appropriate data set for building the decision tree
and apply this knowledge to classify a new sample

Experiment-5

Build an Artificial Neural Network by implementing the Back

5 5 propagation algorithm and test the same using appropriate data
sets
Experiment-6

Assuming a set of documents that need to be classified, use the

6 6 naïve Bayesian Classifier model to perform this task. Built-in
Java classes/API can be used to write the program. Calculate the
accuracy, precision, and recall for your data set.
Experiment-7

Write a program to implement k-Nearest Neighbor algorithm to

7 7
classify the iris data set. Print both correct and wrong predictions
MACHINE LEARNING LAB 239P1D5809

Experiment-1
Exercises to solve the real-world problems using the following machine learning methods:
a) Linear Regression
b) Logistic Regression.

A)Linear Regression:

Source Code:

import numpy as np
import matplotlib.pyplot as plt

def estimate_coef(x, y):

# number of observations/points
n = np.size(x)

# mean of x and y vector

m_x, m_y = np.mean(x), np.mean(y)

# calculating cross-deviation and deviation about x

SS_xy = np.sum(y*x) - n*m_y*m_x
SS_xx = np.sum(x*x) - n*m_x*m_x

# calculating regression coefficients

b_1 = SS_xy / SS_xx
b_0 = m_y - b_1*m_x

return(b_0, b_1)

def plot_regression_line(x, y, b):

# plotting the actual points as scatter plot
plt.scatter(x, y, color = "m",
marker = "o", s = 30)

# predicted response vector

y_pred = b[0] + b[1]*x

# plotting the regression line

plt.plot(x, y_pred, color = "g")

# putting labels
plt.xlabel('x')
plt.ylabel('y')

# function to show plot

plt.show()

def main():
# observations
x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
y = np.array([1, 3, 2, 5, 7, 8, 8, 9, 10, 12])

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

# estimating coefficients
b = estimate_coef(x, y)
print("Estimated coefficients:\nb_0 = {} \
\nb_1 = {}".format(b[0], b[1]))

# plotting regression line

plot_regression_line(x, y, b)

if __name__ == "__main__":
main()

Output:

Graph:

B)Logistic Regression:

Step1: Gather the data / dataset.

Data – User_Data

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

Step2: Import Libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

Step3: Build a data frame & Create the Model in Python (In this example
Logistic Regression)

dataset = pd.read_csv('...\\User_Data.csv')

Step 4: Predict using Test Dataset and Check the score

# input

x = dataset.iloc[:, [2, 3]].values

# output

y = dataset.iloc[:, 4].values
Splitting the dataset to train and test. 75% of data is used for training the model and 25% of it is used to
test the performance of our model.
from sklearn.cross_validation import train_test_split
xtrain, xtest, ytrain, ytest = train_test_split(
x, y, test_size = 0.25, random_state = 0)
from sklearn.preprocessing import StandardScaler
sc_x = StandardScaler()
xtrain = sc_x.fit_transform(xtrain)
xtest = sc_x.transform(xtest)

print (xtrain[0:10, :])

Output :

Step 5: Prediction with a New Set of Data and evaluate the accuracy

Confusion Matrix:

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(ytest, y_pred)

print ("Confusion Matrix : \n", cm)

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:
MACHINE LEARNING LAB 239P1D5809

Output :

Performance measure – Accuracy

from sklearn.metrics import accuracy_score
print ("Accuracy : ", accuracy_score(ytest, y_pred))

Output :

Visualizing the performance of our model.

from matplotlib.colors import ListedColormap
X_set, y_set = xtest, ytest
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1,
stop = X_set[:, 0].max() + 1, step = 0.01),
np.arange(start = X_set[:, 1].min() - 1,
stop = X_set[:, 1].max() + 1, step = 0.01))

plt.contourf(X1, X2, classifier.predict(

np.array([X1.ravel(), X2.ravel()]).T).reshape(
X1.shape), alpha = 0.75, cmap = ListedColormap(('red', 'green')))

plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())

for i, j in enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
c = ListedColormap(('red', 'green'))(i), label = j)

plt.title('Classifier (Test set)')

plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()

Output :

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

Experiment-2

Write a program to Implement Support Vector Machines.

Support Vector Machines.

1. Loading The Data

We are using the cancer data-set in the sklearn library, we will make a classifier to predict whether the
cancer is malignant or benign. We can load the data-set in the following manner.
1 from sklearn import datasets
2
3 cancer_data = datasets.load_breast_cancer()
4 print(cancer_data.data[5]

Output:

After this, we will explore the data. Take a look at various values in the data-set. Check the target
variable, etc.

Explore Data

The shape means that this data-set has 569 rows and 30 columns.
1 print(cancer_data.data.shape)
2 #target set
3 print(cancer_data.target)
Output:

In this, 0 represents malignant, and 1 represents benign.

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

Splitting Data
We will divide the data-set into a training set and test set to get accurate results. After this, we will split
the data using the train_test_split() function. We will need 3 parameters like in the example below. The
features to train the model, the target, and the test set size.
from sklearn.model_selection import train_test_split

cancer_data = datasets.load_breast_cancer()

X_train, X_test, y_train, y_test = train_test_split(cancer_data.data, cancer_data.target,

test_size=0.4,random_state=109)
Generating The Model
To generate the model, we will first import the SVM module from sklearn to create a support vector
classifier in svc() by passing the argument kernel as the linear kernel.
Then we will train the data-set using the set() and make predictions using the predict() function.
from sklearn import svm
#create a classifier
cls = svm.SVC(kernel="linear")
#train the model
cls.fit(X_train,y_train)
#predict the response
pred = cls.predict(X_test)
Evaluating the Model
With this, we can predict how accurately the model or classifier can predict if the patient has heart
disease or not. So we will calculate the accuracy score, recall, and precision for our evaluation.
from sklearn import metrics
#accuracy
print("acuracy:", metrics.accuracy_score(y_test,y_pred=pred))
#precision score
print("precision:", metrics.precision_score(y_test,y_pred=pred))
#recall score
print("recall" , metrics.recall_score(y_test,y_pred=pred))
print(metrics.classification_report(y_test, y_pred=pred))

Output:

Character Recognition With Support Vector Machine

In this example, we will use the existing digit data set and train the classifier. After this, we will use the
classifier to predict a digit and plot the image to be more distinct.
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn import svm
DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:
MACHINE LEARNING LAB 239P1D5809

#loading the dataset

letters = datasets.load_digits()
#generating the classifier
clf = svm.SVC(gamma=0.001, C=100)
#training the classifier
X,y = letters.data[:-10], letters.target[:-10]
clf.fit(X,y)
#predicting the output
print(clf.predict(letters.data[:-10]))
plt.imshow(letters.images[6], interpolation='nearest')
plt.show()

Output:

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

Experiment-3

Write a program to simulate a perception network for pattern classification and function
approximation.

Perception network for pattern classification and function approximation.

Source Code:
# Python program to implement a
# single neuron neural network
# import all necessery libraries
from numpy import exp, array, random, dot, tanh
# Class to create a neural
# network with single neuron
class NeuralNetwork():

def __init__(self):

# Using seed to make sure it'll

# generate same weights in every run
random.seed(1)

# 3x1 Weight matrix

self.weight_matrix = 2 * random.random((3, 1)) - 1

# tanh as activation fucntion

def tanh(self, x):
return tanh(x)

# derivative of tanh function.

# Needed to calculate the gradients.
def tanh_derivative(self, x):
return 1.0 - tanh(x) ** 2

# forward propagation
def forward_propagation(self, inputs):
return self.tanh(dot(inputs, self.weight_matrix))

# training the neural network.

def train(self, train_inputs, train_outputs,
num_train_iterations):

# Number of iterations we want to

# perform for this set of input.
for iteration in range(num_train_iterations):
output = self.forward_propagation(train_inputs)

# Calculate the error in the output.

error = train_outputs - output

# multiply the error by input and then

# by gradient of tanh funtion to calculate
# the adjustment needs to be made in weights
adjustment = dot(train_inputs.T, error *
DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:
MACHINE LEARNING LAB 239P1D5809

self.tanh_derivative(output))

# Adjust the weight matrix

self.weight_matrix += adjustment

# Driver Code
if __name__ == "__main__":

neural_network = NeuralNetwork()

print ('Random weights at the start of training')

print (neural_network.weight_matrix)

train_inputs = array([[0, 0, 1], [1, 1, 1], [1, 0, 1], [0, 1, 1]])

train_outputs = array([[0, 1, 1, 0]]).T

neural_network.train(train_inputs, train_outputs, 10000)

print ('New weights after training')

print (neural_network.weight_matrix)

# Test the neural network with a new situation.

print ("Testing network on new examples ->")
print (neural_network.forward_propagation(array([1, 0, 0])))

Output :

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

Experiment-4
Write a program to demonstrate the working of the decision tree based ID3 algorithm. Use an
appropriate data set for building the decision tree and apply this knowledge to classify a new
sample

ID3 Algorithm:

Training Dataset:

Day Outlook Temperature Humidity Wind PlayTennis

D1 Sunny Hot High Weak No
D2 Sunny Hot High Strong No
D3 Overcast Hot High Weak Yes
D4 Rain Mild High Weak Yes
D5 Rain Cool Normal Weak Yes
D6 Rain Cool Normal Strong No
D7 Overcast Cool Normal Strong Yes
D8 Sunny Mild High Weak No
D9 Sunny Cool Normal Weak Yes
D10 Rain Mild Normal Weak Yes
D11 Sunny Mild Normal Strong Yes
D12 Overcast Mild High Strong Yes
D13 Overcast Hot Normal Weak Yes
D14 Rain Mild High Strong No

Test Dataset:

Day Outlook Temperature Humidity Wind

T1 Rain Cool Normal Strong
T2 Sunny Mild Normal Strong
Program:

import math
import csv
def load_csv(filename):
lines=csv.reader(open(filename,"r"));
dataset = list(lines) headers =
dataset.pop(0) return dataset,headers

class Node:
def init (self,attribute): self.attribute=attribute
self.children=[] self.answer=""

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

def subtables(data,col,delete):
dic={}
coldata=[row[col] for row in data]
attr=list(set(coldata))

counts=[0]*len(attr) r=len(data)
c=len(data[0])
for x in range(len(attr)): for y in range(r):
if data[y][col]==attr[x]: counts[x]+=1

for x in range(len(attr)):
dic[attr[x]]=[[0 for i in range(c)] for j in range(counts[x])]
pos=0
for y in range(r):
if data[y][col]==attr[x]: if delete:
del data[y][col] dic[attr[x]][pos]=data[y]
pos+=1
return attr,dic
def entropy(S):
attr=list(set(S))
if len(attr)==1: return 0

counts=[0,0]
for i in range(2):
counts[i]=sum([1 for x in S if attr[i]==x])/(len(S)*1.0)

sums=0
for cnt in counts:
sums+=-1*cnt*math.log(cnt,2) return sums

def compute_gain(data,col):
attr,dic = subtables(data,col,delete=False)

total_size=len(data)
entropies=[0]*len(attr)
ratio=[0]*len(attr)

total_entropy=entropy([row[-1] for row in data]) for x in range(len(attr)):

ratio[x]=len(dic[attr[x]])/(total_size*1.0) entropies[x]=entropy([row[-1]
for row in
dic[attr[x]]])
total_entropy-=ratio[x]*entropies[x] return total_entropy

def build_tree(data,features):
lastcol=[row[-1] for row in data] if(len(set(lastcol)))==1:
node=Node("")
node.answer=lastcol[0] return node

n=len(data[0])-1
gains=[0]*n
DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:
MACHINE LEARNING LAB 239P1D5809

for col in range(n): gains[col]=compute_gain(data,col)

split=gains.index(max(gains))
node=Node(features[split])
fea = features[:split]+features[split+1:] attr,dic=subtables(data,split,delete=True)

for x in range(len(attr)): child=build_tree(dic[attr[x]],fea) node.children.append((attr[x],child))

return node

def print_tree(node,level):
if node.answer!="":
print(" "*level,node.answer) return

print(" "*level,node.attribute) for value,n in

node.children:
print(" "*(level+1),value)
print_tree(n,level+2)

def classify(node,x_test,features):
if node.answer!="":
print(node.answer) return
pos=features.index(node.attribute) for value, n in
node.children:
if x_test[pos]==value: classify(n,x_test,features)

'''Main program'''
dataset,features=load_csv("data3.csv") node1=build_tree(dataset,features)

print("The decision tree for the dataset using ID3 algorithm is")
print_tree(node1,0) testdata,features=load_csv("data3_test.csv") for xtest
in testdata:
print("The test instance:",xtest)
print("The label for test instance:",end=" ")
classify(node1,xtest,features)

Output :

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

Experiment-5

Build an Artificial Neural Network by implementing the Back propagation algorithm and test the
same using appropriate data sets
BACKPROPAGATION ALGORITHM:

Program:

import numpy as np
X = np.array(([2, 9], [1, 5], [3, 6]), dtype=float)
y = np.array(([92], [86], [89]), dtype=float)
X = X/np.amax(X,axis=0) # maximum of X array longitudinally y = y/100

#Sigmoid Function def

sigmoid (x):
return 1/(1 + np.exp(-x))

#Derivative of Sigmoid Function def

derivatives_sigmoid(x):
return x * (1 - x)

#Variable initialization
epoch=5000 #Setting training iterations lr=0.1
#Setting learning rate
inputlayer_neurons = 2
#number of features in data set hiddenlayer_neurons = 3
#number of hidden layers neurons output_neurons = 1
#number of neurons at output layer

#weight and bias initialization wh=np.random.uniform(size=(inputlayer_neurons,hiddenlayer_neur ons))

bh=np.random.uniform(size=(1,hiddenlayer_neurons))
wout=np.random.uniform(size=(hiddenlayer_neurons,output_neuron s))
bout=np.random.uniform(size=(1,output_neurons))

#draws a random range of numbers uniformly of dim x*y for i in range(epoch):

#Forward Propogation
hinp1=np.dot(X,wh)
hinp=hinp1 + bh
hlayer_act = sigmoid(hinp)
outinp1=np.dot(hlayer_act,wout) outinp= outinp1+ bout
output = sigmoid(outinp)

#Backpropagation EO = y-
output
outgrad = derivatives_sigmoid(output) d_output = EO*
outgrad

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB 239P1D5809

EH = d_output.dot(wout.T)

#how much hidden layer wts contributed to error hiddengrad =

derivatives_sigmoid(hlayer_act) d_hiddenlayer = EH * hiddengrad

# dotproduct of nextlayererror and currentlayerop wout +=

hlayer_act.T.dot(d_output) *lr
wh += X.T.dot(d_hiddenlayer) *lr

print("Input: \n" + str(X)) print("Actual Output: \n" + str(y))

print("Predicted Output: \n" ,output)

Output:

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB
239P1D5809

Experiment-6
Assuming a set of documents that need to be classified, use the naïve Bayesian Classifier
model to perform this task. Built-in Java classes/API can be used to write the program.
Calculate the accuracy, precision, and recall for your data set.
Naive Bayes algorithms for learning and classifying text:
Program:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer from sklearn.naive_bayes
import MultinomialNB
from sklearn import metrics

msg=pd.read_csv('naivetext.csv',names=['message','label']) print('The dimensions of the

dataset',msg.shape) msg['labelnum']=msg.label.map({'pos':1,'neg':0})
X=msg.messag
e
y=msg.labelnu
m

#splitting the dataset into train and test data

xtrain,xtest,ytrain,ytest=train_test_split(X,y)

print ('\n The total number of Training Data :',ytrain.shape) print ('\n The total number of
Test Data :',ytest.shape)

#output of count vectoriser is a sparse matrix

cv = CountVectorizer()
xtrain_dtm = cv.fit_transform(xtrain)
xtest_dtm=cv.transform(xtest)
print('\n The words or Tokens in the text documents \n')
print(cv.get_feature_names())

df=pd.DataFrame(xtrain_dtm.toarray(),columns=cv.get_feature_na mes())

# Training Naive Bayes (NB) classifier on training data.

clf = MultinomialNB().fit(xtrain_dtm,ytrain)
predicted = clf.predict(xtest_dtm)

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB
239P1D5809

#printing accuracy, Confusion matrix, Precision and Recall

print('\n Accuracy of the classifer is’, metrics.accuracy_score(ytest,predicted))

print('\n Confusion matrix') print(metrics.confusion_matrix(ytest,predicted))

print('\n The value of Precision' , metrics.precision_score(ytest,predicted))

print('\n The value of Recall' , metrics.recall_score(ytest,predicted))

Output:

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB
239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB
239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

 MACHINE LEARNING LAB
239P1D5809

Experiment-7

Write a program to implement k-Nearest Neighbor algorithm to classify the iris data set.
Print both correct and wrong predictions

K-Nearest Neighbor Algorithm

Data Set:

Iris Plants Dataset: Dataset contains 150 instances (50 in each of three
classes) Number of Attributes: 4 numeric, predictive attributes and the Class

Program:

from sklearn.model_selection import train_test_split from

sklearn.neighbors import KNeighborsClassifier

from sklearn.metrics import classification_report, confusion_matrix from sklearn import

datasets

""" Iris Plants Dataset, dataset contains 150 (50 in each of three classes)Number of Attributes: 4
numeric, predictive attributes and the Class

"""

iris=datasets.load_iris()

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

 MACHINE LEARNING LAB
239P1D5809

""" The x variable contains the first four columns of thedataset (i.e. attributes) while y contains the
labels.

"""

x = iris.data y =
iris.target

print ('sepal-length', 'sepal-width', 'petal-length','petal-width') print(x)

print('class: 0-Iris-Setosa, 1- Iris-Versicolour, 2- Iris-Virginica') print(y)

""" Splits the dataset into 70% train data and 30% test data. This means that out of total 150
records, the training set will contain

105 records and the test set contains 45 of those records """

x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3)

#To Training the model and Nearest nighbors K=5

classifier = KNeighborsClassifier(n_neighbors=5) classifier.fit(x_train, y_train)

#to make predictions on our test data

y_pred=classifier.predict(x_test)

""" For evaluating an algorithm, confusion matrix, precision, recall and f1 score are the most
commonly used metrics.

"""

print('Confusion Matrix') print(confusion_matrix(y_test,y_pred))

print('Accuracy Metrics') print(classification_report(y_test,y_pred))

Output:

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB
239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO:

MACHINE LEARNING LAB
239P1D5809

DNR COLLEGE OF ENGINEERING & TECHNOLOGY PAGE NO: