Introduction to Machine Learning

The term machine learning was first coined in the 1950s when Artificial
Intelligence pioneer Arthur Samuel built the first self-learning system for playing
checkers. He noticed that the more the system played, the better it performed.
Defn:

Machine learning (ML) is a branch of artificial intelligence (AI) that enables computers to “self-
learn” from training data and improve over time, without being explicitly programmed.

Machine learning is a subfield of artificial intelligence that involves the

development of algorithms and statistical models that enable computers to
improve their performance in tasks through experience. These algorithms and
models are designed to learn from data and make predictions or decisions
without explicit instructions.

How does Machine Learning Work.

A Machine Learning system learns from historical data, builds the prediction
models, and whenever it receives new data, predicts the output for it. The
accuracy of predicted output depends upon the amount of data, as the huge amount
of data helps to build a better model which predicts the output more accurately.

Suppose we have a complex problem, where we need to perform some predictions, so

instead of writing a code for it, we just need to feed the data to generic algorithms,
and with the help of these algorithms, machine builds the logic as per the data and
predict the output. Machine learning has changed our way of thinking about the
problem. The below block diagram explains the working of Machine Learning
algorithm:

Examples of Machine Learning Applications

Variety of applications, including image and speech recognition, self-
driving cars, cyber fraud detection, face recognition, and friend suggestion by
Facebook.

Various top companies such as Netflix and Amazon have built machine
learning models that are using a vast amount of data to analyse the user interest
and recommend product accordingly.

Machine learning in finance, healthcare, hospitality, government, and

beyond, is already in regular use. Businesses are beginning to see the benefits
of using machine learning tools to improve their processes, gain valuable
insights from unstructured data, and automate tasks that would otherwise
require hours of tedious, manual work (which usually produces much less
accurate results).

For example, UberEats uses machine learning to estimate optimum times

for drivers to pick up food orders, while Spotify leverages machine learning to
offer personalized content and personalized marketing. And Dell uses machine
learning text analysis to save hundreds of hours analyzing thousands of
employee surveys to listen to the voice of employee (VoE) and improve
employee satisfaction.

How do you think Google Maps predicts peaks in traffic and Netflix creates
personalized movie recommendations, even informs the creation of new
content ? By using machine learning, of course.

There are many different applications of machine learning, which can benefit
your business in countless ways. You’ll just need to define a strategy to help you
decide the best way to implement machine learning into your existing processes.
In the meantime, here are some common machine learning use cases and
applications that might spark some ideas:

 Social Media Monitoring

 Customer Service & Customer Satisfaction
 Image Recognition
 Virtual Assistants
 Product Recommendations
 Stock Market Trading
 Medical Diagnosis

Social Media Monitoring

Using machine learning you can monitor mentions of your brand on social
media and immediately identify if customers require urgent attention. By
detecting mentions from angry customers, in real-time, you can automatically
tag customer feedback and respond right away.

Customer Service & Customer Satisfaction

Machine learning allows you to integrate powerful text analysis tools with
customer support tools, so you can analyze your emails, live chats, and all
manner of internal data on the go. You can use machine learning to tag support
tickets and route them to the correct teams or auto-respond to common queries
so you never leave a customer in the cold.

Furthermore, using machine learning to set up a voice of customer (VoC)

program and a customer feedback loop will ensure that you follow the customer
journey from start to finish to improve the customer experience (CX), decrease
customer churn, and, ultimately, increase your profits.

Image Recognition

Image recognition is helping companies identify and classify images. For

example, facial recognition technology is being used as a form of identification,
from unlocking phones to making payments.
Self-driving cars also use image recognition to perceive space and obstacles. For
example, they can learn to recognize stop signs, identify intersections, and make
decisions based on what they see.

Virtual Assistants

Virtual assistants, like Siri, Alexa, Google Now, all make use of machine learning
to automatically process and answer voice requests. They quickly scan
information, remember related queries, learn from previous interactions, and
send commands to other apps, so they can collect information and deliver the
most effective answer.

Customer support teams are already using virtual assistants to handle phone
calls, automatically route support tickets, to the correct teams, and speed up
interactions with customers via computer-generated responses.

Product Recommendations

Association rule-learning is a machine learning technique that can be used to

analyze purchasing habits at the supermarket or on e-commerce sites. It works
by searching for relationships between variables and finding common
associations in transactions (products that consumers usually buy together).
This data is then used for product placement strategies and similar product
recommendations.

Associated rules can also be useful to plan a marketing campaign or analyze web
usage.

Stock Market Trading

Machine learning algorithms can be trained to identify trading opportunities, by

recognizing patterns and behaviors in historical data. Humans are often driven
by emotions when it comes to making investments, so sentiment analysis with
machine learning can play a huge role in identifying good and bad investing
opportunities, with no human bias, whatsoever. They can even save time
and allow traders more time away from their screens by automating tasks.

Medical Diagnosis

The ability of machines to find patterns in complex data is shaping the present
and future. Take machine learning initiatives during the COVID-19 outbreak, for
instance. AI tools have helped predict how the virus will spread over time, and
shaped how we control it. It’s also helped diagnose patients by analyzing lung
CTs and detecting fevers using facial recognition, and identified patients at a
higher risk of developing serious respiratory disease.

Types of Machine Learning

At a broad level, machine learning can be classified into three types:

1. Supervised learning
2. Unsupervised learning
3. Reinforcement learning

1.Supervised Learning

Overview:

Supervised learning is a type of machine learning that uses labeled data to train
machine learning models. In labeled data, the output is already known. The model
just needs to map the inputs to the respective outputs.

An example of supervised learning is to train a system that identifies the image of an

animal.

Attached below, you can see that we have our trained model that identifies the
picture of a cat.

Algorithms:

Some of the most popularly used supervised learning algorithms are:

 Linear Regression

 Logistic Regression

 Support Vector Machine

 K Nearest Neighbor

 Decision Tree

 Random Forest

 Naive Bayes

Working:

Supervised learning algorithms take labeled inputs and map them to the known
outputs, which means you already know the target variable.

Now, let’s focus on the training process for the supervised learning method.

Supervised Learning methods need external supervision to train machine learning

models. Hence, the name supervised. They need guidance and additional
information to return the desired result.

Applications:

Supervised learning algorithms are generally used for solving classification and
regression problems.

Few of the top supervised learning applications are weather prediction, sales
forecasting, stock price analysis.
2.Unsupervised Learning

Overview:

Unsupervised learning is a type of machine learning that uses unlabeled data to train
machines. Unlabeled data doesn’t have a fixed output variable. The model learns
from the data, discovers the patterns and features in the data, and returns the
output.

Depicted below is an example of an unsupervised learning technique that uses the

images of vehicles to classify if it’s a bus or a truck. The model learns by identifying
the parts of a vehicle, such as a length and width of the vehicle, the front, and rear
end covers, roof hoods, the types of wheels used, etc. Based on these features, the
model classifies if the vehicle is a bus or a truck.

Algorithms:

Selecting the right algorithm depends on the type of problem you are trying to solve.
Some of the common examples of unsupervised learning are:
 K Means Clusterin

 Hierarchical Clustering

 DBSCAN

 Principal Component Analysis

Working:

Unsupervised learning finds patterns and understands the trends in the data to
discover the output. So, the model tries to label the data based on the features of the
input data.

The training process used in unsupervised learning techniques does not need any
supervision to build models. They learn on their own and predict the output.

Applications:

Unsupervised learning is used for solving clustering and association problems.

One of the applications of unsupervised learning is customer segmentation. Based

on customer behavior, likes, dislikes, and interests, you can segment and cluster
similar customers into a group. Another example where unsupervised learning
algorithms are used is used churn rate analysis.
3.Reinforcement Learning

Overview

Reinforcement Learning trains a machine to take suitable actions and maximize its
rewards in a particular situation. It uses an agent and an environment to produce
actions and rewards. The agent has a start and an end state. But, there might be
different paths for reaching the end state, like a maze. In this learning technique,
there is no predefined target variable.

An example of reinforcement learning is to train a machine that can identify the

shape of an object, given a list of different objects. In the example shown, the model
tries to predict the shape of the object, which is a square in this case.

Algorithms

Some of the important reinforcement learning algorithms are:

1. Q-learning

2. Sarsa

3. Monte Carlo

4. Deep Q network
Working

Reinforcement learning follows trial and error methods to get the desired result. After
accomplishing a task, the agent receives an award. An example could be to train a
dog to catch the ball. If the dog learns to catch a ball, you give it a reward, such as a
biscuit.

Reinforcement Learning methods do not need any external supervision to train

models.

Reinforcement learning problems are reward-based. For every task or for every step
completed, there will be a reward received by the agent. If the task is not achieved
correctly, there will be some penalty added.

Now, let’s see what applications we have in reinforcement learning.

Applications

Reinforcement learning algorithms are widely used in the gaming industries to build
games. It is also used to train robots to do human tasks.
LEARNING ASSOCIATIONS
Association rule mining finds interesting associations and relationships among
large sets of data items. This rule shows how frequently an itemset occurs in a
transaction. A typical example is a Market Based Analysis.
Market Based Analysis is one of the key techniques used by large relations to
show associations between items. It allows retailers to identify relationships between
the items that people buy together frequently.
Given a set of transactions, we can find rules that will predict the occurrence of
an item based on the occurrences of other items in the transaction.

TID Items

1 Bread, Milk

2 Bread, Diaper, Beer, Eggs

3 Milk, Diaper, Beer, Coke

4 Bread, Milk, Diaper, Beer

5 Bread, Milk, Diaper, Coke

Before we start defining the rule, let us first see the basic definitions.
Support Count( ) – Frequency of occurrence of a itemset.

Here ({Milk, Bread, Diaper})=2

Frequent Itemset – An itemset whose support is greater than or equal to

minsup threshold.

Association Rule – An implication expression of the form X -> Y, where X

and Y are any 2 itemsets.

Example: {Milk, Diaper}->{Beer}

Rule Evaluation Metrics –

 Support(s) –
The number of transactions that include items in the {X} and {Y} parts of
the rule as a percentage of the total number of transaction.It is a measure
 of how frequently the collection of items occur together as a percentage of
all transactions.
 Support = (X+Y) total
It is interpreted as fraction of transactions that contain both X and Y.
 Confidence(c) –
It is the ratio of the no of transactions that includes all items in {B} as well
as the no of transactions that includes all items in {A} to the no of
transactions that includes all items in {A}.
 Conf(X=>Y) = Supp(X Y) Supp(X) –
It measures how often each item in Y appears in transactions that
contains items in X also.
 Lift(l) –
The lift of the rule X=>Y is the confidence of the rule divided by the
expected confidence, assuming that the itemsets X and Y are
independent of each other.The expected confidence is the confidence
divided by the frequency of {Y}.
 Lift(X=>Y) = Conf(X=>Y) Supp(Y) –
Lift value near 1 indicates X and Y almost often appear together as
expected, greater than 1 means they appear together more than expected
and less than 1 means they appear less than expected.Greater lift values
indicate stronger association.

Example – From the above table, {Milk, Diaper}=>{Beer}

s= ({Milk, Diaper, Beer}) |T|
= 2/5
= 0.4

c= (Milk, Diaper, Beer) (Milk, Diaper)

= 2/3
= 0.67

l= Supp({Milk, Diaper, Beer}) Supp({Milk, Diaper})*Supp({Beer})

= 0.4/(0.6*0.6)
= 1.11

The Association rule is very useful in analyzing datasets. The data is

collected using bar-code scanners in supermarkets. Such databases consists
of a large number of transaction records which list all items bought by a
customer on a single purchase. So the manager could know if certain groups
 of items are consistently purchased together and use this data for adjusting
store layouts, cross-selling, promotions based on statistics.

Classification
As we know, the Supervised Machine Learning algorithm can be broadly classified into
Regression and Classification Algorithms. In Regression algorithms, we have predicted
the output for continuous values, but to predict the categorical values, we need
Classification algorithms.

Classification Algorithm
The Classification algorithm is a Supervised Learning technique that is used to identify
the category of new observations on the basis of training data. In Classification, a
program learns from the given dataset or observations and then classifies new
observation into a number of classes or groups. Such as, Yes or No, 0 or 1, Spam or
Not Spam, cat or dog, etc. Classes can be called as targets/labels or categories.

Unlike regression, the output variable of Classification is a category, not a value, such
as "Green or Blue", "fruit or animal", etc. Since the Classification algorithm is a
Supervised learning technique, hence it takes labeled input data, which means it
contains input with the corresponding output.

In classification algorithm, a discrete output function(y) is mapped to input variable(x).

1. y=f(x), where y = categorical output

The best example of an ML classification algorithm is Email Spam Detector.

The main goal of the Classification algorithm is to identify the category of a given
dataset, and these algorithms are mainly used to predict the output for the categorical
data.

Classification algorithms can be better understood using the below diagram. In the
below diagram, there are two classes, class A and Class B. These classes have features
that are similar to each other and dissimilar to other classes.
The algorithm which implements the classification on a dataset is known as a classifier.
There are two types of Classifications:

o Binary Classifier: If the classification problem has only two possible outcomes, then it
is called as Binary Classifier.
Examples: YES or NO, MALE or FEMALE, SPAM or NOT SPAM, CAT or DOG, etc.
o Multi-class Classifier: If a classification problem has more than two outcomes, then it
is called as Multi-class Classifier.
Example: Classifications of types of crops, Classification of types of music.

Learners in Classification Problems:

In the classification problems, there are two types of learners:

1. Lazy Learners: Lazy Learner firstly stores the training dataset and wait until it receives
the test dataset. In Lazy learner case, classification is done on the basis of the most
related data stored in the training dataset. It takes less time in training but more time
for predictions.
Example: K-NN algorithm, Case-based reasoning
2. Eager Learners:Eager Learners develop a classification model based on a training
dataset before receiving a test dataset. Opposite to Lazy learners, Eager Learner takes
more time in learning, and less time in prediction. Example: Decision Trees, Naïve
Bayes, ANN.
Regression
Regression analysis is a statistical method to model the relationship between a
dependent (target) and independent (predictor) variables with one or more
independent variables. More specifically, Regression analysis helps us to understand
how the value of the dependent variable is changing corresponding to an independent
variable when other independent variables are held fixed. It predicts continuous/real
values such as temperature, age, salary, price, etc.

We can understand the concept of regression analysis using the below example:

Example: Suppose there is a marketing company A, who does various advertisement

every year and get sales on that. The below list shows the advertisement made by the
company in the last 5 years and the corresponding sales:

Now, the company wants to do the advertisement of $200 in the year 2019 and wants
to know the prediction about the sales for this year. So to solve such type of
prediction problems in machine learning, we need regression analysis.

Regression is a supervised learning technique which helps in finding the correlation

between variables and enables us to predict the continuous output variable based on
the one or more predictor variables. It is mainly used for prediction, forecasting, time
series modeling, and determining the causal-effect relationship between
variables.
In Regression, we plot a graph between the variables which best fits the given
datapoints, using this plot, the machine learning model can make predictions about
the data. In simple words, "Regression shows a line or curve that passes through
all the datapoints on target-predictor graph in such a way that the vertical
distance between the datapoints and the regression line is minimum." The
distance between datapoints and line tells whether a model has captured a strong
relationship or not.

Some examples of regression can be as:

o Prediction of rain using temperature and other factors

o Determining Market trends
o Prediction of road accidents due to rash driving.

Unsupervised Learning
As the name suggests, unsupervised learning is a machine learning technique in which
models are not supervised using training dataset. Instead, models itself find the hidden
patterns and insights from the given data. It can be compared to learning which takes
place in the human brain while learning new things. It can be defined as:

Unsupervised learning is a type of machine learning in which models are trained using unlabeled
dataset and are allowed to act on that data without any supervision.

Unsupervised learning cannot be directly applied to a regression or classification

problem because unlike supervised learning, we have the input data but no
corresponding output data. The goal of unsupervised learning is to find the
underlying structure of dataset, group that data according to similarities, and
represent that dataset in a compressed format.
Example: Suppose the unsupervised learning algorithm is given an input dataset
containing images of different types of cats and dogs. The algorithm is never trained
upon the given dataset, which means it does not have any idea about the features of
the dataset. The task of the unsupervised learning algorithm is to identify the image
features on their own. Unsupervised learning algorithm will perform this task by
clustering the image dataset into the groups according to similarities between images.

Working of Unsupervised Learning

Working of unsupervised learning can be understood by the below diagram:

Here, we have taken an unlabeled input data, which means it is not categorized and
corresponding outputs are also not given. Now, this unlabeled input data is fed to the
machine learning model in order to train it. Firstly, it will interpret the raw data to find
the hidden patterns from the data and then will apply suitable algorithms such as k-
means clustering, Decision tree, etc.

Once it applies the suitable algorithm, the algorithm divides the data objects into
groups according to the similarities and difference between the objects.

Types of Unsupervised Learning Algorithm:

The unsupervised learning algorithm can be further categorized into two types of
problems:

o Clustering: Clustering is a method of grouping the objects into clusters such that
objects with most similarities remains into a group and has less or no similarities with
the objects of another group. Cluster analysis finds the commonalities between the
data objects and categorizes them as per the presence and absence of those
commonalities.
o Association: An association rule is an unsupervised learning method which is used for
finding the relationships between variables in the large database. It determines the set
of items that occurs together in the dataset. Association rule makes marketing strategy
more effective. Such as people who buy X item (suppose a bread) are also tend to
purchase Y (Butter/Jam) item. A typical example of Association rule is Market Basket
Analysis.

Advantages of Unsupervised Learning

o Unsupervised learning is used for more complex tasks as compared to supervised
learning because, in unsupervised learning, we don't have labeled input data.
o Unsupervised learning is preferable as it is easy to get unlabeled data in comparison
to labeled data.

Disadvantages of Unsupervised Learning

o Unsupervised learning is intrinsically more difficult than supervised learning as it does
not have corresponding output.
 o The result of the unsupervised learning algorithm might be less accurate as input data
is not labeled, and algorithms do not know the exact output in advance.

Supervised Machine Learning

Supervised learning is the types of machine learning in which machines are trained
using well "labelled" training data, and on basis of that data, machines predict the
output. The labelled data means some input data is already tagged with the correct
output.

In supervised learning, the training data provided to the machines work as the
supervisor that teaches the machines to predict the output correctly. It applies the
same concept as a student learns in the supervision of the teacher.

Supervised learning is a process of providing input data as well as correct output data
to the machine learning model. The aim of a supervised learning algorithm is to find
a mapping function to map the input variable(x) with the output variable(y).

In the real-world, supervised learning can be used for Risk Assessment, Image
classification, Fraud Detection, spam filtering, etc.

How Supervised Learning Works?

In supervised learning, models are trained using labelled dataset, where the model
learns about each type of data. Once the training process is completed, the model is
tested on the basis of test data (a subset of the training set), and then it predicts the
output.

The working of Supervised learning can be easily understood by the below example
and diagram:
Suppose we have a dataset of different types of shapes which includes square,
rectangle, triangle, and Polygon. Now the first step is that we need to train the model
for each shape.

o If the given shape has four sides, and all the sides are equal, then it will be labelled as
a Square.
o If the given shape has three sides, then it will be labelled as a triangle.
o If the given shape has six equal sides then it will be labelled as hexagon.

Now, after training, we test our model using the test set, and the task of the model is
to identify the shape.

The machine is already trained on all types of shapes, and when it finds a new shape,
it classifies the shape on the bases of a number of sides, and predicts the output.

Steps Involved in Supervised Learning:

o First Determine the type of training dataset
o Collect/Gather the labelled training data.
o Split the training dataset into training dataset, test dataset, and validation dataset.
o Determine the input features of the training dataset, which should have enough
knowledge so that the model can accurately predict the output.
o Determine the suitable algorithm for the model, such as support vector machine,
decision tree, etc.
 o Execute the algorithm on the training dataset. Sometimes we need validation sets as
the control parameters, which are the subset of training datasets.
o Evaluate the accuracy of the model by providing the test set. If the model predicts the
correct output, which means our model is accurate.

Types of supervised Machine learning Algorithms:

Supervised learning can be further divided into two types of problems:

1. Regression

Regression algorithms are used if there is a relationship between the input variable
and the output variable. It is used for the prediction of continuous variables, such as
Weather forecasting, Market Trends, etc. Below are some popular Regression
algorithms which come under supervised learning:

o Linear Regression
o Non-Linear Regression
o Polynomial Regression

2. Classification

Classification algorithms are used when the output variable is categorical, which means
there are two classes such as Yes-No, Male-Female, True-false, etc.

Spam Filtering,

o Random Forest
o Decision Trees
o Support vector Machines
Advantages of Supervised learning:
o With the help of supervised learning, the model can predict the output on the basis of
prior experiences.
o Supervised learning model helps us to solve various real-world problems such as fraud
detection, spam filtering, etc.

Disadvantages of supervised learning:

o Supervised learning models are not suitable for handling the complex tasks.
o Supervised learning cannot predict the correct output if the test data is different from
the training dataset.
o Training required lots of computation times.

Supervised Learning Unsupervised Learning

Supervised learning algorithms are Unsupervised learning algorithms are

trained using labeled data. trained using unlabeled data.

Supervised learning model takes direct Unsupervised learning model does not
feedback to check if it is predicting take any feedback.
correct output or not.

Supervised learning model predicts the Unsupervised learning model finds the
output. hidden patterns in data.

In supervised learning, input data is In unsupervised learning, only input

provided to the model along with the data is provided to the model.
output.

The goal of supervised learning is to train The goal of unsupervised learning is to

the model so that it can predict the find the hidden patterns and useful
output when it is given new data. insights from the unknown dataset.

Supervised learning needs supervision to Unsupervised learning does not need

train the model. any supervision to train the model.
 Supervised learning can be categorized Unsupervised Learning can be classified
in Classification and Regression proble in Clustering and Associations proble
ms. ms.

Supervised learning model produces an Unsupervised learning model may give less
accurate result. accurate result as compared to supervised
learning.

It includes various algorithms such as Linear It includes various algorithms such as

Regression, Logistic Regression, Support Clustering, KNN, and Apriori algorithm.
Vector Machine, Multi-class Classification,
Decision tree, Bayesian Logic, etc.

Perspectives and Issues in Machine

Learning
PERSPECTIVES:
 Machine learning is a subfield of artificial intelligence and machine learning algorithms
are used in other related fields like natural language processing and computer vision.
 In general, there are three types of learning and these are supervised learning,
unsupervised learning, and reinforcement learning.
 Their names tell the main idea behind them actually.
 In supervised learning, your system learns under the supervision of the data outputs so
supervised algorithms are preferred if your dataset contains output information.
 Let me give you an example in there.
 Let’s assume you have a medical statistic company and you have a dataset which
contains patients’ features like blood pressure, sugar rate in their blood, heart rate per
minute, etc.
ISSUES:
LACK OF QUALITY DATA
 One of the main issues in Machine Learning is the absence of good data.
 While, algorithms tend to make developers exhaust most of their time on artificial
intelligence.
FAULT IN CREDIT CARD FRAUD DETECTION
 Although this AI-driven software helps to successfully detect credit card fraud, there are
issues in Machine Learning that make the process redundant.
GETTING BAD RECOMMENDATIONS
 Proposal engines are quite regular today.
 While some might be dependable, others may not appear to provide the necessary
results.
TALENT DEFICIT
 Albeit numerous individuals are pulled into the ML business, however, there are still not
many experts who can take complete control of this innovation.
MAKING THE WRONG ASSUMPTIONS
 ML models can’t manage datasets containing missing data points.
 Thus, highlights that contain a huge part of missing data should be erased.

Concept Learning
Concept Learning in Machine Learning can be thought of as a boolean-valued function
defined over a large set of training data. Taking a very simple example, one possible target
concept may be to Find the day when my friend Ramesh enjoys his favorite sport. We have
some attributes/features of the day like, Sky, Air
Temperature, Humidity, Wind, Water, Forecast and based on this we have a target Concept
named EnjoySport.

We have the following training example available:

Exampl AirTe Humidit Foreca EnjoySp

Sky Wind Water
e mp y st ort

1 Sunny Warm Normal Strong Warm Same Yes

2 Sunny Warm High Strong Warm Same Yes

3 Rainy Cold High Strong Warm Change No

4 Sunny Warm High Strong Cool Change Yes

Let’s Design the problem formally with TPE(Task, Performance, Experience):

Problem: Leaning the day when Ramesh enjoys the sport.

Task T: Learn to predict the value of EnjoySport for an arbitrary day, based on the
values of the attributes of the day.

Performance measure P: Total percent of days (EnjoySport) correctly predicted.

Training experience E: A set of days with given labels (EnjoySport: Yes/No)

Let us take a very simple hypothesis representation which consists of
a conjunction of constraints in the instance attributes. We get a hypothesis h_i with
the help of example i for our training set as below:

hi(x) := <x1, x2, x3, x4, x5, x6>

where x1, x2, x3, x4, x5 and x6 are the values

of Sky, AirTemp, Humidity, Wind, Water and Forecast.

Hence h1 will look like(the first row of the table above):

h1(x=1): <Sunny, Warm, Normal, Strong, Warm, Same > Note: x=1 represents a
positive hypothesis / Positive example

We want to find the most suitable hypothesis which can represent the concept. For
example, Ramesh enjoys his favorite sport only on cold days with high
humidity (This seems independent of the values of the other attributes present in
the training examples).

h(x=1) = <?, Cold, High, ?, ?, ?>

Here ? indicates that any value of the attribute is acceptable. Note: The most generic
hypothesis will be < ?, ?, ?, ?, ?, ?> where every day is a positive example and the
most specific hypothesis will be <?,?,?,?,?,? > where no day is a positive example.

We will discuss the two most popular approaches to find a suitable hypothesis, they
are:

1. Find-S Algorithm
2. List-Then-Eliminate Algorithm

Find-S Algorithm:
Following are the steps for the Find-S algorithm:

1. Initialize h to the most specific hypothesis in H

2. For each positive training example,
1. For each attribute, constraint ai in h
1. If the constraints ai is satisfied by x
2. Then do nothing
3. Else replace ai in h by the next more general constraint that is
satisfied by x
3. Output hypothesis h
The LIST-THEN-ELIMINATE Algorithm:
Following are the steps for the LIST-THE-ELIMINATE algorithm:

VersionSpace <- a list containing every hypothesis in H

For each training example, <x, c(x)>

 Remove from VersionSpace any hypothesis h for which h(x) != c(x)

Output the list of hypotheses in VersionSpace.

Perceptron
Frank Rosenblatt (1928 – 1971) was an American psychologist notable in
the field of Artificial Intelligence.

In 1957 he started something really big. He "invented"

a Perceptron program, on an IBM 704 computer at Cornell Aeronautical
Laboratory.

Scientists had discovered that brain cells (Neurons) receive input from our
senses by electrical signals.

The Neurons, then again, use electrical signals to store information, and to
make decisions based on previous input.

Frank had the idea that Perceptrons could simulate brain principles, with
the ability to learn and make decisions.

The Perceptron
The original Perceptron was designed to take a number of binary inputs,
and produce one binary output (0 or 1).

The idea was to use different weights to represent the importance of

each input, and that the sum of the values should be greater than
a threshold value before making a decision like yes or no (true or false) (0
or 1).
Perceptron Example
Imagine a perceptron (in your brain).

The perceptron tries to decide if you should go to a concert.

Is the artist good? Is the weather good?

What weights should these facts have?

Criteria Input Weight

Artists is Good x1 = 0 or 1 w1 = 0.7

Weather is Good x2 = 0 or 1 w2 = 0.6

 Friend will Come x3 = 0 or 1 w3 = 0.5

Food is Served x4 = 0 or 1 w4 = 0.3

The Perceptron Algorithm

Frank Rosenblatt suggested this algorithm:

1. Set a threshold value

2. Multiply all inputs with its weights
3. Sum all the results
4. Activate the output

1. Set a threshold value:

 Threshold = 1.5

2. Multiply all inputs with its weights:

 x1 * w1 = 1 * 0.7 = 0.7
 x2 * w2 = 0 * 0.6 = 0
 x3 * w3 = 1 * 0.5 = 0.5
 x4 * w4 = 0 * 0.3 = 0
 x5 * w5 = 1 * 0.4 = 0.4

3. Sum all the results:

 0.7 + 0 + 0.5 + 0 + 0.4 = 1.6 (The Weighted Sum)

4. Activate the Output:

 Return true if the sum > 1.5 ("Yes I will go to the Concert")
Linear separability
An ANN does not give an exact solution for a nonlinear problem. However, it
provides an approximate solution to nonlinear problems. Linear separability is the
concept wherein the separation of input space into regions is based on whether the
network response is positive or negative.
A decision line is drawn to separate positive and negative responses. The
decision line may also be called as the decision-making Line or decision-support Line
or linear-separable line. The necessity of the linear separability concept was felt to
clarify classify the patterns based upon their output responses.
Generally, the net input calculated to the output unit is given as –
yin=b+∑ni=1(xiwi)
The linear separability of the network is based on the decision-boundary line. If there
exist weight for which the training input vectors having a positive (correct) response,
or lie on one side of the decision boundary and all the other vectors having
negative, −1−1, response lies on the other side of the decision boundary then we can
conclude the problem is "Linearly Separable".
Consider, a single layer network as shown in the figure.

Fig:- A single Layer Network

The net input for the network shown in the figure is given as-

yin=b+x1w1+x2w2

The separating line for which the boundary lies between the values x1 and x2, so that
the net gives a positive response on one side and negative response on the other side,
is given as,
Linear Regression
Linear regression is one of the easiest and most popular Machine Learning algorithms.
It is a statistical method that is used for predictive analysis. Linear regression makes
predictions for continuous/real or numeric variables such as sales, salary, age,
product price, etc.

Linear regression algorithm shows a linear relationship between a dependent (y) and
one or more independent (y) variables, hence called as linear regression. Since linear
regression shows the linear relationship, which means it finds how the value of the
dependent variable is changing according to the value of the independent variable.
The linear regression model provides a sloped straight line representing the
relationship between the variables. Consider the below image:

Mathematically, we can represent a linear regression as:

y= a0+a1x+ ε

Here,

Y= Dependent Variable (Target Variable)

X= Independent Variable (predictor Variable)
a0= intercept of the line (Gives an additional degree of freedom)
a1 = Linear regression coefficient (scale factor to each input value).
ε = random error

The values for x and y variables are training datasets for Linear Regression model
representation.

Types of Linear Regression

Linear regression can be further divided into two types of the algorithm:
 o Simple Linear Regression:
If a single independent variable is used to predict the value of a numerical dependent
variable, then such a Linear Regression algorithm is called Simple Linear Regression.
o Multiple Linear regression:
If more than one independent variable is used to predict the value of a numerical
dependent variable, then such a Linear Regression algorithm is called Multiple Linear
Regression.

Linear Regression Line

A linear line showing the relationship between the dependent and independent
variables is called a regression line. A regression line can show two types of
relationship:

o Positive Linear Relationship:

If the dependent variable increases on the Y-axis and independent variable increases
on X-axis, then such a relationship is termed as a Positive linear relationship.

o Negative Linear Relationship:

If the dependent variable decreases on the Y-axis and independent variable increases
on the X-axis, then such a relationship is called a negative linear relationship.
Finding the best fit line:
When working with linear regression, our main goal is to find the best fit line that
means the error between predicted values and actual values should be minimized. The
best fit line will have the least error.

The different values for weights or the coefficient of lines (a0, a1) gives a different line
of regression, so we need to calculate the best values for a 0 and a1 to find the best fit
line, so to calculate this we use cost function.

Cost function-
o The different values for weights or coefficient of lines (a0, a1) gives the different line of
regression, and the cost function is used to estimate the values of the coefficient for
the best fit line.
o Cost function optimizes the regression coefficients or weights. It measures how a linear
regression model is performing.
o We can use the cost function to find the accuracy of the mapping function, which
maps the input variable to the output variable. This mapping function is also known
as Hypothesis function.

For Linear Regression, we use the Mean Squared Error (MSE) cost function, which is
the average of squared error occurred between the predicted values and actual values.
It can be written as:

For the above linear equation, MSE can be calculated as:

Where,

N=Total number of observation

Yi = Actual value
(a1xi+a0)= Predicted value.

Residuals: The distance between the actual value and predicted values is called
residual. If the observed points are far from the regression line, then the residual will
be high, and so cost function will high. If the scatter points are close to the regression
line, then the residual will be small and hence the cost function.