Unit 1

Introduction to Machine Learning

K. R Singh
 Why Machine Learning
• Recent progress in algorithms and theory
• Growing flood of online data
• Computational power is available
• Budding industry
• Three niches for machine learning:
– Data mining : using historical data to improve decisions
{medical records ! medical knowledge}
– Software applications we can't program by hand
{ autonomous driving, speech recognition}
– Self customizing programs { Newsreader that learns user
interests}

Problem Solving
Email classification based on contents
 Auto answering emails (or auto answering call centers)
 Product recommendation system
 QA- NLP engine (like IBM Watson – deep blue-Jeopardy)
 Auto interviewing machine
 Paper setting and correction
 Visual positioning system (Scene classification) GPS
 Object recognition and identification from video
 Object recognition and identification from voice
 Driverless car (Tesla, Google, GM, Nvidia …)
 Secrete profile builder from social sites
 Sentiment analysis (crime detection)
Indian context:
 Risk analysis and Forecasting
• Identification of decease from pulse
 Medical diagnosis • Yoga poster correction while doing yoga
 • Horoscope
Search engines
• Mantras correct recital
 Handwriting recognition • Prediction in academics
 Evolutions

Humanoid Robot Sophia: Sophia is a social

humanoid robot developed by Hong Kong-based
company Hanson Robotics. Sophia, the robot
became the first robot to receive citizenship of
 Differences between Machine Learning and Statistical Modeling

Schools they come from

Differences between Machine Learning and Statistical Modeling

Schools they come from

When did they come into existence?
Differences between Machine Learning and
Statistical Modeling

Schools they come from

When did they come into existence?
Assumptions they work on
Differences between Machine Learning and
Statistical Modeling

Schools they come from

When did they come into existence?
Assumptions they work on
Type of data they deal with
Differences between Machine Learning and
Statistical Modeling

Schools they come from,

When did they come into existence?
Assumptions they work on
Type of data they deal with
Nomenclatures of operations and objects
Differences between Machine Learning and
Statistical Modeling

Schools they come from,

When did they come into existence?
Assumptions they work on
Type of data they deal with
Nomenclatures of operations and objects
Techniques used
Differences between Machine Learning and
Statistical Modeling

Schools they come from,

When did they come into existence?
Assumptions they work on
Type of data they deal with
Nomenclatures of operations and objects
Techniques used
Predictive power and human efforts involved to implement
Differences between Machine Learning and
Statistical Modeling

They belong to different schools

Machine Learning is …
a subfield of computer science and artificial
intelligence which deals with building systems
that can learn from data, instead of explicitly
programmed instructions.
Differences between Machine Learning and
Statistical Modeling
They belong to different schools

Machine Learning is …
a subfield of computer science and artificial intelligence
which deals with building systems that can learn from
data, instead of explicitly programmed instructions.

Statistical Modelling is …
a subfield of mathematics which deals with finding
relationship between variables to predict an outcome
Differences between Machine Learning and
Statistical Modeling
They came up in different eras
Statistical modeling has been there for centuries
now.
However, Machine learning is a very recent
development. 1950
Differences between Machine Learning and
Statistical Modeling
They came up in different eras
Statistical modeling has been there for centuries now.
However, Machine learning is a very recent development. 1950

Extent of assumptions involved

Linear relation between independent and dependent variable

Machine Learning algorithms do not specify the distribution of

dependent or independent variable in a machine learning
algorithm.
Differences between Machine Learning and
Statistical Modeling
Types of data they deal with

ML capable of learning from trillions of

observations one by one.
They make prediction and learn simultaneously.
Differences between Machine Learning and
Statistical Modeling
Types of data they deal with

ML capable of learning from trillions of observations one

by one. They make prediction and learn simultaneously.

However statistical modeling are generally applied for

smaller data with less attributes or they end up over
fitting.
 Machine Learning
• Machine learning is a general term used to apply to many
techniques which utilize statistical iteration and feedback so
that correlations or logic is learned rather than dictated.
• Learning itself is the act of gradually improving performance
on a task without being explicitly programmed. This process
mimics human neurological functions.
• 1. Teaching a child about danger, the alphabet, or to
speak (overt repetition and supervision)
• 2. How you catch a baseball or get out of the way of a
moving car. (retina -> brain -> reflex)
 Formulation

Even when the end goal for both machine learning

and statistical modeling is same, the formulation
of two are significantly different.

In a statistical model, we basically

v1 try to estimate
v2

the function f in e1 3 4
e2 5 7

Dependent Variable ( Y ) = f(Independent Variable)

+ error function
 Formulation
Machine Learning takes away the deterministic
function “f” out of the equation. It simply becomes

Output(Y) ----- > Input (X)

It will try to find pockets of X in n dimensions (where n

is the number of attributes), where occurrence of Y
is significantly different.
 Problem Solving
3 entities f (function/algorithm), X (inputs) and Y (outputs)
 Let problem to solve is f (X) = Y
 There 8 possible combinations of f, X &Y
 0 represents entity is unknown and 1 for known entity
f X Y Problem solving techniques
0 0 0 Nothing is known, Impossible to solve
0 0 1 Only output is known, diff/impossible to solve(series & sequences)
0 1 0 Only input is known, impossible to solve
0 1 1 Machine Learning
1 0 0 Typical deductive math problem (test cases)
1 0 1 Optimization problem
1 1 0 Computing problem
1 1 1 Everything is known hence problem is solved

f is modeled --- exact or approximate

 Introduction
Data Program

Computing Problem:
Computer
Algorithmic Solution

Output

Output ---- labels

Data Output
Input - features
Machine Learning Solution
Computer

Program
Paradigm shift from mathematical
thinking to natural thinking Function --- Hypothesis
(Learning)
 Why Learn?
 There is no need to “learn” if it's a computing problem

 Learning is used when:

 Human expertise does not exist (navigating on Mars)

 Humans are unable to explain their expertise (speech

recognition) (God’s Gift) (Human with supernatural power)

 Solution changes in time (routing on a computer

network, driving a car)

 Solution needs to be adapted to particular cases

(recommendations of movies or books on Netflix or Amazon)
 Machine Learning
Learning is an ability to improve one’s behavior based on
experience.

 Machine learning builds a model that automatically improve

with experience.

 Study of Machine learning explores, techniques that can learn

from data and build models from data.

 These models can be used for prediction, decision making or

solving some task.

Machine Learning: Definition (by Tom Mitchell)

A computer program is said to

 Learn from experience E with respect to
 Some class of tasks T and performance measure P,
If its performance of tasks in T, as measured by P, improves
with experience E.
 Learner ’s model
 The learner will learn from experience & background
knowledge and prepare a model.
 Use model or decision making when particular task is
presented to answer.

For Example:
Task:
1. Predict a mood (sad, happy, angry) of a person in the
picture
2. Predict a share price of a particular company.
Experience:
3. Dataset of images of person showing different moods
4. Past history of a share price of company
Performance:
5. Increasing accuracy in prediction Salesman
 How to design a learner
Training Set Experiences
Experiences
Choose
Testing Set
 Choose the training experience: Training Experiences
select experiences that have
variance (capture most of the Learn
possible scenario)
 Choose the target function:
determine exactly what type Knowledge
of knowledge will be learned
Describ
 Choose how to represent the e
target function: choose a Knowledge
representation to describe the representation
knowledge (table base, rule Choose
based, hierarchical,
polynomial, ANN)
M/c learning algorithm
 Choose a learning algorithm to
infer the target function: choose Generate
the proper learning algorithm
to build knowledge from
experience
Sugarcane juice Learner/Model
 Machine Learning Model
From training examples (experiences) design a model that
is a collection of approximations of the target function
 This collection forms a hypothesis space
 Learning algorithm choose a best approximation (i.e.
hypothesis) to represent the target function

T Data Output
r
a
i
n Computer
i
n
g
Program Data
T
e
s
t
Computer
i
n
g Output
 Ty p e s o f M a c h i n e L e a r n i n g
Types of machine learning Algorithms are commonly
divided in the style of learning.

The main categories are the following:

 Supervised Learning

 Unsupervised learning

 Semi-supervised Learning

 Reinforcement Learning
 Data set

Vehicle No Height(m) Length(m) Width(m) Label/ Type

Veh-1 MH-31-A-9999 1.409 4.029 1.740 Car
Veh-2 MH-34-F-9981 3.300 12.190 2.590 Bus
Veh-3 MH-31-F-0729 1.558 4.107 1.760 Car
Veh-4 MH-31-E-1163 1,540 3.600 1.600 Car
Veh-5 MH-49-A-2222 3.500 13.800 2.580 Bus
…….
Veh-n ….. ….. …… …. …..
 Supervised Learning
Given input variables (X) and an output variable (Y) and
an algorithm learn the mapping function from the input
to the output. (Learning a model from labeled data)

 It acts as a classifier when Y is discrete

 As a predictor (regression) when Y is
continuous
 Supervised Learning
Training data: “examples" X with “labels" y.
• (X1, y1),……., (Xn, yn) = xi  Rd
Classification: y is discrete. To simplify, y  {-1,+1}
• f : Rd {-1, +1} f is called a binary classifier.
Example: Approve credit yes/no, spam/ham,
banana/orange.
 Supervised Learning
Training data: “examples" X with “labels" y.
• (X1, y1),……., (Xn, yn) = xi  Rd
Regression: y is a real value (continuous), y  R
• f : Rd R, f is called a regressor.
Example: amount of credit, weight of fruit.

Example: Income in function of age, weight of the fruit in

function of its length.
 Supervised Learning
Supervised learning categories and techniques
 Linear classifier (numerical functions)
 Parametric (Probabilistic functions)
• Naïve Bayes, Gaussian discriminant analysis
(GDA), Hidden Markov models (HMM),
Probabilistic graphical models
 Non-parametric (Instance-based functions)
• K-nearest neighbors, Support Vector Machine,
Kernel density estimation, Local regression
 Non-metric (Symbolic functions)
• Classification and regression tree (CART),
decision tree
 Aggregation
• Bagging (bootstrap + aggregation), Adaboost,
Random forest
 Unsupervised Learning
Given only input data (X) and no corresponding output
variables.
 Model the underlying structure or distribution in the
data in order to learn more about the data
 Input space has a structure such that certain patterns
occur more often than others
 Learning a model from unlabeled data.

Association: discover rules that describe large portions

of data

Clustering: discover the inherent groupings in the data

 Unsupervised Learning
Training data: “examples" X
• X1,…….,Xn = Xi  Rd
Clustering/segmentation:
f : Rd { C1, C2, ….., Ck} (set of clusters)
Example: Find clusters in the population, fruits,
species.
 Unsupervised Learning
Unsupervised learning categories and techniques
 Clustering
• K-means clustering
• Spectral clustering
 Density Estimation
• Gaussian mixture model (GMM)
• Graphical models
 Dimensionality reduction
• Principal component analysis (PCA)
• Factor analysis

Discovering hidden patterns and further

learning
 Semi-supervised Learning
It combines both labeled and unlabeled examples to
generate an appropriate predictor or classifier.

 It can be expensive or time-consuming to label data

as it may require access to domain experts.
 Unlabeled data is cheap and easy to collect and
store.
 Use unsupervised learning techniques to discover
and learn the structure in the input variables.
 Then use supervised learning techniques to make
best guess predictions for the unlabeled data
 Feed that data back into the supervised learning
algorithm as training data and use the model to
make predictions on new unseen data.

Considerable improvement in learning accuracy over

unsupervised learning, but without the time and costs needed
for supervised learning
 Reinforcement learning
Learning from interaction with the environment comes
from our natural experiences.

Let a child in a living room see a fireplace, and approach

it.

It’s warm, it’s positive, he feels good (Positive Reward

+1). Child understand that fire is a positive thing.

But then he tries to touch the fire. It burns his hand

(Negative reward -1).
 Reinforcement learning
In Reinforcement Learning an agent will learn from the
environment by interacting with it and receiving rewards for
performing actions.
M a c h i n e L e a r n i n g Ty p e s
 Why Learn?
 There is no need to “learn” if it's a computing problem

 Learning is used when:

 Human expertise does not exist (navigating on Mars)

 Humans are unable to explain their expertise (speech

recognition) (God’s Gift) (Human with supernatural power)

 Solution changes in time (routing on a computer

network, driving a car)

 Solution needs to be adapted to particular cases

(recommendations of movies or books on Netflix or Amazon)
 GIS is Ideal for ML
Example Use Cases:
• 1. Land classification (vegetation monitoring, growth, decline, change)
• 2. Impervious surface
• 3. Change detection/anomaly
• 4. Geosptatial attribute trending (census, twitter)
• 5. Agriculture
• 6. Road networks
• 7. Object identification & tracking (ships, cars)
• 8. Imagery mosaicing, stitching, pre-processing
• 9. Resolution enhancement
• 10. 3D modeling & Digital Elevation/Surface Mapping
• 11. Coastal vegetation monitoring
• 12. Kriging
 Examples of Machine Learning in ArcGIS
Prediction
• Prediction is about using the known to estimate the
unknown. ArcGIS includes a number of regression and
interpolation techniques that can be used to perform
prediction. Applications include creating an air pollution
surface based on sensor measurements and estimating home
values based on recent sales data and related home and
community characteristics.
• In ArcGIS: Empirical Bayesian Kriging, Areal Interpolation, EBK
Regression Prediction, Ordinary Least Squares Regression and
Exploratory Regression, Geographically Weighted Regression
 Examples of Machine Learning in ArcGIS

Classification
• Classification is the process of deciding to which
category an object should be assigned based on a
training dataset. ArcGIS includes many classification
methods focused on remotely sensed data. These
tools analyze pixel values and configurations to
categorize pixels. Some examples include delineating
land use types or identifying areas of forest loss.
• In ArcGIS: Maximum Likelihood Classification,
Random Trees, Support Vector Machine
 Examples of Machine Learning in ArcGIS
Clustering
• Clustering is the grouping of observations based on similarities
of values or locations. ArcGIS includes a broad range of
algorithms that find clusters based on one or many attributes,
location, or a combination of both. These methods can be
used to do analysis such as segment school districts based on
socioeconomic and demographic characteristics or find areas
with dense social media activity after a natural disaster.
• In ArcGIS: Spatially Constrained Multivariate Clustering,
Multivariate Clustering, Density-based Clustering, Image
Segmentation, Hot Spot Analysis, Cluster and Outlier Analysis,
Space Time Pattern Mining
 Integration of Machine Learning and Deep
Learning with GIS – the new paradigm
• Machine Learning, Deep Learning and AI are increasingly being used
along with GIS for a number of purposes.
• Integrating Machine Learning algorithms with ArcGIS provides better and
more optimum results in less time.
• Satellite images are of different resolution and implementing it
successfully is not at all easy. Earlier it took months for reaching the final
output. But now due to fast-paced innovations, it takes just one day. But
for working with these solutions multiple libraries and project platforms
are needed and not all of them support Machine Learning and Deep
Learning algorithms and applications.
• Esri platform allows a solution-centric approach. AI has seen considerable
use in Computer vision and Language Processing.
• Esri cloud also provides the necessary infrastructure for Machine Learning
and Deep Learning.
 Integration of Machine Learning and Deep
Learning with GIS – the new paradigm
• Esri platform allows a solution-centric approach. AI has seen
considerable use in Computer vision and Language Processing. Esri
cloud also provides the necessary infrastructure for Machine Learning
and Deep Learning.
• “Other than map data, there is also IoT. There are weather sensors and
camera sensors. A lot of these challenges are solved by Machine
Leaning. People nowadays are expecting unprecedented precision. All
of this is converging towards GeoAI as location component is very
smart”, said Abhay Swarup Mittal, CEO, Skymap Global, at Esri UC Delhi
2018.
• In GeoAI, data capturing tool is enhanced by building footprint, road
detection. Geospatial analysis and processing tools are enhanced by
intelligent decision making support in presence of uncertainty and
prediction models of sophisticated phenomenon.
Thank you