Scribd
Upload
263 views

Introduction To Data Science and Machine Learning

1. Data science involves extracting hidden patterns from large datasets using statistical tools and machine learning algorithms. 2. Machine learning is a key part of data science that uses algorithms to identify patterns in datasets and express them as models. 3. Supervised learning uses input data to train algorithms on relationships between independent and dependent variables, while unsupervised learning finds patterns and clusters in data without labeled examples.

Uploaded by

CHETAN V CHAUDHARI
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
263 views

Introduction To Data Science and Machine Learning

1. Data science involves extracting hidden patterns from large datasets using statistical tools and machine learning algorithms. 2. Machine learning is a key part of data science that uses algorithms to identify patterns in datasets and express them as models. 3. Supervised learning uses input data to train algorithms on relationships between independent and dependent variables, while unsupervised learning finds patterns and clusters in data without labeled examples.

Uploaded by

CHETAN V CHAUDHARI
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 23

Introduction to

Data Science and Machine Learning


What is Data Science?

1. Data Science is the science of extracting hidden patterns from


large data sets

2. Hidden patterns can appear in form of trends, cycles,


associations, rules, groups etc. in the data

3. Data sets usually refer to large volume of cleansed, structured


data prepared for the analysis

4. Science refers to the statistical tools and techniques employed


to understand the data and reliability of the identified patterns

a. That part of statistics which is used to understand the data is called


descriptive statistics. Descriptive statistics give vital insights into the
data in terms of central values, spread and distribution shape of the data
What is Machine Learning?

1. Machine Learning is an integral and critical part of data science. It


refers to a collection of algorithms which are used to extract the
hidden patterns from the dataset

2. These algorithms use a learning process through which they


identify the patterns in the dataset. The patterns they learn from the
data are called models

3. The models could be expressed in form of mathematical equations,


rules, probability ratios etc.

4. Machine learning algorithms work on the data prepared for


analytics to express the hidden patterns in form of models

5. For machine learning algorithms to successfully identify reliable


When is machine learning useful ?

1. Cannot express our knowledge about patterns as a


program. For e.g. Character recognition or natural
language processing

2. Do not have an algorithm to identify a pattern of


interest. For e.g. In spam mail detection

3. Too complex and dynamic. For e.g. Weather


forecasting

4. No prior experience or knowledge. For e.g. Mars rover


Machine Learning Applications (examples)

1. Fraud detection

2. Sentiment analysis

3. Credit risk management

4. Prediction of equipment failures

5. New pricing models / strategies

6. Network intrusion detection

7. Pattern and image recognition

8. Email spam filtering


Machine Learning Pre-requisites
1. Rich set of data representing the environment where
the model is to be used

2. Knowledge and skills in


a. Mathematics and statistics (graduate level or more)
b. Programming in any language such as Python or R
(considered as the two most popular languages for
data science)
c. Domain knowledge

3. Usually data science is a team effort where the team


Real World as Mathematical Space
Machine learning happens in mathematical space / feature
space:

1. A data set representing the real world, is a collection


attributes that define an entity

2. Each entity is represented as one record / line in the data set


Attributes / Dimensions
Machine learning happens in mathematical space / feature
space:

1. Each attribute
becomes a
dimension

Sugar
2. Each record
becomes a point in
the space

e
Ag BP level
Heart healthy
Potential heart ailments
Machine learning happens in mathematical space / feature
space:

1. Position of a
point in space is
defined with
respect to the
origin

Sugar
2. The position is

e
decided by the
values of the Ag
BP level
attributes for a
Heart healthy
point Potential heart ailments
Machine learning happens in mathematical space / feature
space:

3. A model represents the


real world process that
generated the different
set of data points

4. The model could be a


simple plane, complex
plane, hyper plane

Sugar
5. But multiple planes can
do the job. Each

e
Ag
representing an alternate
hypothesis
BP level
Heart healthy
6. The learning algorithm Erroneous classification
Potential heart ailments
selects that hypothesis
Machine learning happens in mathematical space / feature
space:

7. In the figure, since


the separator is a
plane, the model
will
ax be the+equation
+ by cz = d
representing the
plane

Sugar

e
8. x , y, z represent the Ag
three dimensions BP level
i.e. BP, Age, Sugar
Heart healthy
Potential heart ailments
while d represents
Machine learning happens in mathematical space / feature
space:

9. A new data point


enters the system

10.It’s x,y and z


values will be fed
into the model to

Sugar
get value of d
(healthy or ailing)

e
Ag
11.The data point will
ax + byabove
be placed + cz or
= d, BP level

below the plane Heart healthy


Potential heart ailments
based on d
Machine learning happens in mathematical space / feature
space:

12.Whether the new


data point is
correctly placed
(above or below the
plane) i.e. correctly
classified as ailing

Sugar
or healthy hear will
be known only after
direct observation

e
Ag
ax + by + cz = d, BP level

Heart healthy
Potential heart ailments
Machine learning happens in mathematical space / feature
space:

13.Only direct test on the


object of interest will
tell whether the
classification is correct
or not

ax + by + cz = d,

Sugar

e
Ag

BP level
15.If majority of new data Heart healthy
Potential heart ailments
points are correctly
classified, the model is
Machine Learning Categories
Machine learning categories:

There are broadly three categories into which the


machine learning algorithms are grouped

1.Supervised Learning

2.Unsupervised Learning

3.Reinforcement Learning
Supervised Machine Learning:

1. Class of algorithms which work in two stages. The first stage is called
training and second one is usually called testing. Sometimes it may
involve validation stage followed by testing

2. At each stage it takes input data prepared for that stage. i.e. for training
data for training stage, test data for test stage, validation data for validation
stage

3. During training, the machine learning algorithm gets the training data
inform of independent and dependent variables

4. In the process of learning, the algorithm learns the relationship between


the dependent and the independent variables

5. This relationship is expressed as a model which can take the form of a


equation, probability ratios, hidden rules etc.
Examples of Supervised Machine Learning:

1.Regression - Predicting mileage of a car given the other features such as


weight, engine capacity, horse power, transmission type, number of cylinders
etc.

a. In this example, mileage is the dependent variable and weight, engine


capacity, horse power, transmission type, number of cylinders are
independent variables

b. Mileage = f ( weight, engine capacity, horsepower, transmission type,


number of cylinders)

2.Classification – Categorizing a mail into spam or ham


a. In this example, the email category (spam or ham) is the target
variable and the occurrences of certain words and their frequency are
independent variables

b. P(ham) = f( words, frequencies) where P stands for probability. 1-


Unsupervised Machine Learning:
1. Class of algorithms which work in a single stage. Unlike supervised
learning algorithms, it does not have a separate training, testing or
validation stage

2. Unsupervised learning algorithms take the data as a whole, not in form of


independent and dependent variables.

3. The algorithms are not used to find any relationship between dependent
and independent variables

4. This class of algorithms usually find patterns in form of clusters and


associations reflecting some kind of commonality, togetherness among the
data points in the given data sets

5. It is the responsibility of the data scientist to analyse the identified


clusters/associations and give meaning to those clusters

6. Clustering and PCA (Principal Component Analysis), a mathematical


Examples of Unsupervised Machine Learning:

1.Clustering - Identifying groups in the given data set where a


group represents some kind of commonality among the data
points. “Birds of same feather, flock together”.

2.Clustering can further be categorized into –

a. Flat clustering, e.g. Kmeans clustering- The clusters


identified are disjoint, non-overlapping. For e.g.
segmenting customers into different groups based on their
purchase amount, frequency of purchase and types of
items purchase

b. Hierarchical clustering, clustering is done at multiple


levels indicating clusters inside clusters indicating some
Reinforced Machine Learning:

1. Reinforcement learning algorithm learns


through trial and error and the feedback it
receives from the environment in which it
learns.

2. During the initial stages of learning, it is likely


to commit many errors in learning the
patterns, however, through a process of reward
and punishment, it learns to identify the
patterns correctly.
Thank You

You might also like