Introduction to

Machine Learning
What is Learning?
• Learning is defined as modification in behaviour through experience
• ”Learning is any process by which a system improves performance
from experience.” - Herbert Simon
• It is relatively permanent change that is result from experience.
• Learning modifies the agent's decision mechanisms to improve
performance
Types of Learning
• Induction vs deduction
• Rote learning (memorization)
• Advice or instructional learning
• Learning by example or practice
• Most popular; many applications
• Learning by analogy; transfer learning
• Discovery learning
• Others?
Inductive Vs Deductive
• Inductive Learning
• Inductive reasoning is a bottom-up
approach
• Inductive reasoning takes you from the
specific to the general
• Deductive Learning
• Top Down Approach
• deductive reasoning, you make
inferences by going from general
premises to specific conclusions.
Machine Learning
• ML is a branch of artificial intelligence: It focuses on the use of data
and algorithms to imitate the way that humans learn, gradually
improving its accuracy.
• Uses computing based systems to make sense out of data
• Extracting patterns, fitting data to functions, classifying data, etc
• ML systems can learn and improve
• With historical data, time and experience
• Bridges theoretical computer science and real noise data.
• Learning by Observations
Formal Definition- Machine Learning
 Definition by Tom Mitchell (1998): Machine Learning is the study of
algorithms that
o improve their performance P
o at some task T
o with experience E.
o A well-defined learning task is given by <P, T, E> .
Traditional Programming Vs Machine
Learning
Traditional Programming

Data
Computer Output
Program
Machine Learning

Data
Computer Program
Output
Relationship : AI & ML & Deep
Learning
Related Disciplines
• Artificial Intelligence
• Data Mining
• Probability and Statistics
• Information theory
• Numerical optimization
• Computational complexity theory
• Control theory (adaptive)
• Psychology (developmental, cognitive)
• Neurobiology
• Linguistics
• Philosophy
10
History of Machine Learning
 1943 — The first neutral network with electric circuit was developed by
Warren McCulloch and Walter Pitts in.
 1950s-1960s —
o 1950 —The Turing Test is a test of artificial intelligence proposed by
mathematician Alan Turing.
o 1952-59 — Arthur Samuel wrote a learning-based checkers program (using
alpha-beta pruning) that could defeat him.
o 1957 — Perceptron algorithm (implemented as a circuit!). Frank Rosenblatt
was a psychologist who is most famous for his work on machine learning
o Pattern recognition
o 1969 — Minsky and Papert’s book Perceptrons (limitations of linear
models)
History of Machine Learning
 1970s —
o 1974- Backpropagation
o 1979- The Stanford Cart (a remote-controlled robot that can move
independently in space)
o AI Winter -The low point for AI was known as the AI winter, which
happened in the late 70s to the 90s.
History of Machine Learning
 1980s — Some foundational ideas
o Decision Trees and rule learning
o Connectionist psychologists explored neural models of cognition
o 1984 — Leslie Valiant formalized the problem of learning as PAC learning
o 1988 — Backpropagation (re-)discovered by Geoffrey Hinton and
colleagues
o 1988 — Judea Pearl’s book Probabilistic Reasoning in Intelligent Systems
introduced Bayesian networks
History of Machine Learning
o 1990s —90s were also sort of o variational inference
a golden age for ML research o kernels and support
( ML and Statistics) vector machines
o Support Vector Machines
o boosting
o Data Mining
o convolutional networks
o Adaptive Learning
o Text Learning
o reinforcement learning
o Ensembles o 1994 – Self driving cars
o Bays Net o 1997- Deep Blue beats
o Markov chain Monte Carlo Garry Kasparov
History of Machine Learning
 2000s — applied AI fields (vision, NLP, etc.) adopted ML
o 2009 -Google build self-driving cars
 2010s — deep learning
o 2011- Watson wins jeopardy
o 2010–2012 — neural nets smashed previous records in speech-to-text and object
recognition increasing adoption by the tech industry
o 2014- Machine vision surpass human vision
o 2016 — AlphaGo defeated the human Go champion
o 2018-now — generating photorealistic images and videos
o 2020 — GPT3 language model
 now — increasing attention to ethical and societal implications
Application areas of Machine
Learning
References
• Textbooks and other suggestive readings
Required Reading
 TB 1: Tom Mitchell (1997).Machine Learning, McGraw Hill.
 TB 2: S. Rogers and M. Girolami (2011). A first course in Machine Learning, CRC Press.
Recommended Readings
 RB 1: C. Bishop(2007). Pattern Recognition and Machine Learning,Springer
 RB 2: D. Barber(2012). Bayesian Reasoning and Machine Learning. Cambridge
university press.
 RB 3: Duda, Hart and Stork( 2001).Pattern Classification, Wiley.
Journals /Web references
 Journal of Machine Learning Research (JMLR) - Website: http://www.jmlr.org/
 IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)- Website:
https://www.computer.org/csdl/journal/tp
 Machine Learning- Website: https://www.springer.com/journal/10994
 Journal of Artificial Intelligence Research (JAIR)- Website: http://www.jair.org/
 International Journal of Machine Learning and Cybernetics- Website: