Introduction, course logistics

CS771: Introduction to Machine Learning

Nisheeth Srivastava
 2
Course Logistics
 Course Name: Introduction to Machine Learning – CS771
 An introductory course – supposed to be your first intro to the subject

 3 lectures every week on Zoom, MWF 1800-1915 (link)

 Classes will be recorded and uploaded to mooKIT for later viewing
 mooKIT URL: https://hello.iitk.ac.in/cs771a/ (CC id and password to be used for login)

 Friday lecture will double as an interactive discussion session

 All material will be posted on the mooKIT page for the course

 Q/A and announcements on mooKIT

CS771: Intro to ML
 Course Team
 Abhas ([email protected])
 Abhishek Jaiswal ([email protected])
 Abhishek Krishna ([email protected])
 Avideep ([email protected])
 Dhanajit ([email protected])
 Jay ([email protected])

3
CS771: Intro to ML
 Course Team
 Mahesh ([email protected])
 Mosab ([email protected])
 Neeraj ([email protected])
 Rahul ([email protected])
 Yatin ([email protected])
 Nisheeth ([email protected])

4
CS771: Intro to ML
 5
Workload and Grading Policy
 4 homework assignments (theory + programming) worth 60%
 Theory part: Derivations/analysis
 Programming part: Implement/use ML algos, analysis of results. Must be done in
Python (learn if not already familiar)
 Must be typeset in LaTeX (learn if not already familiar)
 To be submitted via Gradescope (login details will be provided)
 Will also involve a viva
 Quizzes and exams (mid-sem and end-sem) worth 40%
 Will be held online – details later
 Quizzes will be worth 5% of the total grade
 Mid-sem and end-sem will be worth 10, 20% each of the total grade

CS771: Intro to ML
 6
Reference materials

 Course slides and notebooks will mostly be sourced from previous

iterations of CS771
 No fixed textbook
 Sections from Bishop, Murphy, Stork, Hart & Duda will be assigned as reading
from time to time
 Different books vary in terms of
 Set of topics covered
 Flavor (e.g., classical statistics, deep learning, probabilistic/Bayesian, theory)
 Terminology and notation (beware of this especially)

CS771: Intro to ML
 7
Course Goals

Credit: Rishika Agarwal (EE, graduated 2017) CS771: Intro to ML

 8
Course Goals..
 Introduction to the foundations of machine learning models
 Focus on developing the ability to
 Understand the underlying principles behind ML models and algos
 Understand how to implement and evaluate them
 Understand/develop intuition on choosing the right ML model/algo for your problem
 Focus on how to interpret ML algorithms’ outputs in a rigorous manner
 Over-optimistic interpretation of ML has led to a reproducibility crisis in the field
 Graduates of this course should be able to separate truth from falsehood (and BS) in ML
evaluations

 Not an introduction to popular software frameworks and libraries, such as

scikit-learn, PyTorch, Tensorflow, etc
 Can explore once you have some understanding of various ML techniques
CS771: Intro to ML
What is Machine Learning

CS771: Intro to ML
 How do we classify?
 Let's say we want to classify the climate in a region
 We may say something like
• If the summer temperature is above 35 C for more than 20 days during May-
July, the climate is tropical
• If the total rainfall is less than 10 cm all year, the climate is desert
 Basic principles
• Think of simple rules that place thresholds on some measurable attributes of
the object
• Combine rules to maximize coverage of classification

CS771: Intro to ML
 Classification using expert advice
 Who should be screened for COVID-19?
 Attributes
1. History of international travel
2. Flu-like symptoms
3. Contact with anyone reporting 1 or 2
 Advantages:
• Simple to understand and critique
 Disadvantages:
• Hard to implement without data and actuators

CS771: Intro to ML
 Supervised classification using machine learning
Human Computer
 Instead of asking experts to generate
Triangle
rules
• Ask them to say 'wrong' or 'right' Triangle
whenever a computer program classifies
something Triangle
 Program is programmed to want to

Training
Square
make as few mistakes as possible
Revises its internal rules such that the

Testing 
 Wrong Triangle
expert has to say 'wrong' less often
Right Triangle
 Rules are learned using expert
supervision Right Triangle

CS771: Intro to ML
 Advantages of ML classification
 Rules are learned from data instead of generated by experts so
• Rules have a lower chance of being contaminated by an expert's bias
• Rules are not restricted in complexity
• Rules are more likely to be testable
• Rules are likely to have greater predictive accuracy
 But, for the same reason, 
• Rules have a higher chance of missing important aspects of the situation not
captured within data
• Rules can become over-complex and hard to interpret
• Rules become less likely to generalize well to new situations

CS771: Intro to ML
 14
Machine Learning (ML)
 Designing algorithms that use data to programmatically learn a model of it
 The learned model can be used to
 Detect patterns/structures/themes/trends etc. in the data
 Make predictions about future data and make decisions

 Modern ML algorithms are heavily “data-driven”

 No need to pre-define and hard-code all the rules (infeasible/impossible anyway)
 The rules are not “static”; can adapt as the ML algo ingests more and more data
CS771: Intro to ML
 15
ML: From What It Does to How It Does It?
 ML enables intelligent systems to be data-driven rather than rule-driven
 How: By supplying training data and building statistical models of data
 Pictorial illustration of an ML model for binary classification:

A Linear Classifier (the statistical model)

CS771: Intro to ML
 16
ML: From What It Does to How It Does It?
 ML enables intelligent systems to be data-driven rather than rule-driven
 How: By supplying training data and building statistical models of data
 Pictorial illustration of an ML model for binary classification:
A Probabilistic Classifier (the statistical model)
P(“cat”|image) P(“dog”|image)

CS771: Intro to ML
 17
Generalization
 The most accurate model for your training data is a lookup table

 A good ML model must generalize well on unseen (test data)

 To avoid approximating a lookup table for the training data
 Simpler models should be preferred over more complex ones!
 An article of religious faith
CS771: Intro to ML
 18
ML Applications Abound..

Picture courtesy: gizmodo.com,rcdronearena.com,www.wiseyak.com,www.charlesdong.com CS771: Intro to ML

 19
Key Enablers for Modern ML
 Availability of large amounts of data to train ML models

 Increased computing power (e.g., GPUs)

CS771: Intro to ML
 20
ML: Some Success Stories

CS771: Intro to ML
 21
ML: Some Success Stories

CS771: Intro to ML
 22
ML: Some Success Stories

Picture courtesy: https://news.microsoft.com/ CS771: Intro to ML

 23
ML: Some Success Stories
 Automatic Program Correction

Example from “Compilation error repair: for the student programs, from the student programs”, Ahmed et al (2018) CS771: Intro to ML
 ML: Some success stories

* From Pant, Singh & Srivastava (2021)

CS771: Intro to ML
 25
Politically correct ML Systems
 Good ML should not just be about getting high accuracies; they are now also expected to be
politically correct
 Should also ensure that the ML models are fair and unbiased

Criminals?

Not Criminals?

An image captioning Don’t want a self-driving car Don’t want a predictive policing
system should not always that is more likely to hit black system that predicts criminality
assume a specific gender people than white people using facial features

 A lot of recent focus on Fairness and Transparency of ML systems

Picture courtesy: Bhargava and Forsyth (2019), https://www.thestranger.com/, Xiaolin Wu and Xi Zhang, “Automated Inference on Criminality Using Face Images” CS771: Intro to ML
 Learning about ML
We will also focus on
these Data

Representations

Algorithms

Outputs

We will also focus on

these Interpretation

26
CS771: Intro to ML
 27
Looking Back Before We Start: History of ML

- Human-like text generators (GPT-3)

CS771: Intro to ML
 28
Next Class

 Various Flavors of ML problems

 Data and features

 Basic mathematical operations on data and features

CS771: Intro to ML