CIS 472/572         Machine Learning

Course Description

Machine learning is about building predictive or descriptive models automatically from data. There are two central challenges in machine learning. The first is generalizing from data to future situations. A model that performs very well on past data may nonetheless perform very poorly on unseen examples, a phenomenon known as "overfitting". The second challenge is building models efficiently, especially in cases where the dataset is very large and the patterns are complex. We will cover standard methodologies, models, and algorithms for machine learning. Specific topics include decision trees, instance-based learning, linear classifiers, probabilistic classifiers, support vector machines, deep learning, model ensembles, and learning theory.

Instructor

Thien Huu Nguyen, [email protected]

Lectures

Prerequisites

• CIS 315 - Intermediate Algorithms

Textbooks and Readings

• CIML:

  Hal Daume III, A Course in Machine Learning 0.99, 2015 (Online!)

• Mitchell:

  Tom Mitchell, Machine Learning, 1997.

• Murphy:

  Kevin Murphy, Machine Learning: A Probabilistic Perspective, 2012.

• ESL:

  Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, 2009 (Online!)

• DL:

  Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, 2016 (Online!)

• Domingos:

  Pedro Domingos’ video lectures on YouTube

Major Topics

• Machine learning introduction, decision trees: 2 lectures
• Inductive learning and nearest neighbor: 2 lectures
• Perceptron, linear regression, logistic regression: 2 lectures
• Kernel methods, SVMs: 2 lectures
• Neural Networks: 2 lectures
• Deep Learning : 4 lectures
• Convolutional Neural Networks: 2 lectures
• Optimization and Initialization, Recurrent Neural Networks: 2 lectures

Expected Learning Outcomes

This course covers standard methodologies, models, and algorithms for machine learning. Specific topics include decision trees, instance-based learning, linear classifiers, probabilistic classifiers, support vector machines, model ensembles, and learning theory. Especially, we will spend a large amount of time for deep learning, the recent approach to machine learning that has achieved very high performance for tasks in different application domains (e.g., computer vision, natural language processing).

Upon successful completion of the course, students will be able to:

• demonstrate facility with the cycle of machine learning model development, including training, developing, and testing;
• demonstrate facility with the fundamental concepts and problems in machine learning, including (hyper-)parameters, loss functions, overfitting, underfitting, cross-validation, regularization, evaluation measures;
• demonstrate facility with different types of learning, i.e., supervised learning, semi-supervised learning, and unsupervised learning;
• demonstrate facility with traditional machine learning methods, i.e., decision trees, nearest neighbor, perceptron, linear and logistic regression, and support vector machines;
• demonstrate facility with the basic concepts in deep learning, i.e., feed-forward networks, activation functions, computational graphs, back-propagation, and (stochastic) gradient decent;
• demonstrate facility with advanced deep learning models, including convolutional neural networks, recurrent neural networks, residual models, initialization, optimization, and their applications to different types of data;

Acquired Skills

Upon successful completion of the course, students will have acquired the following skills:

• be able to identify machine learning problems and design appropriate features and models to solve those problems;
• be able to leverage available machine learning frameworks (pytorch, scikit-learn) to implement models and apply them for data of interested domains;
• be able to analyze the operation/outputs of basic machine learning models for debugging and performance improvement purposes.

Tentative Schedule

Assignments

• Assignment 1 (written): Link (posted on April 7), due date: April 15 at 11:55pm.

• Assignment 2 (programming): Link (posted on April 16), due date: April 29 at 11:55pm.

• Assignment 3 (programming): Link (posted on April 30), due date: May 12 at 11:55pm.

• Assignment 4 (programming): Link (posted on May 23), due date: June 3 at 11:55pm.

Projects

• Instructions for project report: Link
• Sample project report: Link

• Final Project Proposal Due: May 7 (11:55 pm)

• Final paper due: Finals week (June 10, 11:55pm)

• Helpful links

  • Kaggle ML competitions

  • Stanford ML projects (also look at the previous years)

  • Stanford ML project ideas

  • UCI ML repository

  • scikit-learn

  • Weka

Supplementary Materials

• Pedro Domingos’ Machine Learning course on Coursera

• CMU 10-701, Tom Mitchell

• You are encouraged to explore deep learning with PyTorch in this class.

Course Requirements and Grading

This REMOTE course will be taught entirely using Zoom, Canvas, and Piazza.

Grading will be based on the following criteria:

Grading Scale