STAT535: Foundations of Machine Learning (2017)

This is a 10-week course focused on introducing foundations of machine learning from philosophical, methodological, and theoretical perspectives. Physically, this is a course fully focused on "supervised learning", or even more narrowly, about "classification". Metaphysically, stemmed from this seemingly simple task, our ultimate goal is to appreciate and celebrate the statistical thinking.

Check the Syllabus for detailed course plan.


Instructor: Fang Han (

TA: Kunhui Zhang (

Lectures: TTH 10:00-11:20, in SMI 405

Office hour: Fang (Tu, 3:30-4:30PM; PDL B-308); Kunhui (Wed & Thu, 3-4PM; PDL B-302)

Midterm: 11/07 (Tu), in-class

Lecture notes:

Lecture note 1

technical supplementary to Lecture note 1

Lecture note 2

Lecture note 3

Homework assigments:

Homework 1 (Due Date: 10/15).

Homework 2 (Due Date: 10/29).

Homework 3 (Due Date: 11/20).

Homework 4 (Due Date: 12/08).

Final project:

The final project is on your own (teamwork is not allowed), and can be about any topic related to the course (machine learning theory, method, or application). Possible topics include but are not limited to:

1. Picking an area that interests you, and writing a survey overview of a related set of books/papers (e.g., metric entropy calculation, generic chaining, oracle inequality, neural network, support vector machine, kernel methods, spectral methods, ...).

2. Extending an existing theoretical or methodological result.

3. Implementing a classification algorithm and applying it to study a real/simulated data.

The project will be evaluated by your written report (no more than 20 pages). The written report will be due by Dec. 10, and no free day is allowed to be used (reasons to be discussed in-class :) ).

Suggested readings

Probability Tools:

Vershynin's note

Duchi's note

One hundred probability inequalities

Appendix A in "Empirical Processes with Applications to Statistics" (2009) by Galen Shorack and Jon Wellner

Statistical Learning Theory:

A Probabilistic Theory of Pattern Recognition (1997) by Luc Devroye, Laszlo Gyorfi, and Gabor Lugosi

Convex Analysis:

Appendix A in Duchi's Information Theory note


Marina's STAT535 (2015) note

Larry Wasserman's introductionary note of function space