This is a 10-week course focused on introducing foundations of machine learning from philosophical, methodological, and theoretical perspectives. Physically, this is a course fully focused on "supervised learning", or even more narrowly, about "classification". Metaphysically, stemmed from this seemingly simple task, our ultimate goal is to appreciate and celebrate the statistical thinking.
STAT535: Foundations of Machine Learning (2017)
Check the Syllabus for detailed course plan.
Instructor: Fang Han (email@example.com)
TA: Kunhui Zhang (firstname.lastname@example.org)
Lectures: TTH 10:00-11:20, in SMI 405
Office hour: Fang (Tu, 3:30-4:30PM; PDL B-308); Kunhui (Wed & Thu, 3-4PM; PDL B-302)
Midterm: 11/07 (Tu), in-class
Lecture note 1technical supplementary to Lecture note 1
Lecture note 2
Lecture note 3
Homework 1 (Due Date: 10/15).
Homework 2 (Due Date: 10/29).
Homework 3 (Due Date: 11/20).
Homework 4 (Due Date: 12/08).
The final project is on your own (teamwork is not allowed), and can be about any topic related to the course (machine learning theory, method, or application). Possible topics include but are not limited to:
1. Picking an area that interests you, and writing a survey overview of a related set of books/papers (e.g., metric entropy calculation, generic chaining, oracle inequality, neural network, support vector machine, kernel methods, spectral methods, ...).
2. Extending an existing theoretical or methodological result.
3. Implementing a classification algorithm and applying it to study a real/simulated data.The project will be evaluated by your written report (no more than 20 pages). The written report will be due by Dec. 10, and no free day is allowed to be used (reasons to be discussed in-class :) ).
One hundred probability inequalities
Appendix A in "Empirical Processes with Applications to Statistics" (2009) by Galen Shorack and Jon Wellner
Statistical Learning Theory:
A Probabilistic Theory of Pattern Recognition (1997) by Luc Devroye, Laszlo Gyorfi, and Gabor Lugosi
Appendix A in Duchi's Information Theory note
Marina's STAT535 (2015) note
Larry Wasserman's introductionary note of function space