Syllabus

Supervised Learning (Prediction)
• Predictor examples
• Basic concepts: decision region, loss function, generative vs discriminative, bias-variance tradeoff
• Training predictors: gradient descent, [Newton method]
• [Combining predictors: bagging, boosting, additive models]
• Regularized predictors: model selection, support vector machines, L1 regularization
• Learning theory and model selection basics

  Unsupervised Learning

• Density estimation: mixture models, kernel density estimation
• Clustering: parametric, non-parametric; model-based and cost-based
• Model selection in clustering
• [Graphical models intro],[Non-linear dimension reduction and geometric learning]I[Semi-supervised learning],[Reinforcement Learning]

Grading:The grade is based on homework + quizzes (65-70%), project (25-30%), and class participation (5%). These percentages are approximative, and may change by up to 5%.
The grading policies described here apply to students in good standing; students who engage in misconduct will be reported to the office of Community Standards & Student Conduct. Their grades will be handled differently.

• Assignments: Typically assignments will be posted on the web on Tuesdays and will be due the next Tuesday at the beginnining of class. More generally, homework will be due one week after it is posted, and we will post a homework approximately every week. Graded homeworks and solutions will be available about a week after the homework due date.
  Submit each homework as a single .pdf file through Canvas.
  The assignments will consist of (1) short programming assignments (typically, to implement a version or a special case of an algorithm presented in the lecture) to be done in the programming language of your choice and (2) problems or other questions, including proofs. The programming assignments will be split into two separate parts:
  • Output and results are the only part of the programming assignment that is graded; they must be included in the single .pdf file that you submit.
  • The code, which is submitted electronically, and separately from the homework pdf. You can submit the code in one or more files, a zip archive, or any usual and convenient format. Code is typically not graded.

    With the exception of generic libraries (like plotting, matrix functions) you must write your own code. In particular, you are not to use the matlab, R or python code that is made availabe with the textbook(s).

  Late homeworks will be accepted in exceptional circumstances. Please let us know in advance if you think you will be late.
  Teamwork: Each class participant submits her/his homework individually. Unless explicitly allowed to do so, you are required to write your own code. Discussing homework questions is acceptable as long as hints or solutions are not asked for or given; for example, a discussion to clarify what a question requires, on the discussion board or elsewhere, is acceptable.
• Quizzes I will be exploring together with you the possibility of having on-line quizzes. If we do, all quizzes will be announced, will take 10-15 minutes at the beginning of class, and the grades for them will be lumped together with the homework part of the grade.
• To earn your Participation part of the grade During lectures: Ask questions in class (chat OK). Reply to the questions I pose during lectures (chat OK).
  On the discussion board: Answer questions asked by others on course topics. Replies to homework related questions: you can answer to a question asking for clarification, but not to a "how to solve this" question. If your answer is correct then it is considered. If you find an error in the homework or project and are the first to point it out to us, congratulations! we consider that participation.
  How much participation is enough? Once a week on average, either in class or on discussion board, is plenty.
• Project Each project will have the same training set as starting point. You will perform a prediction task on this data set. You will submit a report describing what you did, submit your code, and test your trained predictor(s) on a test set with hidden labels. Evaluation of the project is based partly on the report and partly on the test set results. Time permitting, we will have short presentations/Q\&A during the last lecture time. More details about the project will be announced around November 1.

What will the course be about?
The class will teach the basic principles of Machine Learning, and in particular will highlight the intimate connection between statistics and computation (meaning algorithms, data structures, and optimization) in modeling large or high-dimensional data. Solutions that are algorithmically elegant, often end up being also statistically sound, and sometimes when the model estimation program runs fast, we find that the model fits the data well. These principles will be illustrated during the study of a variety of models, problems and methods..

Who is this class for?
This class is a core class in the Machine Learning/Big Data PhD Track in Statistics. For any Statistics student who wants to learn Machine Learning/Big Data, this class is the fist in the triplet of graduate courses 535 --> 538/548 and serves as a prerequisite to
STAT 538 Advanced Machine Learning (taught in Winter), and
STAT 548 Machine Learning for Big Data

Prerequisites

• A course in probability, including basic multivariate analysis (conditional probability, independence, marginals, expectation, variance in multivariate seeting) at the level of e.g. STAT/MATH 394-5.
• Fundamentals of statistics: Maximum Likelihood Estimation, MAP estimation, priors, likelihood, estimating parameters of usual distributions (normal, multinomial), Bayes' formula.
• Calculus and linear algebra: partial derivatives, gradient, the chain rule, vectors and matrices, matrix multiplications, eigenvalues and eigenvectors, positive definite matrices.
• Algorithms and data structure at a basic level (arrays, lists, sets, O( ) notation).
• Medium (beyond beginner) ability with a computer programming language (like python, C, C++, Java or Matlab, Splus, R) at the level of STAT 534.

Notice to Students - Zoom Recordings This course is scheduled to run synchronously at your scheduled class time via Zoom. These Zoom class sessions will be recorded. The recording will capture the presenter’s audio, video and computer screen. Student audio and video and chat will be recorded if they share their computer audio and video during the recorded session. The recordings will only be accessible to students enrolled in the course to review materials. These recordings will not be shared with or accessible to the public.

The University and Zoom have FERPA-compliant agreements in place to protect the security and privacy of UW Zoom accounts. Students who do not wish to be recorded should:

• Change their Zoom screen name to hide any personal identifying information like their name or UW Net ID; and
• Not share their computer audio or video during their Zoom sessions

Religious accomodation Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or fororganized religious activities. The UW’s policy, including more information about how to requestan accommodation, is available at Religious Accommodations Policy. Accommodations must berequested within the first two weeks of this course using the Religious Accommodations Request Form.