Lectures: Tuesdays, & Thursdays 12:30:1:50 THO 125 or (when necessary) on zoom

Office hours: Monday 2:30-3:30 (may change by +/-30 min -- )

TA: Vydhourie Thiageswaran

Course web page: http://www.stat.washington.edu/mmp/courses/535/fall23 will be used to post lecture notes prior to lecture, and homeworks. The resources page is useful for addional reading. All other materials will be posted on Canvas. Everything posted here that is directly relevant to the learning objectives will be linked to from Canvas, hence you won't need to visit the web site once you are registered.

Class mailing list: stat535a_au23 at UW will be used sparingly by me mainly for announcements, but is open for posting by everyone on the class mailing list.

Learning Goals

• Mastery of machine learning concepts: supervised/unsupervised/reinforcement learning, training/testing/generalization/Bayes error, loss functions, bias-variance trade-off, overfitting, curse of dimensionality
• Working knowledge of the common families of predictors for regression and classification: linear and logistic regression, nearest neighbor classifiers, regression and decision trees, neural networks.
• Working knowledge of paradigms and algorithms for unsupervised learning: density estimation and clustering.
• Understanding and ability to apply model selection and uncertainty quantification methods such as Cross-Validation, AIC, BIC
• Basic knowledge of the training algorithms in Machine Learning.
• Ability to analyze the computational complexity of training and predicting with a machine learning model
• Ability to implement basic machine learning algorithms
• Ability to express formally in writing and to comprehend formally written ML results
• Ability to apply the knowledge gained in a data analysis project.

Syllabus

Supervised Learning (Prediction)
• Predictor examples
• Basic concepts: decision region, loss function, generative vs discriminative, bias-variance tradeoff
• Training predictors: gradient descent, stochastic gradient
• [Combining predictors: bagging, boosting, additive models]
• Regularized predictors: model selection, support vector machines, L1 regularization, the double descent phenomenon
• Learning theory and model selection basics

  Unsupervised Learning

• [Density estimation: mixture models, kernel density estimation]
• Clustering: parametric, [non-parametric]; model-based and cost-based
• Model selection in clustering
• [Graphical models intro],[Non-linear dimension reduction and geometric learning]I[Semi-supervised learning],[Reinforcement Learning]

Grading:The grade is based on homework + quizzes (60-70%), project (15-25%), class participation (5-10%). These percentages are approximative, and may be refined later in the quarter.
The grading policies described here apply to students in good standing; students who engage in misconduct will be reported to the office of Community Standards & Student Conduct. Their grades may be handled differently.

• Assignments: Typically assignments will be posted on the web on Thursday evening and will be due the next Thursday at the beginnining of class. More generally, homework will be due one week after it is posted, and we will post a homework approximately every week. Graded homeworks and solutions will be available about a week after the homework due date.
  Submit each homework as a single .pdf file through Canvas.
  The assignments will consist of (1) programming assignments (typically, to implement a version or a special case of an algorithm presented in the lecture) to be done in the programming language of your choice and (2) problems or other questions, including proofs. The programming assignments will be split into two separate parts:
  • Output and results are the only part of the programming assignment that is graded; they must be included in the single .pdf file that you submit.
  • The code, which is submitted electronically, and separately from the homework pdf. You can submit the code in one or more files, a zip archive, or any usual and convenient format. Code is typically not graded.

    With the exception of generic libraries (like plotting, matrix functions) you must write your own code. In particular, you are not to use matlab, R or python code for machine learning that is available on the web or with the textbook(s).

  Late homeworks will be accepted in exceptional circumstances. Please let us know in advance if you think you will be late.
  Teamwork: Each class participant submits her/his homework individually. Unless explicitly allowed to do so, you are required to write your own code. Discussing homework questions is acceptable as long as hints or solutions are not asked for or given; for example, a discussion to clarify what a question requires, on the discussion board or elsewhere, is acceptable.
• Quizzes will be announced. A quiz will take 10-15 minutes at the beginning of class; . We cannot give the quizzes on-line, but we are considering ways to compensate if a student misses a quiz. We are also planning to allow a good response to Q1 to replace part of a homework, and will announce these changes in advance of the respective quiz and homework.
• Participation What counts as participation? During lectures: Ask questions in class (chat OK if class on-line). Reply to the questions I pose during lectures.
  On the discussion board: Answer questions on the discussion board which are marked as counting for participation asked by instructor, or TA, or other students. Note: discussion about the homework typically does not qualify for participation, but there are exceptions. For example, if you find an error in the homework and are the first to point it out to us, congratulations! we consider that participation.
  How much participation is enough? Once a week on average, either in class or on discussion board, is plenty.
  Note that class attendance by itself is not graded as participation, and is not required.
• Project Each project will have the same training set as starting point. You will perform a prediction task on this data set. You will submit a report describing what you did, submit your code, and test your trained predictor(s) on a test set with hidden labels. Evaluation of the project is based partly on the report and partly on the test set results. Time permitting, we will have short presentations/Q\&A during the last lecture time. More details about the project will be announced around November 1.

What will the course be about?
The class will teach the basic principles of Machine Learning, and in particular will highlight the intimate connection between statistics and computation (meaning algorithms, data structures, and optimization) in modeling large or high-dimensional data. Solutions that are algorithmically elegant, often end up being also statistically sound, and sometimes when the model estimation program runs fast, we find that the model fits the data well. These principles will be illustrated during the study of a variety of models, problems and methods..

Who is this class for?
This class is a core class in the Machine Learning/Big Data PhD Track in Statistics. For any Statistics student who wants to learn Machine Learning/Big Data, this class is the fist in the triplet of graduate courses 535 --> 538/548 and serves as a prerequisite to
STAT 538 Advanced Machine Learning (Winter), and
STAT 548 Machine Learning for Big Data (Winter)

Prerequisites

• A course in probability, including basic multivariate analysis (conditional probability, independence, marginals, expectation, variance in multivariate seeting) at the level of e.g. STAT/MATH 394-5.
• Fundamentals of statistics: Maximum Likelihood Estimation, MAP estimation, priors, likelihood, estimating parameters of usual distributions (normal, multinomial), Bayes' formula.
• Calculus and linear algebra: partial derivatives, gradient, the chain rule, vectors and matrices, matrix multiplications, eigenvalues and eigenvectors, positive definite matrices.
• Algorithms and data structure at a basic level (arrays, lists, sets, O( ) notation).
• Medium (beyond beginner) ability with a computer programming language (like python, C, C++, Java or Matlab, Splus, R) at the level of STAT 534.

Notice on Zoom class activities We do not plan to record the office hours, or other class activities thay may take place on zoom, with the exception of lectures. If we decide that the learning goals are better achieved by recording, we will announce it in advance. Any recordings we may make will only be accessible to students enrolled in the course to review materials. These recordings will not be shared with or accessible to the public.

The University and Zoom have FERPA-compliant agreements in place to protect the security and privacy of UW Zoom accounts.

Staying safe from infectious diseases during the in-person lectures; read the information here (TB posted). Updates will be made as circumstances change and our experience grows.

Religious accomodation Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or fororganized religious activities. The UW’s policy, including more information about how to requestan accommodation, is available at Religious Accommodations Policy. Accommodations must berequested within the first two weeks of this course using the Religious Accommodations Request Form.