Syllabus
-
Introduction: what is statistical learning (1 lecture)
- supervised and unsupervised learning
- a little history
- the curse of dimensionality
- Basics: statistical inference and probabilistic independence (2 lectures)
- multivariate distributions, sample space and random variables
- operations: inference, sampling, maximum liklihood config, ..
- examples and applications
- Graphical Models (4 lectures)
- graphical representations of conditional independence
- directed and undirected graphical models
- Markov properties
- expression of joint distribution
- factor graphs
- log-linear models
- conditional independence; d-maps and i-maps
- explaining away; correlation and causation
- latent vs. observed variables
- entropy and mutual information as measures of edge strength
- Inference in graphical models (4 lectures)
- inference as summation over configurations
- using graph structure to simplify calculations
- variable elimination
- moral, chordal, and decomposable graphs; triangulation
- the junction-tree algorithm
- computational complexity, including tree width, cut-sets and
phase transitions
- Approximate inference and sampling (~3 lectures)
- belief propagation and message passing algorithms
- forward sampling and importance sampling
- Gibbs sampling, the Swendsen-Wang algorithm
- Model estimation (4 lectures)
- Parameter estimation: mutinomial distribtions, iterative proportional fitting
- Priors for parameters
- induction of tree graphs
- structure estimation
- Clustering (4 lectures)
- Model based clustering and the EM algorithm
- Clustering as optimization
- Clustering on graphs
- Cluster validation and finding the number of clusters
- Guest Lectures -- if time permits
|