Course Notes and Lectures
- Lecture template I Sample spaces, multinomial, repeated sampling
- Lecture template II ML estimation
- Lecture template III Estimating small probabilities
- Lecture template IV Parametric density estimation
- LIV supplement Correcting the ML estimator for the uniform distribution
- Lecture template V Non-parametric density estimation
- Lecture template VI Model selection with AIC, BIC
- Lecture template VII Linear regression
Lecture notes Linear and logistic regression
Lecture template VIII.1 Double descent
Lecture template VIII.2 Double descent example for linear regressionCourse notes
- l1-jan7 Sample spaces , outcomes
- l2: Sample spaces, probability and math refresher, repeated iid sampling
- l3-jan14 Large discrete sample spaces, repeated iid sampling (continued), binomial and multinomial
- l4-jan16 Maximum Likelihood
- l5-jan21 Maximum Likelihood - examples
- l6-jan23 Maximum Likelihood estimate as random variable
- l7-jan28 Estimating small probabilities (smoothing)
- l8-jan30 Estimating small probabilities. Density estimation.
- l9-feb4 Parametric density estimation.
- l10-feb6 Parametric density estimation by gradient ascent.
- l11-feb11.pdf Kernel density estimation
- l12-feb13.pdf Kernel density estimation-effect of h
- l13-feb18.pdf CV, K-fold CV, model selection
- l14-feb20.pdf Model selection, Linear regression
- l15-feb25.pdf Linear regression
- l16-feb27.pdf Double descent, Classification, logistic regression
- l17-mar4.pdf Logistic regression as a linear classifier
- l18-mar6.pdf Classification, Statistical decisions
- l19-mar11.pdf Statistical decisions, Probabilistic Reasoning
- l20-mar13.pdf Probabilistic Reasoning
Links to figures
Estimating the probabilities of Chinese characters from a small corpus
fig-smooth-chinf-hist.png Histogram of word frequencies
fig-smooth-chinf-histhist-no0.png Histogram of character distribution (=observed counts, sorted)
fig-smooth-chinf-histhist.png Histogram of counts distribution (=fingerprint)
fig-smooth-chinf-theta-large.png Smoothed char probabilities, frequent characters
fig-smooth-chinf-theta-small.png Smoothed char probabilities, rare characters
fig-smooth-chinf-thetas.png Smoothed char probabilities, all characters
fig-smooth-chinf-eval.png How close are the smoothed probability estimates from the truth?
Counts and fingerprint for uniform (top) and 1/j decaying distributions with |S|=m=50 for n=100 samples (bottom)
Unedited course notes
Lecture templates
