Contents
Preface x
CHAPTER 1 Introduction 1
1.1. Randomness 1
1.2. Stochastic processes 5
1.3. Purposes of stochastic models 9
1.4. Overview 12
1.5. Bibliographic remarks 13
1.6. Exercises 14
CHAPTER 2 Discrete time Markov chains 16
2.1. Precipitation at Snoqualmie Falls 16
2.2. The marginal distribution 21
2.3. Classification of states 23
2.4. Stationary distribution 35
2.5. Long term behavior 43
2.6. Markov chain Monte Carlo methods 52
2.7. Likelihood theory for Markov chains 58
2.8. Higher order chains 70
2.9. Chain-dependent models 74
2.10. Random walks and harmonic analysis 82
2.11. Bienaym-Galton-Watson branching processes 90
2.12. Hidden Markov models 103
2.13. Bibliographic remarks 112
2.14. Exercises 114
CHAPTER 3 Continuous time Markov chains 125
3.1. The soft component of cosmic radiation 125
3.2. The pure birth process 128
3.3. The Kolmogorov equations 133
3.4. A general construction 140
3.5. Queueing systems 147
3.6. An improved model for cosmic radiation 151
3.7. Statistical inference for continuous time Markov chains 153
3.8. Modeling neural activity 164
3.9. Blood formation in cats 172
3.10. Bibliographic remarks 181
3.11. Exercises 181
CHAPTER 4. Markov random fields 189
4.1. The Ising model of ferromagnetism 189
4.2. Markov random fields 191
4.3. Phase transitions in Markov random fields 196
4.4. Likelihood analysis of the Ising model 200
4.5. Reconstruction of astronomical images 203
4.6. Image analysis and pedigrees 209
4.7. Bibliographic remarks 219
4.8. Exercises 219
CHAPTER 5. Point processes 227
5.1. A model of traffic patterns 227
5.2. General concepts 230
5.3. Estimating second-order parameters for stationary point processes 238
5.4. Relationships between processes 241
5.5. Modeling the complete intensity 245
5.6. Marked point processes 250
5.7. Spatial point processes 260
5.8. Bibliographic remarks 268
5.9. Exercises 270
CHAPTER 6. Brownian motion and diffusion 276
6.1. Brownian motion 276
6.2. Second-order processes 280
6.3. The Brownian motion process 283
6.4. A more realistic model of Brownian motion 289
6.5. Diffusion equations 294
6.6. Likelihood inference for stochastic differential equations 301
6.7. The Wright-Fisher model of diploid populations 305
6.8. Bibliographic remarks 311
6.9. Exercises 311
APPENDIX A. Some statistical theory 318
A.1. Multinomial likelihood 318
A.2. The parametric case 319
A.3. Likelihood ratio tests 320
A.4. Sufficiency 322
APPENDIX B. Linear difference equations with constant coefficients 325
B.1. The forward shift operator 325
B.2. Homogeneous difference equations 325
B.3. Non-homogeneous difference equations 327
APPENDIX C. Some theory of partial differential equations 329
C.1. The method of auxiliary equations 329
C.2. Some applications 330
References 332
Index of results 349
Applications and examples 351
Index of notation 354
Index of terms 359
Data sets 371