**
MATH 285: INTRODUCTION TO STOCHASTIC PROCESSES (SPRING 2013)
**

** Professor: **
Professor R. J. Williams.

** Office: ** AP&M 6121.

** Professor's Office Hours: ** M, W 3-3.50pm

** Class Time: ** MW, 5pm-6.35pm.

** Class Place: ** AP&M 5402.

There will be no class on Monday, April 29, Monday May 13
or Wednesday May 15. This class time will be made up at other times.

** Teaching Assistant: ** David Lipshutz, AP&M 6343.

** TA Office hour/problem session: ** 11am-noon, Thursdays.

**
DESCRIPTION: **
This one quarter course on stochastic processes is intended
to introduce beginning mathematics graduate students and graduate
students from other scientific and engineering disciplines to some
fundamental stochastic processes used in stochastic modeling. For the
mathematics students, this will provide valuable preparation and
motivation for the more advanced graduate probability sequence, Math
280ABC. For students from other disciplines, the course will provide
a theoretical basis for pursuing applied work involving
stochastic models.

** PREREQUISITES: ** Math 180A or
equivalent probability course or consent of
instructor.
(If you have not taken Math 180A at UCSD, you may need to email Professor Williams
with information about any prior probability class you have taken, in order
to obtain permission to enrol for this course.)

** TENTATIVE COURSE TOPICS: **

Fundamental elements of stochastic processes.
Markov chains.
Hidden Markov models.
Martingales.
Brownian motion.
** HOMEWORK: ** Click here to go to the homework.

** TEXT: ** The text book
is
J. Norris, Markov Chains, Cambridge University Press, 1997.
The book by Karlin and Taylor, listed below, is also a good fundamental
reference, with many examples.
Also, the book by Lawler has an introduction to a variety of topics
for stochastic processes.

** NOTES: ** For class notes, mostly taken by students, click here.
(Username and password are required - given out in class.)

** REFERENCES: **

* General Stochastic Processes and Markov Processes:*

S. Karlin and H. M. Taylor, A First Course in Stochastic Processes,
Academic Press.
G. F. Lawler, Introduction to Stochastic Processes, Chapman and Hall, New York.
* Reversible Markov Chains: *
F. P. Kelly,
Reversibility and Stochastic Networks,
Wiley, 1979. This book is now out of print, but is freely available
online by clicking on the author's name above.

* Hidden Markov Models: *

P. Clote and R. Backofen, Computational Molecular Biology, An Introduction, Wiley, 2000; Chapter 5.
L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition, Prentice Hall, 1993; Chapter 6.

Iain L. MacDonald and Walter Zucchini, Hidden Markov and other Models
for Discrete-valued Time Series, Chapman and Hall/CRC Press, 1997.

* Hidden Markov Models -- more advanced text: *
R. J. Elliott, L. Aggoun, and J. B. Moore, Hidden Markov Models: Estimation
and Control, Springer-Verlag, 1995.

* Brownian Motion: *

K. L. Chung, Green, Brown and Probability, World Scientific, 1995.

* Gaussian Processes: *

R. J. Adler, An Introduction to Continuity, Extrema and Related
Topics for General Gaussian Processes, IMS Lecture Notes--Monograph
Series, Vol. 12, 1990.
S. M. Berman,
Sojourns and Extremes of Stochastic Processes,
Wadsworth & Brooks/Cole Advanced Books & Software, Pacific Grove, CA, 1992.

* Markov Chain Monte Carlo Simulation Methods: *
C. Robert
and G. Casella, Monte Carlo Statistical Methods, Springer, 1999.
C. Robert, Discretization and MCMC Convergence, Lecture Notes 135,
Springer, 1998.

* Markov Decision Processes *
M. L. Puterman, Markov Decision Processes, Wiley, 1994.

E. Altman, Constrained Markov Decision Processes, Chapman and Hall, CRC Press, 1999.

* Fitting Stochastic Models to Data: *

E. A. Thompson, Statistical Inference from Genetic Data on Pedigrees,
NSF-CBMS Regional Conference Series in Probability and Statistics, Vol. 6,
Institute of Mathematical Statistics, 2000.

B. J. T. Morgan, Applied Stochastic Modelling, Arnold Publishing, London, 2000.

* Selected Applications of Stochastic Modeling: *

P. Baldi and S. Brunak, Bioinformatics: The Machine
Learning Approach, MIT Press, Second Edition, 2001.
R. Durbin, S. Eddy, A. Krogh, and G. Mitchison, Biological Sequence
Analysis, Cambridge University Press, 1998.
M. S. Waterman, Introduction to Computational Biology, Chapman and Hall/CRC Press, 1995.
G. Winkler, Image Analysis, Random Fields, and Dynamic Monte Carlo Methods, Springer, 1995.
** SOFTWARE **

Laird Breyer's page on Metropolis-Hastings algorithms and more.
Hidden Markov Model Matlab Toolbox (GNU software)
** USEFUL INFORMATIONAL LINKS **

Some information about the Perron-Frobenius theorem which guarantees
uniqueness of the stationary distribution for an irreducible, finite
state Markov chain.
Web sources on hidden Markov models (under
construction)
Web sources on bioinformatics (under construction)
Brownian motion: a description of aspects of this process
and
some of its applications
(provided
by Y.K.Lee and Kelvin Hoon)

** Contact information: ** If you have questions about this course,
please send email to williams at math dot ucsd dot edu stating your background in
probability, your department
and any questions you might have.

Last updated April 1, 2013.