Intelligent Systems

Machine Learning Reading List

General:

Christopher M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press (1995).
T. Mitchell (1997). Machine Learning, chapters 2-3, 6-7, 13.

Foundations and theory:

J. Langford, Tutorial on Practical Prediction Theory for Classification Journal of Machine Learning Research 6 (Mar): 273--306, 2005. (jmlr.org)
J. Kleinberg (2002). An Impossibility Theorem for Clustering. Advances in Neural Information Processing Systems (NIPS).

Graphical models:

E. Charniak (1991). "Bayesian Networks without Tears", AI magazine.
P. Smyth (1998). Belief networks, hidden Markov models, and Markov random fields: a unifying view. Pattern Recognition Letters.
F. Jensen (2001). Bayesian Networks and Decision Graphs, Springer, chapters 1-2, 5.
J Yedidia, W. Freeman, Y Weiss (2003). "Understanding Belief Propagation and Its Generalizations", Exploring Artificial Intelligence in the New Millennium, ISBN 1558608117, Chap. 8, pp. 239-236, January 2003 (Science & Technology Books). An online tech report is available at: http://www.merl.com/papers/TR2001-22/

Kernel methods:

C. Burges. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery. pp 121-167, 1998

Reinforcement learning:

L.P. Kaelbling, M.L. Littman, & A.W. Moore (1996), Reinforcement Learning: A Survey, Journal of Artificial Intelligence Research, 4:237-285.

Ensemble approaches:

Robert E. Schapire (1999). A brief introduction to boosting. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence.

Symbolic approaches:

T. Mitchell, R. Keller & S. Kedar-Cabelli (1986), Explanation-Based Generalization: A Unifying View. Machine Learning 1; reprinted in Shavlik & Dietterich (eds.), Readings in Machine Learning, section 4.2.1.
Case-Based Reasoning: Experiences, Lesons, and Future Directions, David B. Leake, editor (AAAI Press / MIT Press, 1996). Chapter on "CBR in Context: The Present and Future" only.
Continuous Case-Based Reasoning, Ashwin Ram, Juan Carlos Santamaria. Artificial Intelligence, (90)1-2:25--77, 1997
J.W. Murdock and A.K. Goel (2001). Learning about Constraints by Reflection. In 14th Biennial Conference of Canadian AI Society, pp. 131-140; available as Lecture Notes in AI - 2006, Springer.

Specific methods:

L. R. Rabiner (1989). A tutorial on hidden Markov models and its application to speech recognition. in Proc. IEEE, vol. 77, pp. 257-286.
Bell A.J. and Sejnowski T.J. (1995). An information maximisation approach to blind separation and blind deconvolution, Neural Computation, 7, 6, 1129-1159
Roweis and Saul, Nonlinear Dimensionality Reduction by Locally Linear Embedding, Science 2000 290: 2323-2326

Computation:

A. Gray and A. Moore (1999). N-body Problems in Statistical Learning. Advances in Neural Information Processing Systems (NIPS).
Doucet, de Freitas, and Gordon (2001). An Introduction to Sequential Monte Carlo Methods. in Sequential Monte Carlo Methods in Practice, pages 3-14, New York: Springer-Verlag, January 2001.

Cool recent applications:

C. Isbell and P. Viola (1998). Restructuring Sparse High Dimensional Data for Effective Retrieval. Advances in Neural Information Processing Systems (NIPS).
F. Dellaert, S. Seitz, C. Thorpe, and S. Thrun (2000). Feature Correspondence: A Markov Chain Monte Carlo Approach. Advances in Neural Information Processing Systems (NIPS).
V. Pavlovic, J. M. Rehg, and J. MacCormick (2000). Learning Switching Linear Models of Human Motion. Advances in Neural Information Processing Systems (NIPS).