PhD CS – Intelligent Systems Body of Knowledge

Machine Learning Reading List

General:

• Christopher M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press (1995).
• T. Mitchell (1997). Machine Learning, chapters 2-3, 6-7, 13.

Foundations and theory:

• J. Langford, Tutorial on Practical Prediction Theory for Classification Journal of Machine Learning Research 6 (Mar): 273--306, 2005. (jmlr.org)
• J. Kleinberg (2002). An Impossibility Theorem for Clustering. Advances in Neural Information Processing Systems (NIPS).

Graphical models:

• E. Charniak (1991). "Bayesian Networks without Tears", AI magazine.
• P. Smyth (1998). Belief networks, hidden Markov models, and Markov random fields: a unifying view. Pattern Recognition Letters.
• F. Jensen (2001). Bayesian Networks and Decision Graphs, Springer, chapters 1-2, 5.
• J Yedidia, W. Freeman, Y Weiss (2003). "Understanding Belief Propagation and Its Generalizations", Exploring Artificial Intelligence in the New Millennium, ISBN 1558608117, Chap. 8, pp. 239-236, January 2003 (Science & Technology Books). An online tech report is available at: http://www.merl.com/papers/TR2001-22/

Kernel methods:

• C. Burges. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery. pp 121-167, 1998

Reinforcement learning:

• L.P. Kaelbling, M.L. Littman, & A.W. Moore (1996), Reinforcement Learning: A Survey, Journal of Artificial Intelligence Research, 4:237-285.

Ensemble approaches:

• Robert E. Schapire (1999). A brief introduction to boosting. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence.

Symbolic approaches:

• T. Mitchell, R. Keller & S. Kedar-Cabelli (1986), Explanation-Based Generalization: A Unifying View. Machine Learning 1; reprinted in Shavlik & Dietterich (eds.), Readings in Machine Learning, section 4.2.1.
• Case-Based Reasoning: Experiences, Lesons, and Future Directions, David B. Leake, editor (AAAI Press / MIT Press, 1996). Chapter on "CBR in Context: The Present and Future" only.
• Continuous Case-Based Reasoning, Ashwin Ram, Juan Carlos Santamaria. Artificial Intelligence, (90)1-2:25--77, 1997
• J.W. Murdock and A.K. Goel (2001). Learning about Constraints by Reflection. In 14th Biennial Conference of Canadian AI Society, pp. 131-140; available as Lecture Notes in AI - 2006, Springer.

Specific methods:

• L. R. Rabiner (1989). A tutorial on hidden Markov models and its application to speech recognition. in Proc. IEEE, vol. 77, pp. 257-286.
• Bell A.J. and Sejnowski T.J. (1995). An information maximisation approach to blind separation and blind deconvolution, Neural Computation, 7, 6, 1129-1159
• Roweis and Saul, Nonlinear Dimensionality Reduction by Locally Linear Embedding, Science 2000 290: 2323-2326

Computation:

• A. Gray and A. Moore (1999). N-body Problems in Statistical Learning. Advances in Neural Information Processing Systems (NIPS).
• Doucet, de Freitas, and Gordon (2001). An Introduction to Sequential Monte Carlo Methods. in Sequential Monte Carlo Methods in Practice, pages 3-14, New York: Springer-Verlag, January 2001.

Cool recent applications:

• C. Isbell and P. Viola (1998). Restructuring Sparse High Dimensional Data for Effective Retrieval. Advances in Neural Information Processing Systems (NIPS).
• F. Dellaert, S. Seitz, C. Thorpe, and S. Thrun (2000). Feature Correspondence: A Markov Chain Monte Carlo Approach. Advances in Neural Information Processing Systems (NIPS).
• V. Pavlovic, J. M. Rehg, and J. MacCormick (2000). Learning Switching Linear Models of Human Motion. Advances in Neural Information Processing Systems (NIPS).