PhD CS – Machine Learning Body of Knowledge


Prepared by: Maria-Florina Balcan, Alexander Gray, Charles Isbell, and Guy Lebanon

The exam will be divided to four areas: core, statistical methods and models, learning theory,
and decision processes. There will be three questions in each area. Each student has to answer
two out of the three questions in the core area. In addition, each student has to select two out
of the remaining three areas where he or she will answer two out of the three questions.

Reading List:

Core

  • C. Bishop. Pattern Recognition and Machine Learning, Second Edition, Springer 2007. Chapters 15.

Statistical Models and Methods

  • C. Bishop. Pattern Recognition and Machine Learning, Second Edition, Springer 2007.
  • T. Hastie et. al. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition, Springer 2009 (first edition is also OK for this year).

Learning Theory

  • M. Kearns and U. Vazirani. An Introduction to Computational Learning Theory. The MIT Press 1994. Chapters 1, 2, 3, 4, 5, 7.
  • L. Devroye et. al. A Probabilistic Theory of Pattern Recognition, Springer 1996. Chapters 2, 8, 12.
  • A. Blum. OnLine Algorithms in Machine Learning. In Fiat and Woeginger (eds.), Online Algorithms: the state of the art, LNCS 1442, Springer 1998. Available at http://www.cs.cmu.edu/~avrim/Papers/survey.ps

Decision Processes

  • R. Sutton and A. Barto. Reinforcement Learning: An Introduction. The MIT Press 1998.
  • L. P. Kaelbling et. al. Reinforcement Learning: A Survey. Journal of Artificial Intelligence 4:237285, 1996.
  • Y. Shoham et. al. If multi-agent learning is the answer, what is the question? Artificial Intelligence 171(7):365377 2007.