CS 7641 & 4641
Machine Learning
Handouts
Assignments:
Supervised Learning
Randomized Optimization
Unsupervised Learning and
Dimensionality Reduction
Markov Decision Processes
Implementation resources:
For the most part you should be able to find what you need for the
assignments by searching the web, but in case you need it, here are
some starting points.

Kevin Murphy's MDP Toolbox for Matlab

INRA's MDP Toolbox for Matlab
 Some MDP code in Java from the University of Rochester. I haven't looked at it in detail, so your mileage may vary...it seems to have policy and value iteration, though.
 Reinforcement Learning Repository at UMass, Amherst. Mostly has links to other stuff, including a C++ RL toolbox.
 A list of RL software at Rich Sutton's homepage. Again, mostly links to other stuff.
 Reinforcement Learning and Artificial Intelligence (RLAI). Has links to Python code (?).
 Unfortunately, I don't think there is a "standard" RL package out there (like Weka, for instance)...basically, just google for any combination of {MDP,Markov Decision Process[es],TDlambda,reinforcement learning} + {code,software,package} [ + {Java, Matlab, C++, Python, ...} ].
Supplemental Reading:
Support Vector Machines and Kernel Methods (all taken from www.kernelmachines.org's
tutorial page )
 Burges' tutorial on Support Vector Machines (risk, and VC dimension) (ps)
 Burbidge & Buxton's Introduction to SVMs (pdf). Some consider this a
gentler introduction to SVMs.
 Scholkopf's NIPS tutorial slides on SVMs and kernel methods ps)
Boosting
 Schapire's Introduction to Boosting (ps)
 Boosting and margins (pdf)
Information Theory (pdf)
Randomized Search/MIMIC (pdf)
Berkhin's clustering survey (pdf)
Kleinberg's NIPS 2002 impossibility
result for clustering (pdf)
Fodor's survey of dimensionality reduction techniques (pdf)
 Reinforcement Learning
 Kaelbling, Littman and Moore's survey of reinforcement learning (pdf)

Reinforcement Learning: An Introduction, by Richard
S. Sutton and Andrew G. Barto. Check out the link to the HTML version
at the bottom of the intro page.
 Michael
Littman's slides on RL. In particular, the last slide contains
the simplified TDlambda update rule we covered in class.
Andrew Moore's tutorial slides on
zerosum game theory and
nonzerosum game theory.