CSE 8803ML: Machine Learning II: Advanced Topics
Spring 2012
Lecture Time
Tuesday and Thursday 1:35 -- 2:55pm in Klaus 1447 (starting Jan 10th)
Course Description
This class will cover several advanced machine learning topics, including graphical models, kernel methods, boosting, bagging, semi-supervised and active learning, and tensor approach to data analysis. The focus of the class will be graphical models and kernel methods, which are currently the major paradigms for building advanced and sophisticated machine learning models for complex real world problems.
Graphical models provide a unified view for a wide range of problems with a very large number of attributes and huge datasets, where we want to obtain a coherent global conclusion from local information. Concepts and algorithms from graphical models enable efficient inference, decision-making and learning in a variety of problems including artificial intelligence, statistics, computer systems, computer vision, natural language processing, and computational biology.
Kernel methods provide a general framework for extending algorithms designed for finding linear relations and patterns to nonlinear cases. Kernel methods approach the problem by mapping the data into a high dimensional feature space; and in that space, a variety of methods can be used to find relations and patterns in the data. Since the mapping can be quite general (eg., not necessarily linear), the relations found in this way are accordingly very general, and the type of data kernel methods can be applied to is also very general (eg., sequence data, audios, images and graph data).
This graduate-level class will provide you with a strong foundation for both applying machine learning to complex real world problems and for addressing core research topics in machine learning. Students entering the class should have a pre-existing working knowledge of probability, statistics, linear algebra and algorithms, though the class has been designed to allow students with a strong numerate background to catch up and fully participate.
Textbooks
- Pattern Recognition and Machine Learning, Chris Bishop
- Kernel Methods for Pattern Analysis, John Shawe-Taylor & Nello Cristianini
- Gaussian Processes for Machine Learning, Carl Edward Rasmussen & Christopher Williams
- Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond Bernhard Schlkopf & Alex Smola
- Probabilistic Graphical Models: Principles and Techniques Daphne Koller & Nir Friedman
People
Instructor: Le Song, Klaus 1340, Office Hours: Thursday 3-4pm
Guest Lecturer: Mariya Ishteva, Klaus 1336
TA: Krishnakumar Balasubramanian, Klaus 1305, Office Hours: Tuesday 3:30-5pm
Class Assistant: Michael Terrell, Klaus 1321, Office Hours: TBA
Mailing list
Web address: groups.google.com/group/cse8803ml2
Email address: cse8803ml2@googlegroups.com
Syllabus and Schedule
| Date | Lecture & Topics | Readings & Useful Links |
Handouts |
|---|---|---|---|
| Tue 1/10 |
Lecture 0: Introduction
|
Slides |
|
| Graphical Models | |||
| Thu 1/12 |
Lecture 1: representation of Bayesian Networks (directed GM)
|
Slides |
|
| Tue 1/17 | Lecture 2: representation of undirected GM | Slides |
|
| Thu 1/19 | Lecture 3: unified view of directed and undirected GM | Slides |
|
| Tue 1/24 | Lecture 4: inference --- variable elimination and message passing algorithm | Slides |
|
| Thu 1/26 | Lecture 5: inference --- junction tree algorithm | Slides |
|
| Tue 1/31 | Lecture 6: inference --- variational inference | Slides |
|
| Thu 2/2 | Lecture 7: inference --- sampling | Slides |
|
| Tue 2/7 | Lecture 8: parameter learning | Slides |
|
| Thu 2/9, Tue 2/14 | Lecture 9, 10: parameter learning from partially observed data --- EM | Slides Slides |
|
| Thu 2/16, Tue 2/21 | Lecture 11, 12: structure learning | Slides Slides |
|
| Thu 2/23 | Lecture 13: Latent Dirichlet Allocation | ||
| Thu 2/28 | Lecture 14: Kalman Filter, Hidden Markov Models, Conditional Random Fields | Slides |
|
| Tue 3/1 | Lecture 15: Collaborative Filtering | Slides | |
| Kernel Methods | |||
| Thu 3/8 | Lecture 16: kernels, kernel classifier and regression | Slides | |
| Tue 3/13 | Lecture 17: kernel PCA, clustering, canonical correlation analysis | Slides | |
| Thu 3/15 | Lecture 18: two sample test and measure of dependence beyound vector data (eg. sequences and graphs) | Slides | |
| Tue 3/27 | Lecture 19: Gaussian process I | Slides | |
| Thu 3/29 | Lecture 20: Gaussian process II | Slides | |
| Tue 4/3 | Lecture 21: fast kernel methods and random features | Slides | |
| Other Advanced Models and Topics | |||
| Thu 4/5 | Lecture 22: bagging and boosting | Slides | |
| Tue 4/10 | Lecture 23: semi-supervised learning | Slides | |
| Thu 4/12 | Lecture 24: active learning | Slides | |
| Tue 4/17 | Tensor data analysis I | Slides | |
| Thu 4/19 | Tensor data analysis II | Slides Slides | |
| Project Presentations | |||
| Tue 4/24 | Project Presentation | ||
| Thu 4/26 | Project Presentation | ||