Codes & Data

CSE 8803ML: Machine Learning II: Advanced Topics

Spring 2012

Lecture Time

Tuesday and Thursday 1:35 -- 2:55pm in Klaus 1447 (starting Jan 10th)

Course Description

This class will cover several advanced machine learning topics, including graphical models, kernel methods, boosting, bagging, semi-supervised and active learning, and tensor approach to data analysis. The focus of the class will be graphical models and kernel methods, which are currently the major paradigms for building advanced and sophisticated machine learning models for complex real world problems.

Graphical models provide a unified view for a wide range of problems with a very large number of attributes and huge datasets, where we want to obtain a coherent global conclusion from local information. Concepts and algorithms from graphical models enable efficient inference, decision-making and learning in a variety of problems including artificial intelligence, statistics, computer systems, computer vision, natural language processing, and computational biology.

Kernel methods provide a general framework for extending algorithms designed for finding linear relations and patterns to nonlinear cases. Kernel methods approach the problem by mapping the data into a high dimensional feature space; and in that space, a variety of methods can be used to find relations and patterns in the data. Since the mapping can be quite general (eg., not necessarily linear), the relations found in this way are accordingly very general, and the type of data kernel methods can be applied to is also very general (eg., sequence data, audios, images and graph data).

This graduate-level class will provide you with a strong foundation for both applying machine learning to complex real world problems and for addressing core research topics in machine learning. Students entering the class should have a pre-existing working knowledge of probability, statistics, linear algebra and algorithms, though the class has been designed to allow students with a strong numerate background to catch up and fully participate.


Other useful books:


Instructor: Le Song, Klaus 1340, Office Hours: Thursday 3-4pm

Guest Lecturer: Mariya Ishteva, Klaus 1336

TA: Krishnakumar Balasubramanian, Klaus 1305, Office Hours: Tuesday 3:30-5pm

Class Assistant: Michael Terrell, Klaus 1321, Office Hours: TBA

Mailing list

Web address: groups.google.com/group/cse8803ml2

Email address: cse8803ml2@googlegroups.com

Syllabus and Schedule

Date Lecture & Topics Readings & Useful Links
Tue 1/10 Lecture 0: Introduction
  • What is Machine Learning?
  • Applications of Machine Learning
  • Basic Machine Learning Models
  • Building Advanced Models with Graphical Models
  • Building Advanced Models with Kernel Methods
  • Other Advanced Models
  • Logistics
  • Review of probability and conditional independence

  • JavaBayes
  • Science special issue on data science
  • Nature special issue on big data

  • Slides
    Graphical Models
    Thu 1/12 Lecture 1: representation of Bayesian Networks (directed GM)
    • Local Markov Assumption encoded by BN
    • Factorization of distribution according to BN
    • I-map
    • D-separation
    • Limitations of BN

  • Bayesian Networks Without Tears, E. Charniak, 1991.

  • Slides
    Tue 1/17 Lecture 2: representation of undirected GM

    Thu 1/19 Lecture 3: unified view of directed and undirected GM

    Tue 1/24 Lecture 4: inference --- variable elimination and message passing algorithm

    Thu 1/26 Lecture 5: inference --- junction tree algorithm

    Tue 1/31 Lecture 6: inference --- variational inference

    Thu 2/2 Lecture 7: inference --- sampling

    Tue 2/7 Lecture 8: parameter learning

    Thu 2/9, Tue 2/14 Lecture 9, 10: parameter learning from partially observed data --- EM

    Slides Slides
    Thu 2/16, Tue 2/21 Lecture 11, 12: structure learning

    Slides Slides
    Thu 2/23 Lecture 13: Latent Dirichlet Allocation
  • Latent Dirichlet Allocation
  • Finding Scientific Topics

  • Thu 2/28 Lecture 14: Kalman Filter, Hidden Markov Models, Conditional Random Fields
  • Kalman Filter: Introduction
  • Hidden Markov Models: Tutorial
  • Conditional Random Fields
  • Conditional Random Fields: Introduction

  • Slides
    Tue 3/1 Lecture 15: Collaborative Filtering
  • Probablistic Matrix Factorization
  • Probabilistic Matrix Factorization using MCMC

  • Slides
    Kernel Methods
    Thu 3/8 Lecture 16: kernels, kernel classifier and regression

    Tue 3/13 Lecture 17: kernel PCA, clustering, canonical correlation analysis

    Thu 3/15 Lecture 18: two sample test and measure of dependence beyound vector data (eg. sequences and graphs)

    Tue 3/27 Lecture 19: Gaussian process I

    Thu 3/29 Lecture 20: Gaussian process II

    Tue 4/3 Lecture 21: fast kernel methods and random features

    Other Advanced Models and Topics
    Thu 4/5 Lecture 22: bagging and boosting

    Tue 4/10 Lecture 23: semi-supervised learning

    Thu 4/12 Lecture 24: active learning

    Tue 4/17 Tensor data analysis I

    Thu 4/19 Tensor data analysis II

    Slides Slides
    Project Presentations
    Tue 4/24 Project Presentation

    Thu 4/26 Project Presentation