Machine Learning Seminar and Reading Group

Spring 2009 Focus: Combinatorial Methods in Non-Parametric Estimation

 

General Information
Time and Place: F 15:05-15:55, KACB 1315
Course number: CSE 8001 CNP.
Instructor/Coordinator: Guy Lebanon

The seminar will cover the basic theory of non-parametric estimation with a focus on combinatorial methods such as concentrartion inequalities and VC-theory. We will use the book Combinatorial Methods in Density Estimation by Devroye and Lugosi as the main reference and the book Introduction to Nonparametric Estimation by Tsybakov as a secondary reference. In the second part of the semester I will introduce a few open problems and we will try to make some progress on them. The seminar will be offered as a 1 credit course CSE 8001 CNP.

Prerequisites: basic knowledge of machine learning, probability and statistics. The course will be more mathematically oriented than previous semesters. However, advanced analysis such as measure theory is not required.

 

Spring 2009 (Focus area: Combinatorial Methods in Non-Parametric Density Estimation)

4/24/09 Project presentations  
4/17/09 Project and individual meetings (no class)  
4/10/09 Overview of prospectve projects  
4/3/09 Asymptotic bias of Nadaraya Watson regression, local PCA, local logistic regression  
3/27/09 Local polynomial regression, Nadaraya Watson regression  
3/20/09 Spring Break  
3/13/09 L1 analysis of the kernel density estimator, universal bandwidths, research problem 2: mixture kernels  
3/6/09 Skeleton estimates, research problem 1: concentration inequality and VC analysis for privacy preservation  
2/27/09 Advantage of L1 estimates over L2 and MLE. Scheffe's estimates, minimum distance estimators  
2/20/09 Shatter coefficient, VC dimension, and covering numbers  
2/13/09 Mcdiarmid's inequality and its applications.  
2/6/09 Class cancelled.  
1/30/09 Application of Hoeffding's inequality to classification error rate estimation. Role of train error, validation error, and expected error in the bounds.  
1/23/09 Bandwidth selection, undersmoothing and oversmoothing, Markov, Chebyshev, and Hoeffding's inequalities.  
1/16/09 Asymptotic expansion of bias and variance, consistency conditions, n^(-4/5) convergence rate of the MSE.  
1/9/09 Univariate kernel density estimation, MSE and its bias-variance decomposition.  

 

Fall 2008 (Focus area: probabilistic graphical models)

12/5/1008 Factorial Hidden Markov Models (Jaegul Choo).  
11/21/1008 Max-Margin Markov Networks (Jingu Kim).  
11/14/1008 Conditional Random Fields and an Application to Plot Units (Darren Appling).  
11/7/1008 Latent Dirichlet Allocation (John Bowlan).  
10/31/1008 Pseudo-Likelihood and Composite-Likelihood for Markov Random Fields.  
10/24/1008 Dependency Networks and an Application to Collaborative Filtering.  
10/17/1008 Maximum Likelihood for Markov random fields and its Connection to Maximum Entropy.  
10/10/1008 Special seminar: Statistical Machine Translation.  
10/3/1008 The Max Product Algorithm. Loopy Belief Propagation.  
9/26/1008 The Sum Product Algorithm  
9/19/1008 The Elimination Algorithm  
9/12/1008 The Hammersley Clifford theorem  
9/5/1008 Markov random fields  
8/29/2008 Bayesian networks and their conditional independence relation  
8/22/2008 Introduction to graphical models  
     

 

 

 

 

 

Previous seminar organized by Guy Lebanon at Purdue University