CS 4803/8803 PAR
Pattern Recognition

Spring 2006
CCB 102
MWF 4:00 - 5:00

Problem Sets        Syllabus        Projects

Note: The take-home midterm will be handed out at the end of class on Feb. 24. It will be due in class on Mon Feb 27.

This course will cover basic and advanced techniques in Pattern Recognition. The goal of this class is to gain theoretical and practical understanding of classification techniques and their application to real-world problems.

Instructor

Jim Rehg
Email: rehg@cc.gatech.edu
Office: CoC Bldg (CCB) 253
Office hours: After class or by appointment
Phone: 404-894-9105 (email preferred)

Teaching Assistant

TBD
Email:
Office:
Office hours:

Prerequisites

Familiarity with basic probability and linear algebra. This is an introductory course and the lectures will be self-contained..

Text

The class textbook is Pattern Classification by Duda, Hart, and Stork.

Other texts:

• Neural Networks for Pattern Recognition by C. Bishop. Oxford University Press, 1995. [Amazon]
• Linear Algebra and Its Applications or Introduction to Linear Algebra by G. Strang. [Amazon]
• Matrix Reference Manual [online]
• Introduction to Probability by D. P. Bertsekas and J. N. Tsitsiklis. [online]
• Probability, Random Variables, and Stochastic Processes by A. Papoulis.
  Classic text for probability theory and its application. Also see on-line lecture notes for EE178 at Stanford.
• Probability Theory: The Logic of Science by E. T. Jaynes. [online]
  Classic text on probability theory, chapters 1 and 2 in particular are good background reading.

Help with Matlab

• Alex's Matlab tutorial (taken from CS7495 website).
• Ben's Matlab tutorial

Organization

Grades will be assessed as follows:

Collaboration on problem sets is encouraged at the "white board interaction" level. That is, share ideas and technical conversation, but write your own code, do your own detailed derivations, etc. A few problem sets may require you to work in teams of 2-3. I plan to grade and return problem sets promptly. As a result, I will require all problem sets to be turned in on time.

No late submissions will be accepted without prior permission of the instructor. If you need an extension, let me know in advance.

I will grade undergraduate and graduate students on separate criteria in this course.

Problem Sets

PS 1 [pdf] Due in class on Wed Feb 22.

Topics

• Introduction to Statistical Pattern Recognition
  • Review of multivariate probability and statistics
  • Example applications
• Bayesian Decision Theory
  • Philosophy and Methodology
  • Classifiers, Discriminant Functions, and Decision Surfaces
  • Neyman-Pearson Criteria and Bayes Risk
  • Discriminant Functions for Multivariate Normal Densities
  • Bayes Error and Loss Bounds
  • Application: Color-Based Skin Detection in Images
• Parameter Estimation
  • Maximum Likelihood Principle and Bayesian Estimation
  • Gaussian Learning in the Univariate and Multivariate Cases
  • Sufficient Statistics and the Exponential Family
  • Bias-Variance Dilemma
  • Computational Complexity
  • Application: Gene Expression Levels
• Nonparametric Methods
  • Kernel Density Estimators, Convergence Rates, and Error Bounds
  • Nearest Neighbor Rule, Convergence Rates, and Error Bounds
  • Computational Complexity and Dimensionality
  • Application: Robot Juggling Using Locally-Weighted Regression
• Feature Selection and Generation
  • Principal Components Analysis
  • Fisher’s Linear Discriminant
  • CART Trees
  • Multilayer Neural Networks
  • Application: Handwritten Character Recognition
• Model Selection
  • No Free Lunch Theorem
  • Occam’s Razor and Minimum Description Length Principle
  • Ensemble Methods: Bagging and Boosting
  • Bayesian Model Selection
  • Application: Audio Classification
    
• Learning Theory
  • VC Dimension
  • Structured Risk Minimization
  • Support Vector Machines
  • PAC Learning

Final Projects

I am very happy for you to do a final project that is related to some research project that you are involved in. I am also happy to work with to identify an appropriate final project topic.