CS 4476 / 6476 Computer Vision

Fall 2017, MWF 11:15 to 12:05, College of Computing room 16
Instructor: James Hays

TAs: Varun Agrawal, Samarth Brahmbhatt, Cusuh Ham, Eunji Chong, Wenqi Xian, Wengling Chen, Albert Shaw, Stefan Stojanov, Jonathan Balloch

Computer Vision, art by kirkh.deviantart.com

Course Description

This course provides an introduction to computer vision including fundamentals of image formation, camera imaging geometry, feature detection and matching, stereo, motion estimation and tracking, image classification and scene understanding. We'll develop basic methods for applications that include finding known models in images, depth recovery from stereo, camera calibration, image stabilization, automated alignment, tracking, boundary detection, and recognition. The focus of the course is to develop the intuitions and mathematics of the methods in lecture, and then to learn about the difference between theory and practice in the projects.

The difference between the undergraduate version of the class (CS4476) and the graduate version (CS6476) will be the requirements on the projects. In particular, more challenging extensions of the projects will be extra credit for CS4476 but required for CS6476.

The Advanced Computer Vision course (CS7476) in spring will build on this course and deal with advanced and research related topics in Computer Vision, including Machine Learning, Graphics, and Robotics topics that impact Computer Vision.

Learning Objectives

Upon completion of this course, students should be able to:

1. Recognize and describe both the theoretical and practical aspects of computing with images. Connect issues from Computer Vision to Human Vision
2. Describe the foundation of image formation and image analysis. Understand the basics of 2D and 3D Computer Vision.
3. Become familiar with the major technical approaches involved in computer vision. Describe various methods used for registration, alignment, and matching in images.
4. Get an exposure to advanced concepts leading to object and scene categorization from images.
5. Build computer vision applications.

Prerequisites

No prior experience with computer vision is assumed, although previous knowledge of visual computing or signal processing will be helpful. The following skills are necessary for this class:

Data structures: You'll be writing code that builds representations of images, features, and geometric constructions.
Programming: Projects are to be completed and graded in Matlab. All lecture code and project starter code will be in MATLAB. TA's will support questions about MATLAB. If you've never used MATLAB that is OK, as long as you have programming experience.
Math: Linear algebra, vector calculus, and probability. Linear algebra is the most important and students who have not taken a linear algebra course have struggled in the past.

Grading

Your final grade will be made up from

80% 6 programming projects
20% 2 written quizzes

You will lose 10% each day for late projects. However, you have three "late days" for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible.

These late days are intended to cover unexpected clustering of due dates, travel commitments, interviews, hackathons, etc. Don't ask for extensions to due dates because we are already giving you a pool of late days to manage yourself.

Graduate Credit

If you are enrolled in the graduate section CS 6476 then you will be expected to do additional work on each project. Each project will list several extra credit opportunities available and CS 6476 students will be required to do at least 10 points worth of extra credit (for which you will not get extra credit, unless you do more than 10 points worth).

Academic Integrity

Academic dishonesty will not be tolerated. This includes cheating, lying about course matters, plagiarism, or helping others commit a violation of the Honor Code. Plagiarism includes reproducing the words of others without both the use of quotation marks and citation. Students are reminded of the obligations and expectations associated with the Georgia Tech Academic Honor Code and Student Code of Conduct, available online at www.honor.gatech.edu. For quizzes, no supporting materials are allowed (notes, calculators, phones, etc).

You are expected to implement the core components of each project on your own, but the extra credit opportunties often build on third party data sets or code. That's fine. Feel free to include results built on other software, as long as you are clear in your handin that it is not your own work.

Learning Accommodations

If needed, we will make classroom accommodations for students with documented disabilities. These accommodations must be arranged in advance and in accordance with the ADAPTS office (www.adapts.gatech.edu).

Important Links:

Piazza for CS 4476 / 6476. This should be your first stop for questions and announcements.
t-square.gatech.edu will be used to hand in assignments.
Matlab Tutorial
Get Matlab from software.oit.gatech.edu

Contact Info and Office Hours:

If possible, please use Piazza to ask questions and seek clarifications before emailing the instructor or staff.

James: hays[at]gatech.edu
Varun Agrawal: varunagrawal[at]gatech.edu
Cusuh Ham: cusuh[at]gatech.edu
Samarth Brahmbhatt: samarth.robo[at]gatech.edu
Eunji Chong: eunjichong[at]gatech.edu
Albert Shaw: ashaw596[at]gatech.edu
Wengling Chen: wchen342[at]gatech.edu
Wenqi Xian: wxian3cmpe[at]gatech.edu
Stefan Stojanov: sstojanov[at]gatech.edu
Jonathan Balloch: balloch[at]gatech.edu

Office Hours

James, Monday and Friday, 12 to 1 (CCB 315).
TA hours: Monday 12-2, Tuesday 11-1, Wednesday 12-2, Thursday 11-1. (CCB 340)

Assignments	Highlighted projects	All Results
(Optional) Eclipse viewing
Image Filtering and Hybrid images		Project 1 results
Local Feature Matching		Project 2 results
Camera Calibration and Fundamental Matrix Estimation with RANSAC		Project 3 results
Scene Recognition with Bag of Words		Project 4 results
Face Detection with a Sliding Window
Deep Learning

It is strongly recommended that all projects be completed in Matlab. All starter code will be provided for Matlab. Students may implement projects through other means but it will generally be more difficult.

Textbook

Readings will be assigned in "Computer Vision: Algorithms and Applications" by Richard Szeliski. The book is available for free online or available for purchase.

Syllabus

Class Date	Topic	Slides	Reading	Projects
Mon, Aug 21	No Lecture Optional assignment 0, eclipse pinhole photography
Wed, Aug 23	Introduction to computer vision	pptx, pdf	Szeliski 1	Project 1 out
Image Formation and Filtering (Szeliski chapters 2 and 3)
Fri, Aug 25	Cameras and Optics	pptx, pdf	Szeliski 2.1, especially 2.1.5
Mon, Aug 28	Light and Color	pptx, pdf	Szeliski 2.2 and 2.3
Wed, Aug 30	Image Filtering	pptx, pdf	Szeliski 3.2
Fri, Sept 1	Thinking in frequency	pptx, pdf	Szeliski 3.4
Mon, Sept 4	No classes, Institute holiday
Wed, Sept 6	Thinking in frequency part 2	pptx, pdf	Szeliski 3.5.2 and 8.1.1
Feature Detection and Matching
Fri, Sept 8	Edge detection	pptx, pdf	Szeliski 4.2
Mon, Sept 11	No classes, Tropical Storm Irma			Project 2 out
Wed, Sept 13	Interest points and corners	pptx, pdf	Szeliski 4.1.1
Fri, Sept 15	Local image features	pptx, pdf	Szeliski 4.1.2
Mon, Sept 18	Feature matching and hough transform	pptx, pdf	Szeliski 4.1.3 and 4.3.2
Wed, Sept 20	Model fitting and RANSAC	pptx, pdf	Szeliski 6.1 and 2.1
Multiple Views and Motion
Fri, Sept 22	Stereo intro	pptx, pdf	Szeliski 11
Mon, Sept 25	Camera Calibration	pptx, pdf	Szeliski 6.2.1
Wed, Sept 27	Epipolar Geometry and Structure from Motion	pptx, pdf	Szeliski 7	Project 3 out
Fri, Sept 29	Stereo Correspondence	pptx, pdf	Szeliski 11
Mon, Oct 2	Optical Flow	pptx, pdf	Szeliski 8.4
Machine Learning Crash Course
Wed, Oct 4	Machine learning	pptx, pdf	Szeliski 5.3
Fri, Oct 6	Quiz 1
Recognition
Mon, Oct 9	No classes, Institute holiday
Wed, Oct 11	Recognition overview and bag of features	pptx, pdf	Szeliski 14
Fri, Oct 13	Large-scale instance recognition	pptx, pdf	Szeliski 14.3.2
Mon, Oct 16	Large-scale retrieval: Vocabulary Tree	pptx, pdf		Project 4 out
Wed, Oct 18	Large-scale retrieval: Spatial Verification, TF-IDF, Query Expansion	pptx, pdf
Fri, Oct 20	Large-scale category recognition and advanced feature encoding	pptx, pdf
Mon, Oct 23	Detection with sliding windows: Viola Jones	pptx, pdf	Szeliski 14.1 and 14.2
Wed, Oct 25	Detection with sliding windows: Dalal Triggs	pptx, pdf	Szeliski 14.1
Fri, Oct 27	Pascal VOC and Big Data	pptx, pdf	Szeliski 14.5
Mon, Oct 30	Big Data 2: im2gps and Tiny Images	pptx, pdf		Project 5 out
Wed, Nov 1	Crowdsourcing and Human Computation	pptx, pdf
Fri, Nov 3	Attributes	pptx, pdf
Mon, Nov 6	Modern boundary detection	pptx, pdf	Szeliski 4.2
Wed, Nov 8	Scene Parsing / Semantic Segmentation	pptx, pdf
Deep Learning
Fri, Nov 10	Neural networks Basics	pptx, pdf
Mon, Nov 13	Convolutional Networks	pptx, pdf
Wed, Nov 15	Object Detectors Emerge in Deep Scene CNNs	pdf		Project 6 out
Fri, Nov 17	COCO and Deeper Deep Architectures	pptx, pdf, resnet pdf
Mon, Nov 20	Structured Output from Deep Networks	pptx, pdf
Wed, Nov 22	No classes, Institute holiday
Fri, Nov 24	No classes, Institute holiday
Mon, Nov 27	Guest Lecture: Deep Learning for Semantic Segmentation by Fisher Yu	pdf
Wed, Nov 29	"Unsupervised" Learning and Colorization	pptx, pdf
Fri, Dec 1	Quiz 2
Mon, Dec 4	No classes, final instructional period
Wed, Dec 6	No classes, reading period
	Final Exam Period - not used

Acknowledgements

The materials from this class rely significantly on slides prepared by other instructors, especially Derek Hoiem and Svetlana Lazebnik. Each slide set and assignment contains acknowledgements. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgements.