CS 7643 Deep Learning
Fall 2017, TR 4:30 - 5:45 pm, Clough 144
This is an exciting time to be studying (Deep) Machine Learning, or Representation Learning, or for lack of a better term, simply Deep Learning!
Deep Learning is rapidly emerging as one of the most successful and widely applicable set of techniques across a range of domains (vision, language, speech, reasoning, robotics, AI in general), leading to some pretty significant commercial success and exciting new directions that may previously have seemed out of reach.
This course will expose students to cutting-edge research — starting from a refresher in basics of neural networks, to recent developments. The course is structured in “modules” (background, CNNs, RNNs, Deep Structured Prediction, Deep Reinforcement Learning). Each “module” will begin with instructor lectures to present context and background material. The emphasis will then switch to student-led paper presentations and discussions.
- Dhruv Batra
- Teaching Assistants
- Abhishek Das
- Office Hours
- Abhishek Das: Thursday, 11:00 am - 12:00 pm, CCB 2nd Floor Common Area
Michael Cogswell: Wednesday, 4:30 pm - 5:30 pm, CCB 2nd Floor Common Area
Zhaoyang Lv: Tuesday, 3:30 pm - 4:30 pm, CCB 2nd Floor Common Area
- Class meets
- Tue, Thu 4:30 - 5:45 pm, Clough 144
- Staff Mailing List
PS = Problem Set (written)
HW = Homework (implementation)
HR = Historically Relevant
|Date||Topic||Primary Reading||Optional Reading|
|W1: Aug 22||
Class Administrativia, HW0 out
Slides (pptx), Slides (pdf).
|W1: Aug 24||
|W2: Aug 29||
Background: Notation + supervised learning review, linear classifiers, modular design of neural networks
Slides, Slides (annotated).
|W2: Aug 31||Background: Regularization, modular design of neural networks, computing gradients Slides, Slides (annotated), Notes.|
|W2: Sep 1||PS1 out|
|W3: Sep 5||
Computational Graphs, Computing Gradients, Forward mode vs Reverse mode AD
Slides, Slides (annotated),
W3: Sep 6
12:30 - 1:30 pm (Lunch at Noon)
Marcus Nanotechnology Building 1118
Invited talk: Soumith Chintala
PyTorch, Static vs. Dynamic Computation Graphs
|W3: Sep 7||
Invited talk: Nathan Silberman
TF-Slim: A lightweight library for defining, training and evaluating complex models in TensorFlow
|W3: Sep 8||PS1 due
|W4: Sep 12||No class|
|W4: Sep 14||(Finish) Forward mode vs Reverse mode AD, and Convolutions. Slides, Slides (annotated),|
|W5: Sep 19||
CNNs: Convolutions, stride, padding, 1x1 convolutions
CNN Architectures: AlexNet, NIN, VGG/GoogLeNet, ResNets
|W5: Sep 21|
|W5: Sep 22||HW1 due|
|W6: Sep 26||Segmentation CNNs, Visualizing CNNs|
|W6: Sep 28||Fooling CNNs and Adversarial Examples|
|W7: Oct 3||No class|
|W7: Oct 5|
|W8: Oct 10||Fall student recess: no class|
|W8: Oct 12|
|W9: Oct 17|
|W9: Oct 19|
|W10: Oct 24||ICCV: no class|
|W10: Oct 26||ICCV: no class|
|W11: Oct 31|
|W11: Nov 2|
|W12: Nov 7|
|W12: Nov 9|
|W13: Nov 14||CVPR Deadline: no class|
|W13: Nov 16|
|W14: Nov 21|
|W14: Nov 23||Thanksgiving break: no class|
|W15: Nov 28|
|W15: Nov 30|
|W16: Dec 5||NIPS: no class|
Late policy for deliverables
- No penalties for medical reasons (bring doctor’s note) or emergencies.
- Every student has 7 free late days (7 x 24-hour chunks) for this course; these cannot be used for HW0, the reviews or the presentation.
- After all free late days are used up, penalty is 25% for each additional late day.
CS 7643 is an ADVANCED class. This should not be your first exposure to machine learning. Ideally, you need:
- Intro-level Machine Learning
- CS 7641/ISYE 6740/CSE 6740 or equivalent
- Dynamic programming, basic data structures, complexity (NP-hardness)
- Calculus and Linear Algebra
- positive semi-definiteness, multivariate derivates (be prepared for lots and lots of gradients!)
- This is a demanding class in terms of programming skills.
- HWs will involve a mix of languages (Python, C++) and libraries (PyTorch).
- Your language of choice for project.
- Ability to deal with abstract mathematical concepts
The class is full. Can I still get in?
It is unlikely. The class is full, with a 200 person wait-list. It is unlikely that the class size will increase further. We recommend coming to the first class. There will be a HW0 released on day 1 to check background preparation. Our past experience with HW0 suggests that slots will open up after some students drop the class.
I have not taken an graduate-level “Machine Learning” class or I am taking it in parallel. Can I still take this class?
No. Graduate-level machine learning is a pre-requisite.
Can I audit this class or take it pass/fail?
No. Due to the large demand for this class, we will not be allowing audits or pass/fail. Letter grades only. This is to make sure students who want to take the class for credit can.
Can I simply sit in the class (no credits)?
In general, we welcome members of the Georgia Tech community (students, staff, and/or faculty) to sit-in. Out of courtesy, we would appreciate if you let us know beforehand (via email or in person). If the classroom is full, we would ask that you please allow registered students to attend.
I have a question. What is the best way to reach the course staff?
Registered students – your first point of contact is Piazza (so that other students may benefit from your questions and our answers). If you have a personal matter, email us at the class mailing list firstname.lastname@example.org.
- You are encouraged to try out interesting applications of deep learning (vision, NLP, computational biology, UAVs, etc!)
- The project must be done in this semester. No double counting.
- You may combine with other course project but must delineate the different parts
- Extra credit for shooting a publication
- Main Categories:
- Application/survey - compare a bunch of algorithms on a new application domain of your interest
- Formulation/Development - Formulate a new model or algorithm for a new/old problem
- Theory: Theoretically analyze an existing algorithm.
Related Classes / Online Resources
- CS231n Convolutional Neural Networks for Visual Recognition, Stanford
- Machine Learning, Oxford
- Deep Learning, New York University
- Deep Learning, CMU
- Deep Learning, University of Maryland
- Hugo Larochelle’s Neural Networks class
Note to people outside Georgia Tech
Feel free to use the slides and materials available online here. If you use our slides, an appropriate attribution is requested. Please email the instructor with any corrections or improvements.