CS 7643 Deep Learning

Fall 2017, TR 4:30 - 5:45 pm, Clough 144

Course Information

This is an exciting time to be studying (Deep) Machine Learning, or Representation Learning, or for lack of a better term, simply Deep Learning!

Deep Learning is rapidly emerging as one of the most successful and widely applicable set of techniques across a range of domains (vision, language, speech, reasoning, robotics, AI in general), leading to some pretty significant commercial success and exciting new directions that may previously have seemed out of reach.

This course will expose students to cutting-edge research — starting from a refresher in basics of neural networks, to recent developments. The course is structured in “modules” (background, CNNs, RNNs, Deep Structured Prediction, Deep Reinforcement Learning). Each “module” will begin with instructor lectures to present context and background material. The emphasis will then switch to student-led paper presentations and discussions.

Dhruv Batra

Teaching Assistants
Abhishek Das
Michael Cogswell
Zhaoyang Lv

Office Hours
Abhishek Das: Thursday, 11:00 am - 12:00 pm, CCB 2nd Floor Common Area
Michael Cogswell: Wednesday, 6:00 pm - 7:00 pm, CCB 2nd Floor Common Area
Zhaoyang Lv: Tuesday, 3:30 pm - 4:30 pm, CCB 2nd Floor Common Area

Class meets
Tue, Thu 4:30 - 5:45 pm, Clough 144

Staff Mailing List


PS = Problem Set (written)
HW = Homework (implementation)
HR = Historically Relevant

Date Topic Primary Reading Optional Reading
W1: Aug 22 Class Administrativia, HW0 out
Slides (pptx), Slides (pdf).
W1: Aug 24 HW0 due
No class
W2: Aug 29 Background: Notation + supervised learning review, linear classifiers, modular design of neural networks
Slides, Slides (annotated).
W2: Aug 31 Background: Regularization, modular design of neural networks, computing gradients Slides, Slides (annotated), Notes.
W2: Sep 1 PS1 out
W3: Sep 5 Computational Graphs, Computing Gradients, Forward mode vs Reverse mode AD
Slides, Slides (annotated),
W3: Sep 6
12:30 - 1:30 pm (Lunch at Noon)
Marcus Nanotechnology Building 1118
Invited talk: Soumith Chintala
PyTorch, Static vs. Dynamic Computation Graphs
W3: Sep 7 Invited talk: Nathan Silberman
TF-Slim: A lightweight library for defining, training and evaluating complex models in TensorFlow
W3: Sep 8 PS1 due
HW1 out
W4: Sep 12 No class
W4: Sep 14 (Finish) Forward mode vs Reverse mode AD, and Convolutions. Slides, Slides (annotated), Notes.
W5: Sep 19 CNNs: Convolutions, stride, padding, 1x1 convolutions, Slides, Slides (annotated), Notes.
W5: Sep 21 transposed convolutions, dilated/atrous convolutions; CNN Architectures: AlexNet, NIN, VGG/GoogLeNet, ResNets
Slides, Slides (annotated).
W5: Sep 22 HW1 due
W6: Sep 26 Segmentation CNNs (and other pixel-level prediction)
Invited talk: Sumit Chopra
PS2 out (tex source)
HW2 out
Fully Convolutional Networks for Semantic Segmentation
W6: Sep 28 Visualizing CNNs
Slides, Slides (annotated).
Methods for Interpreting and Understanding Deep Neural Networks
W7: Oct 3 No class
W7: Oct 5 Invited talk: Stefan Lee
An Introduction to Visually Grounded Question Answering, Dialog, and Beyond
W8: Oct 10 Fall student recess: no class
W8: Oct 12 Fooling CNNs and Adversarial Examples Explaining and Harnessing Adversarial Examples
W8: Oct 13 PS2 due, HW2 due
W8: Oct 14 PS3 out (tex source)
HW3 out
W9: Oct 17 Recurrent Neural Networks
Slides, Slides (annotated), Notes.
W9: Oct 19 Sequence to Sequence Learning with Neural Networks
W10: Oct 24 ICCV: no class
W10: Oct 26 ICCV: no class
W10: Oct 28 PS3 due, HW3 due
W11: Oct 31 CNNs + RNNs Show and Tell: A Neural Image Caption Generator
W11: Nov 2 Deep Reinforcement Learning
Slides, Slides (annotated).
W12: Nov 7 Sequence Level Training with Recurrent Neural Networks
W12: Nov 9 Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
W13: Nov 14 CVPR Deadline: no class
W13: Nov 16 Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
W14: Nov 21 Variational Autoencoders and Generative Adversarial Networks
Slides, Slides (annotated).
W14: Nov 23 Thanksgiving break: no class
W15: Nov 28 Tutorial on Variational Autoencoders
W15: Nov 30 Generative Adversarial Networks
W16: Dec 5 NIPS: no class


  • 50% Homework (3 homeworks + 3 problem sets)
  • 15% Paper Presentation
  • 10% Paper Reviews
  • 20% Final Project
  • 5% Class Participation

Late policy for deliverables

  • No penalties for medical reasons (bring doctor’s note) or emergencies.
  • Every student has 7 free late days (7 x 24-hour chunks) for this course; these cannot be used for HW0, the reviews or the presentation.
  • After all free late days are used up, penalty is 25% for each additional late day.


CS 7643 is an ADVANCED class. This should not be your first exposure to machine learning. Ideally, you need:

  • Intro-level Machine Learning
    • CS 7641/ISYE 6740/CSE 6740 or equivalent
  • Algorithms
    • Dynamic programming, basic data structures, complexity (NP-hardness)
  • Calculus and Linear Algebra
    • positive semi-definiteness, multivariate derivates (be prepared for lots and lots of gradients!)
  • Programming
    • This is a demanding class in terms of programming skills.
    • HWs will involve a mix of languages (Python, C++) and libraries (PyTorch).
    • Your language of choice for project.
  • Ability to deal with abstract mathematical concepts


  • The class is full. Can I still get in?

    It is unlikely. The class is full, with a 200 person wait-list. It is unlikely that the class size will increase further. We recommend coming to the first class. There will be a HW0 released on day 1 to check background preparation. Our past experience with HW0 suggests that slots will open up after some students drop the class.

  • I have not taken an graduate-level “Machine Learning” class or I am taking it in parallel. Can I still take this class?

    No. Graduate-level machine learning is a pre-requisite.

  • Can I audit this class or take it pass/fail?

    No. Due to the large demand for this class, we will not be allowing audits or pass/fail. Letter grades only. This is to make sure students who want to take the class for credit can.

  • Can I simply sit in the class (no credits)?

    In general, we welcome members of the Georgia Tech community (students, staff, and/or faculty) to sit-in. Out of courtesy, we would appreciate if you let us know beforehand (via email or in person). If the classroom is full, we would ask that you please allow registered students to attend.

  • I have a question. What is the best way to reach the course staff?

    Registered students – your first point of contact is Piazza (so that other students may benefit from your questions and our answers). If you have a personal matter, email us at the class mailing list cs-7643-f17-staff@googlegroups.com.


  • You are encouraged to try out interesting applications of deep learning (vision, NLP, computational biology, UAVs, etc!)
  • The project must be done in this semester. No double counting.
  • You may combine with other course project but must delineate the different parts
  • Extra credit for shooting a publication
  • Main Categories:
    • Application/survey - compare a bunch of algorithms on a new application domain of your interest
    • Formulation/Development - Formulate a new model or algorithm for a new/old problem
    • Theory: Theoretically analyze an existing algorithm.

Note to people outside Georgia Tech

Feel free to use the slides and materials available online here. If you use our slides, an appropriate attribution is requested. Please email the instructor with any corrections or improvements.