COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Regular lectures will be Tue & Th 1:00PM - 2:15PM, Thompson Hall, Room 106.
  • There will be a few optional discussion sections organized by the TAs on Fridays (shown in green)
  • Slides will be finalized after the lecture and Echo360 recordings accessible via Canvas.
Event TypeDateDescriptionCourse Materials
Lecture Tuesday, Sep 3 Course logistics and overview
Historical context		 [slides]
[python/numpy tutorial]
[software setup for assignments]
Lecture Thursday, Sep 5 Image classification and the data-driven approach
  • K-nearest neighbor
  • Linear classification
Optimization
  • Loss function I
 [slides]
[image classification notes]
[linear classification notes]
Lecture Tuesday, Sep 10 Optimization
  • Loss function II
  • Regularization
  • Gradient computation
[slides]
Lecture Thursday, Sep 12 Optimization:
  • Stochastic gradient descent
  • Optimization challenges
  • Momentum, AdaGrad, Adam
[slides]
[optimization and sgd notes]
Optional discussion Friday, Sep 13, 11-12pm, CS 142 Python setup, Google collab, Basics of Python and Numpy [Notes]
Lecture Tuesday, Sep 17 Learning rate schedules
Neural networks
 [slides]
Lecture Thursday, Sep 19 Backpropagation
Vector, matrix, and tensor derivatives
 [slides]
[backprop notes]
handout 1: vector, matrix, and tensor derivatives
handout 2: derivatives, backpropagation, and vectorization
[efficient backprop] (optional)
related: [1], [2], [3] (optional)
Optional discussion Friday, Sep 20, 11-12pm, CS 142 Reviewing the chain rule, Applying the chain rule to vectors [slides]
Lecture Tuesday, Sep 24 Training neural networks
Activation Functions
 [slides]
Lecture Thursday, Sep 26 Training neural networks II
Weight initialization
Batch normalization 		[slides]
[neural nets notes 2]
[batch norm]
Optional discussion Friday, Sep 27, 11-12pm, CS 142 Vector, Matrix, and Tensor Derivatives [slides]
Lecture Tuesday, Oct 1 Project I
Expectations and timeline
Overview of projects by the TAs
 [slides 1]
[slides 2]
[project ideas]
Lecture Thursday, Oct 3 Project II
Work on project proposal in class
Optional discussion Friday, Oct 4, 11-12pm, CS 142 Batch normalization [slides]
Lecture Tuesday, Oct 8 Training neural networks III
Hyperparameter optimization
Model ensembles, dropout
Convolutional neural networks
 [slides]
Lecture Thursday, Oct 10 Guest lecture: Xiaolong Wang (UIUC)
Tuesday, Oct 15 No class, Monday schedule
Lecture Thursday, Oct 17 Convnets for spatial localization I:
Object detection
 [slides]
Lecture Tuesday, Oct 22 Convnets for spatial localization II:
Image segmentation
 [slides]
Lecture Thursday, Oct 24 Guest Lecture: Alex Wong (Yale)
The know-how of multimodal depth perception
Lecture Tuesday, Oct 29 Understanding and visualizing convnets
 [slides]
[lecture recording, 2023]; this year's version has missing audio
[visualization notes]
Lecture Thursday, Oct 31 Guest lecture: Boqing Gong (Google, BU)
From domain adaptation to videoprism
Tuesday, Nov 5 No class, Election Day
Lecture Thursday, Nov 7 Adversarial examples, Neural texture synthesis and style transfer [slides]
Lecture Tuesday, Nov 12 Recurrent neural networks [slides]
Lecture Thursday, Nov 14 Transformers [slides]
Lecture Tuesday, Nov 19 Self-supervised learning [slides]
Lecture Thursday, Nov 21 Guest lecture: Chen Sun (Brown)
Title: Grounding deep generative models in the physical world
Lecture Tuesday, Nov 26 No regular class, work on projects
Thursday, Nov 28 No class, Thanksgiving Break
Lecture Tuesday, Dec 3 Project presentations, Group 1 [slides]
Lecture Thursday, Dec 5 Project presentations, Group 2 [slides]
Lecture Tuesday, Dec 10 Project presentations, Group 3 [slides]