COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Regular lectures will be Tue & Th 4:00PM - 5:15PM, Computer Science Labs E110 (new CS building).
  • Slides will be finalized after the lecture and echo360 recordings (if available) accessible via Canvas.
Event TypeDateDescriptionCourse Materials
Lecture Thursday, Jan 29 Course logistics and overview
Historical context		 [slides]
[python/numpy tutorial]
[software setup for assignments]
Lecture Tuesday, Feb 3 Image classification and the data-driven approach
  • K-nearest neighbor
  • Linear classification
Optimization
  • Loss function
 [slides]
[image classification notes]
[linear classification notes]
Lecture Thursday, Feb 5 Optimization
  • Loss function
  • Regularization
  • Gradient computation
[slides]
[optimization and sgd notes]
Lecture Tuesday, Feb 10 Optimization
  • Stochastic gradient descent
  • Optimization challenges
  • Momentum, AdaGrad, Adam
[slides]
Lecture Thursday, Feb 12 Optimization
  • Learning rate schedules
Neural networks
 [slides]
[Neural Nets notes 1]
Lecture Tuesday, Feb 17 Backpropagation
Vector, matrix, and tensor derivatives
 [slides]
[backprop notes]
handout 1: vector, matrix, and tensor derivatives
handout 2: derivatives, backpropagation, and vectorization
[efficient backprop] (optional)
related: [1], [2], [3] (optional)
Thursday, Feb 19 No class, Monday schedule
Lecture Tuesday, Feb 24 Training neural networks
  • Activation functions
  • Data preprocessing
  • Weight initialization
  • Batch normalization
 [slides]
[neural nets notes 2]
[batch norm]
[Bengio 2012]
Lecture Thursday, Feb 26 Training neural networks
  • Regularization
  • Model ensembles and dropout
  • Data augmentation
  • Hyperparameter search
 [slides]
[neural nets notes 3]
Optional Discussion Friday, Feb 27 Vector, Matrix, Tensor Derivatives and Chain Rule [slides]
Lecture Tuesday, Mar 3 Image classification with CNNs
 [slides]
[conv net notes]
LeNet (optional)
Lecture Thursday, Mar 5 CNN architectures
Tuesday, Mar 10 Midterm #1 in class
Lecture Thursday, Mar 12 Project guidelines, resources, ideas
Tuesday, Mar 17 No class, Spring break
Thursday, Mar 19 No class, Spring break
Lecture Tuesday, Mar 24 Convnets for object detection
Lecture Thursday, Mar 26 Convnets for image segmentation
Lecture Tuesday, Mar 31 Recurrent networks
Lecture Thursday, Apr 2 Adversarial examples, texture synthesis, style transfer
Lecture Tuesday, Apr 7 Understanding and visualizing convnets
Lecture Thursday, Apr 9 Transformers I
Lecture Tuesday, Apr 14 Transformers II
Lecture Thursday, Apr 16 Self-supervised learning I
Lecture Tuesday, Apr 21 Self-supervised learning II
Lecture Thursday, Apr 23 Generative models
Tuesday, Apr 28 Midterm #2 in class
Lecture Thursday, Apr 30 Multi-modal learning
Lecture Tuesday, May 5 Project presentations (tentative)
Lecture Thursday, May 7 Project presentations (tentative)