COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Regular lectures will be Tue & Th 4:00PM - 5:15PM, Computer Science Labs E110 (new CS building).
  • Slides will be finalized after the lecture and echo360 recordings (if available) accessible via Canvas.
Event TypeDateDescriptionCourse Materials
Lecture Thursday, Jan 29 Course logistics and overview
Historical context		 [slides]
[python/numpy tutorial]
[software setup for assignments]
Lecture Tuesday, Feb 3 Image classification and the data-driven approach
  • K-nearest neighbor
  • Linear classification
Optimization
  • Loss function
 [slides]
[image classification notes]
[linear classification notes]
Lecture Thursday, Feb 5 Optimization
  • Loss function
  • Regularization
  • Gradient computation
[slides]
[optimization and sgd notes]
Lecture Tuesday, Feb 10 Optimization
  • Stochastic gradient descent
  • Optimization challenges
  • Momentum, AdaGrad, Adam
[slides]
Lecture Thursday, Feb 12 Optimization
  • Learning rate schedules
Neural networks
 [slides]
[Neural Nets notes 1]
Lecture Tuesday, Feb 17 Backpropagation
Vector, matrix, and tensor derivatives
 [slides]
[backprop notes]
handout 1: vector, matrix, and tensor derivatives
handout 2: derivatives, backpropagation, and vectorization
[efficient backprop] (optional)
related: [1], [2], [3] (optional)
Thursday, Feb 19 No class, Monday schedule
Lecture Tuesday, Feb 24 Training neural networks
  • Activation functions
  • Data preprocessing
  • Weight initialization
  • Batch normalization
 [slides]
[neural nets notes 2]
[batch norm]
[Bengio 2012]
Lecture Thursday, Feb 26 Training neural networks
  • Regularization
  • Model ensembles and dropout
  • Data augmentation
  • Hyperparameter search
 [slides]
[neural nets notes 3]
Optional Discussion Friday, Feb 27 Vector, Matrix, Tensor Derivatives and Chain Rule [slides]
Lecture Tuesday, Mar 3 Image classification with CNNs
 [slides]
[conv net notes]
LeNet (optional)
Lecture Thursday, Mar 5 CNN architectures
 [slides]
AlexNet, ZFNet, GoogleNet, VGGNet, ResNet
Tuesday, Mar 10 Midterm #1 in class
Lecture Thursday, Mar 12 CNN architectures
Project guidelines, resources, ideas
 [slides]
project guidelines
Tuesday, Mar 17 No class, Spring break
Thursday, Mar 19 No class, Spring break
Lecture Tuesday, Mar 24 Convnets for image segmentation
 [slides]
Lecture Thursday, Mar 26 Convnets for object detection
Lecture Tuesday, Mar 31 Recurrent networks
Lecture Thursday, Apr 2 Adversarial examples, texture synthesis, style transfer
Lecture Tuesday, Apr 7 Understanding and visualizing convnets
Lecture Thursday, Apr 9 Transformers I
Lecture Tuesday, Apr 14 Transformers II
Lecture Thursday, Apr 16 Self-supervised learning I
Lecture Tuesday, Apr 21 Self-supervised learning II
Lecture Thursday, Apr 23 Generative models
Tuesday, Apr 28 Midterm #2 in class
Lecture Thursday, Apr 30 Multi-modal learning
Lecture Tuesday, May 5 Project presentations (tentative)
Lecture Thursday, May 7 Project presentations (tentative)