COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Regular lectures will be Tue & Th 1:00PM - 2:15PM, Thompson Hall, Room 106.
  • There will be a few optional discussion sections organized by the TAs on Fridays (shown in green)
  • Slides will be finalized after the lecture and Echo360 recordings accessible via Canvas.
Event TypeDateDescriptionCourse Materials
Lecture Tuesday, Sep 3 Course logistics and overview
Historical context		 [slides]
[python/numpy tutorial]
[software setup for assignments]
Lecture Thursday, Sep 5 Image classification and the data-driven approach
  • K-nearest neighbor
  • Linear classification
Optimization
  • Loss function I
 [slides]
[image classification notes]
[linear classification notes]
Lecture Tuesday, Sep 10 Optimization
  • Loss function II
  • Regularization
  • Gradient computation
[slides]
Lecture Thursday, Sep 12 Optimization:
  • Stochastic gradient descent
  • Optimization challenges
  • Momentum, AdaGrad, Adam
[slides]
[optimization and sgd notes]
Optional discussion Friday, Sep 13, 11-12pm, CS 142 Python setup, Google collab, Basics of Python and Numpy [Notes]
Lecture Tuesday, Sep 17 Learning rate schedules
Neural networks
 [slides]
Lecture Thursday, Sep 19 Backpropagation
Vector, matrix, and tensor derivatives
 [slides]
[backprop notes]
handout 1: Vector, Matrix, and Tensor Derivatives
handout 2: Derivatives, Backpropagation, and Vectorization
[Efficient BackProp] (optional)
related: [1], [2], [3] (optional)
Optional discussion Friday, Sep 20, 11-12pm, CS 142 Reviewing the chain rule, Applying the chain rule to vectors
Lecture Tuesday, Sep 24 Training neural networks
Weight initialization
Batch normalization
Lecture Thursday, Sep 26 Training neural networks II
Babysitting the learning process
Hyperparameter optimization
Optional discussion Friday, Sep 27, 11-12pm, CS 142 Vector, Matrix, and Tensor Derivatives
Lecture Tuesday, Oct 1 Project discussion led by TAs
Expectations and timeline
Overview of projects outlined by the TAs
Lecture Thursday, Oct 3 Training neural networks
Ensembles, dropout
Convolutional neural neoworks
Optional discussion Friday, Oct 4, 11-12pm, CS 142 Batch normalization
Lecture Tuesday, Oct 8 Convnets for spatial localization I
Object Detection
Lecture Thursday, Oct 10 Convnets for spatial localization II
Image Segmentation
Tuesday, Oct 15 No Class, Monday Schedule
Lecture Thursday, Oct 17 Understanding and visualizing convnets
Backprop into image: visualizations
Lecture Tuesday, Oct 22 Neural texture synthesis and style transfer
Lecture Thursday, Oct 24 Understanding and visualizing convnets
Backprop into image: visualizations
Lecture Tuesday, Oct 29 Neural Texture Synthesis and Style Transfer
Lecture Thursday, Oct 31 Generative AI
Tuesday, Nov 5 No class, Election Day
Lecture Thursday, Nov 7 Recurrent Neural Networks
Lecture Tuesday, Nov 12 Transformers
Lecture Thursday, Nov 14 (TBD)
Lecture Tuesday, Nov 19 (TBD)
Lecture Thursday, Nov 21 (TBD)
Lecture Tuesday, Nov 26 (TBD)
Thursday, Nov 28 No Class, Thanksgiving Break
Project Presentation Tuesday, Dec 3 Group 1
Project Presentation Thursday, Dec 5 Group 2
Project Presentation Tuesday, Dec 10 Group 3