COMPSCI 682 Neural Networks: A Modern Introduction

Important Dates (tentative)

Course project proposal due: 10/3
Course project milestone due: 10/31
Final course project write-up due: 12/2
Project presentations: 12/03 - 12/10

Overview

The course project is an opportunity for you to apply what you have learned in class to a problem of your interest.

Your are encouraged to select a topic and work on your own project. Potential projects usually fall into these two tracks:

• Applications. If you're coming to the class with a specific background and interests (e.g. biology, engineering, physics), we'd love to see you apply deep neural networks to problems related to your particular domain of interest. Pick a real-world problem and apply deep neural networks to solve it.
• Models. You can build a new model (algorithm) with deep neural networks, or a new variant of existing models, and apply it to tackle vision tasks. This track might be more challenging, and sometimes leads to a piece of publishable work.

Here you can find some sample project ideas:

• Sample project ideas from TAs, Fall 2024 (Google docs)
• Sample project ideas from TAs, Fall 2023 (Google Docs)
• Sample project ideas from Prof. Erik Learned-Miller last semester (Google Docs)
• [Stanford cs231n project reports: spring 2017]
• [Stanford cs231n project reports: winter 2016]
• [Stanford cs231n project reports: winter 2015

To inspire ideas, you might look at recent deep learning publications from top-tier vision conferences, as well as other resources below.

• Awesome Deep Vision
• CVPR: IEEE Conference on Computer Vision and Pattern Recognition
• ICCV: International Conference on Computer Vision
• ECCV: European Conference on Computer Vision
• NIPS: Neural Information Processing Systems
• ICLR: International Conference on Learning Representations
• Kaggle challenges: An online machine learning competition website. For example, a Yelp classification challenge.

For applications, this type of projects would involve careful data preparation, an appropriate loss function, details of training and cross-validation and good test set evaluations and model comparisons. Don't be afraid to think outside of the box. Some successful examples can be found below:

• Teaching Deep Convolutional Neural Networks to Play Go
• Playing Atari with Deep Reinforcement Learning
• Winning the Galaxy Challenge with convnets

Deep neural networks also run in real time on mobile phones and Raspberry Pi's - feel free to go the embedded way. You may find this TensorFlow demo on Android helpful.

For models, deep neural networks have been successfully used in a variety of computer vision and NLP tasks. This type of projects would involve understanding the state-of-the-art vision or NLP models, and building new models or improving existing models. The list below presents some papers on recent advances of deep neural networks in the computer vision community.

• Object recognition: [Krizhevsky et al.], [Russakovsky et al.], [Szegedy et al.], [Simonyan et al.], [He et al.]
• Object detection: [Girshick et al.], [Sermanet et al.], [Erhan et al.]
• Image segmentation: [Long et al.]
• Video classification: [Karpathy et al.], [Simonyan and Zisserman]
• Scene classification: [Zhou et al.]
• Face recognition: [Taigman et al.]
• Depth estimation: [Eigen et al.]
• Image-to-sentence generation: [Karpathy and Fei-Fei], [Donahue et al.], [Vinyals et al.]
• Visualization and optimization: [Szegedy et al.], [Nguyen et al.], [Zeiler and Fergus], [Goodfellow et al.], [Schaul et al.]

We also provide a list of popular computer vision datasets:

• Meta Pointer: A large collection organized by CV Datasets.
• Yet another Meta pointer
• ImageNet: a large-scale image dataset for visual recognition organized by WordNet hierarchy
• SUN Database: a benchmark for scene recognition and object detection with annotated scene categories and segmented objects
• Places Database: a scene-centric database with 205 scene categories and 2.5 millions of labelled images
• NYU Depth Dataset v2: a RGB-D dataset of segmented indoor scenes
• Microsoft COCO: a new benchmark for image recognition, segmentation and captioning
• Flickr100M: 100 million creative commons Flickr images
• Labeled Faces in the Wild: a dataset of 13,000 labeled face photographs
• Human Pose Dataset: a benchmark for articulated human pose estimation
• YouTube Faces DB: a face video dataset for unconstrained face recognition in videos
• UCF101: an action recognition data set of realistic action videos with 101 action categories
• HMDB-51: a large human motion dataset of 51 action classes

Grading Policy

  Final Project: 50% of final grade
  - Proposal: 5% of final grade
  - Milestone: 15% of final grade
  - Final Report: 25% of final grade
     write-up (20% of Final Report):
      •  clarity, structure, language, references
      •  background literature survey, good understanding of the problem
      •  good insights and discussions of methodology, analysis, results, etc.
     technical (30% of Final Report):
      •  correctness
      •  depth
      •  innovation
     evaluation and results (30% of Final Report):
      •  sound evaluation metric
      •  thoroughness in analysis and experimentation
      •  results and performance
  - Project Presentation: 5% of final grade
  

Project Proposal

The project proposal should be concise (~400 words). You can use the provided template. Your proposal should contain:

• Group Members: Who are the (2~3) group members? What will each person do? (This needs to be a separate detailed paragraph)
• Motivation: What is the problem that you will be investigating? Why is it interesting?
• Literature Review: What reading will you examine to provide context and background?
• Data: What data will you use? If you are collecting new datasets, how do you plan to collect them? If the datasets are huge what compute resources are you using?
• Approach: What method or algorithm are you proposing? If there are existing implementations, will you use them and how? How do you plan to improve or modify such implementations?
• Evaluation Metric: How will you evaluate your results? Qualitatively, what kind of results do you expect (e.g. plots or figures)? Quantitatively, what kind of analysis will you use to evaluate and/or compare your results (e.g. what performance metrics or statistical tests)?
• References: Bibliography of papers based on which your project idea is based.

Submission: Please upload a PDF file to Gradescope. Please coordinate with your teammate and submit only under ONE of your accounts, and add your teammate on Gradescope.

Project Milestone

Your project milestone report should be between 2 - 3 pages using the template (pdf, latex source). The following is a suggested structure for your report:

• Title, Author(s)
• Introduction: This section introduces your problem, motivation, and the overall plan. It should describe your problem precisely specifying the dataset to be used, expected results and evaluation.
• Related work: A literature survey of past work on this topic. Introduce the baselines you will compare to and the weakness you plan to address. This section should be nearly complete.
• Technical Approach: Describe the methods you intend to apply to solve the given problem.
• Intermediate/Preliminary Results: State and evaluate your results upto the milestone.

Submission: Please upload a PDF file to Gradescope. Please coordinate with your teammates and submit only under ONE of your accounts, and add your teammates on Gradescope.

Final Submission

Your final write-up should be between 6 - 8 pages using the same template as the milestone report (pdf, latex source) and fully flesh out the sections in the milestone. After the class, we may post all the final reports online so that you can read about each others' work. If you do not want your writeup to be posted online, then please let us know at least a week in advance of the final writeup submission deadline.

Project Presentation

All teams will present their work during class in the last three lectures. The order will be randomized and you are expected to be present in all of the three days. Each team will get 2-3 minutes (depending on the number of projects) to present thier work. This is similar to a spotlight presentation at conferences such as CVPR and NeurIPS. Note, the remote participants will be asked to pre-record and upload a video of their presentation.

More details on the presentation format will be provided later in the semester.

Collaboration Policy

You can work in teams of 2 or 3 people. Individual projects or larger group sizes are permitted only in exceptional circumstances (e.g., remote students, etc) and need prior permission from the instructor. Similar, group projects larger than three We do expect that projects done with 3 people have more impressive writeup and results than 2 person projects.

Honor Code

You may consult any papers, books, online references, or publicly available implementations for ideas and code that you may want to incorporate into your strategy or algorithm, so long as you clearly cite your sources in your code and your writeup. However, under no circumstances may you look at another group’s code or incorporate their code into your project.