Your course project is an opportunity to explore an interesting problem using real-world datasets within the broad domain of computer vision. This includes, but is not limited to, topics such as machine learning over visual data, interaction with visual data, computational photography, computer graphics, vision-language models, and the application of computer vision in fields like medical imaging, ecology, astronomy, remote sensing, and beyond. Interdisciplinary and novel applications are highly encouraged, and it is also perfectly acceptable to extend an existing paper in novel directions—whether by exploring different assumptions, datasets, or constraints. The ultimate goal is to develop a project that, in ideal situations, could be submitted to a workshop at a computer vision conference.

While the instructor and TAs will provide guidance and feedback, the responsibility for defining and executing an impactful piece of work lies with you and your team. This project is designed to teach you how to conduct research in the field of computer vision, including determining relevant related work, performing experiments, and clearly communicating your results. Your project will constitute 100% of your final grade and will be evaluated based on originality, technical depth, quality of writing, and presentation skills. Active participation from all team members in each of the presentations is required, and clear, well-structured class presentations and a polished final poster presentation are crucial to achieving a successful grade. The project has seven required deliverables:

  1. Project Proposal Ideas: Individually propose 3 project ideas (10%)
  2. Proposal Report: 2 pages excluding references (10%)
  3. Proposal Presentation: Teams present their proposed projects in 3 minutes (10%)
  4. Midpoint Report: 4 pages excluding references (15%)
  5. Midpoint Presentation: Teams present their midpoint results in 4 minutes (15%)
  6. Final Report: 8 pages excluding references (20%)
  7. Final Presentation: Poster presentation in 5 minutes (20%)

All write-ups should use the CVPR style.

Project Proposal Ideas

Each student must individually propose three potential research projects. These proposals should be concise, offering a clear overview of the idea or hypothesis, along with practical considerations for data and compute resources. While the ideas do not need to be fully developed, the proposals should outline a rough scope of work and a basic plan for data and compute needs. Each proposal should be around two paragraphs and include citations to relevant literature.

Each project proposal should include the following elements:

  • Title
  • Overview: A brief description of the idea, its goals, and its connection to existing work.
  • Data Requirements: The source, volume, and storage plan for the required data.
  • Compute Requirements: Estimated compute needs, including whether GPUs or other specialized hardware are required.

Grading Breakdown

  • 40%: Clarity and conciseness of the project descriptions.
  • 30%: Quality and relevance of the literature survey.
  • 30%: Practicality of the data and compute requirements.

Team Formation and Project Proposals

Team Formation

After submitting individual project ideas, students will have the opportunity to view all proposed ideas and form teams of 2-3 members. Teams will be fixed for the remainder of the semester. Individual projects will only be considered under exceptional circumstances. Once teams are formed, they will select and refine a single project from the pool of ideas to focus on for the semester.

Project Proposal

Each team will submit a two-page project proposal (excluding references) that outlines their chosen project. The proposal should be concise but thorough, covering the following elements:

  • Title and Group Members: The project title and names of all team members.
  • Overview: A clear description of the project idea, including the main question or hypothesis, project goals, and what existing work the project will build upon or extend. Explain the expected beneficial outcome of the project.
  • Literature Survey: A review of relevant work, including related research projects and sources of inspiration.
  • Data Requirements: Specify if the project requires data or annotations. Detail where the data will come from, how much data is needed, and how it will be stored and accessed.
  • Compute Requirements: Describe any specific compute needs for the project (e.g., GPU requirements) and how the necessary compute resources will be obtained.
  • Plan of Activities: Outline the expected milestones, including what the team aims to complete by the midpoint presentation and the final poster presentation.

Grading Breakdown for Project Proposal

  • 25%: Clarity and conciseness of the proposed method.
  • 25%: Quality of the literature survey.
  • 25%: Detailed and realistic data and compute requirements.
  • 25%: Well-structured and thought-out plan of activities.

Project Proposal Presentation

Teams will prepare a 3-minute presentation to share their project proposal with the class. The order of presentations will be randomized, and all teams will use a shared Google Slides deck, presenting one after the other. Teams must adhere strictly to the 3-minute time limit; they may use less time if desired, but cannot exceed 3 minutes. All team members must be present and must participate in the presentation unless precluded by illness. Depending on the number of teams, there may be 1-2 minutes for questions and feedback if time permits.

Grading Breakdown for Proposal Presentation

  • 30%: Clear description of the project and its motivation.
  • 30%: Feasibility and thoroughness of the plan of activities.
  • 40%: Clarity and effectiveness of the presentation.

Midpoint Project Report and Presentation

Midpoint Project Report

Teams are required to submit a 4-page midpoint project report (excluding references) that extends their initial project proposal. Submissions cannot exceed 4-pages, excluding references. The midpoint report should include the following:

  • Preliminary Results: Present a preliminary set of results from your experiments.
  • Results Analysis: Provide a detailed analysis of these preliminary results, discussing what they indicate and any initial conclusions you can draw.
  • Additional Experiments and Analysis: Outline the next steps, including additional experiments and further analysis that will be conducted to refine your project.

The project description, related work, and dataset and compute requirements sections should be updated to reflect any new insights or adjustments as the project scope becomes clearer.

Grading Breakdown for Midpoint Project Report

  • 25%: Clarity and conciseness of the project method description and an explantion of novelty.
  • 10%: Quality and thoroughness of the literature survey.
  • 15%: Clear description of experiment procedure, including data and compute requirements.
  • 40%: Clear description and effective visualization of preliminary experiment results.
  • 10%: Well-defined plan for the next set of experiments and analyses.

Midpoint Presentation

Teams will also deliver a 4-minute presentation to update the class on their project’s progress. The order of presentations will be randomized, and all teams will present using a shared Google Slides deck. Each team will present one after the other. Teams must strictly adhere to the 4-minute time limit; they may use less time if needed, but cannot exceed 4 minutes. All team members must be present and must participate in the presentation unless precluded by illness. Depending on the number of teams, there may be 1-2 minutes for questions and feedback if time permits.

Grading Breakdown for Midpoint Presentation

  • 25%: Clear description and motivation of the project and your method.
  • 25%: Presentation of current results and their significance.
  • 25%: Feasibility and clarity of the plan for upcoming activities.
  • 25%: Clarity and engagement of the presentation.

Final Project Report and Poster Presentation

Final Project Report

Teams will extend their midpoint project reports to include the final results, an analysis of those results, and the overall conclusions drawn from the project. Additionally, the project description, related work, and dataset and compute requirements sections should be finalized to reflect the full scope and refinement of the project. Submissions cannot exceed 8-pages, excluding references.

The final report should include the following:

  • Final Results: A complete presentation of the final experimental results.
  • Results Analysis: A thorough analysis of the final results, discussing their significance and implications.
  • Conclusions and Findings: Summarize the overall conclusions and key findings from the project.

Grading Breakdown for Final Project Report

  • 5%: Introduction: project motivation and summarized contributions
  • 10%: Related work: clear overview of prior work in relevant topic areas; explicit differences or extensions to prior work; thoroughness
  • 15%: Method: description of components
  • 25%: Experiments: description, completeness, ablations, reproducibility, compute & data needs
  • 25%: Results: analysis and discussion, adequate comparisons to prior work, takeaways and conclusions
  • 20%: Figures and tables: usefulness, clarity, descriptive captions

Final Poster Presentation

At the end of the semester, all teams are required to create a poster summarizing their project and present it during a designated poster session. Details on poster size, formatting, and printing deadlines will be provided later in the semester. All team members must be present for the poster presentation, and each member must participate in the presentation (unless precluded by illness). The presentation order will be randomized.

During the session, each team will have exactly 5 minutes to present their poster, regardless of team size. Teams may choose to use less time but cannot exceed the 5-minute limit. Following each presentation, the instructor and TAs will ask questions. While other teams are presenting, students are expected to listen to a randomly assigned subset of their peers’ presentations, though they are encouraged to view all posters.

Grading Breakdown for Final Poster Presentation

  • 25%: Clear description of the project and its motivation.
  • 25%: Clear explanation of the data and methods used.
  • 25%: Clear presentation of results and analysis.
  • 25%: Quality of the poster layout, including the use of minimal but informative text, and the effectiveness of figures and visualizations.

Project Suggestions

You are encouraged to propose your own topics – some of you already may have done so. Take a look at the the resources listed at the end of this page for potential topics. Below are some ideas:

  • Take a look at the latest papers from CVPR, ECCV, ICCV, NeurIPS, and ICML to find topics, software, datasets which you can build upon.
  • Also check out the workshops associated with these conferences. For example, take a look at the recent Fine-grained Visual Recognition workshops for datasets and Kaggle challenges related to fine-grained classifiction tasks such as recognizing animal species, or product images.
  • The website https://paperswithcode.com tracks the state of the art across datasets. This is a quick way to find baselines to compare with or build upon.
  • The wesbite https://registry.opendata.aws contains a number of publicly available datasets hosted on AWS. These include satellite imagery, RADAR and other data on which you can try out some computer vision techniques. For example see MistNet and RoostNet projects (contact Zezhou for details).
  • Explore the use of computer vision services on the cloud to solve some challenging problems. Some choices are AWS Rekognition, Google cloud, and Microsoft Azure.
  • Generative modeling: Train and generate data on novel domains using GAN, VQ-GAN, Diffusion models. Build an interface for interactively edit images.
  • Probing and understanding language and vision models: Models such as CLIP (OpenAI) and ALIGN (Google) train multi-modal aligned representations of images and text, which can be applied for example to retrive images based on text, or even generate images based on language (e.g., DALL-E). Huggingface has open-source implementation of many of these models – try to probe these models on new datasets to understand their capabilities and biases.

A sample of projects from a prior offering of the course:

  • Scene text recognition
  • Improving object detection using depth estimation
  • Dust removal from images
  • Fast face-retrieval using vocabulary trees on deep features
  • Hyperspectral image classification
  • Character recognition in movies
  • Could motion analysis
  • Analysis of medical images
  • Stereo reconstruction survey
  • Counting heads in images
  • Implementation of a VR engine
  • Poselet based person identification
  • Gaze tracker
  • Photo stitching across seasons/day-night
  • Segmentation using CNNs

Computing Resources

Some vision projects may involve large scale data and require GPU computing resources. We recommend you to check out “AWS Education” and “Google Cloud Platform”.

  • AWS: https://aws.amazon.com/education/awseducate
    • UMass is an “AWS member institution”, so you are in the higher allowance tier. Use your .edu email and the full school name “University of Massachusetts Amherst” when you register to get the full benefits (a total of $100 annually).
    • To get GPUs, use g3 (up to 4 NVIDIA Tesla M60 GPUs) or p2 (up to 16 NVIDIA K80 GPUs) instances in EC2. Check the pricing first and make your plan accordingly!
  • Google Cloud Platform: https://cloud.google.com
    • You get $300 credits for the first 12 months, and always free on their free-tier resources (not including GPUs)