High-level Vision: From Neurons to Deep Neural Networks

High-level Vision: From Neurons to Deep Neural Networks 

The goal of this seminar is to understand the underlying computational principles behind visual learning, combining techniques from experimental neuroscience, psychology, and neural network modeling. 

Heavily supervised deep neural networks are surprisingly accurate models of the adult human vision system. In contrast, they entirely fail as models of how this system comes to be in the first place. That is, they fail to explain how real systems learn and develops in interaction with the world, without access to large amount of supervision and labelled data. However, recent advances in unsupervised visual learning with deep neural networks provide exciting new ideas for computational modeling of visual learning and development. 

Here, we will discuss recent empirical findings from neuroscience and psychology regarding visual development, gleaning insight into how
visual experience builds visual representations in humans. We will also review major recent advances in unsupervised learning with deep neural networks, as well as the burgeoning field of interactive visual learning systems.  We will evaluate where the experimental data and neural network learning processes match up, and where they diverge — seeking to identify where the biggest opportunities for understanding human neural development and improving learning in neural networks lie.

Mondays/Wednesdays 4-5:20pm, zoom

Instructors: Prof. Kalanit Grill-Spector & Prof. Dan Yamins

Contact: kalanit@stanford.edu; yamins@stanford.edu

Requirements:

Readings:  We expect you to critically read each assigned paper and write comments in the form of bullet points for the paper. We will pick topics from your answers to jumpstart the zoom discussion on the relevant paper. 

Bullet points should include:

- the main findings/conceptual advancement of the study

- thoughts and questions about the impact and findings of the study

- constructive critique about the methods/experiments/conclusions

Participation: We will have synchronous classes and in-class discussions on each paper. We would like all students to participate in the class discussions, and we will lean on your reading commentaries to jumpstart these discussions.

Presentations: Each student will present several times during the quarter by presenting and leading discussion on one paper from the syllabus, presenting a section from a review paper, and participating in an in-class debate about a central debate in the field: Is visual experience necessary for the formation of functional domains?

For your paper presentation prepare to describe the methods, results, and conclusions of the paper. We expect a short presentation of 10-15 minutes to enable time for discussion. Be prepared to have discussion and questions peppered during the presentation rather than at the end of the presentation. 

Presentation assignments will be made at the beginning of the quarter.

End of term project:  Students enrolled for 3 units of credit, will be additionally required to submit a computational project implementing either a biological aspect of baby vision (for example, spatial filtering images according to the spatial acuity of infants) or a deep neural network that employs an unsupervised learning algorithm. 

Grading Rubric: 

1 unit:   25% readings, 25% participation, 50% presentations.  

3 units: 15% readings, 15% participation, 35% presentations, 35% project.

 

Schedule with links

Week 1.1: Introduction & processing streams

Week 1.2: Using goal-driven deep learning models to understand sensory cortex

Week 2.2: From brain measurements to deep neural networks.

Week 3.1: Synapic activity and the construction of cortical circuits

Week 3.2: What do babies actually see?

Week 4.1: What do babies & toddlers look at?

Week 4.2: Using what babies look at to train visual models

Week 5.1: How do babies decide what to look at?

Week 6.2: Debate: Is visual experience necessary for the formation of cortical domains?

Week 7.1: How do spatiotopic maps emerge in visual cortex?

Week 7.2: Five lessons from infant learning

Week 8.1: The possibility of self supervised learning.

Week 8.2: Unsupervised learning and the brain

Week 9.1: Using what babies look at to train networks to see

Week 9.2: What category representations develop first in the baby brain?

Week 10.1: Cortical recycling of category representations during childhood

Week 10.2: Semantic development models