New Approaches to 3D Vision
New volume from the The Royal Society
On AI, animal navigation, and human vision
Overview of articles below 👇🧵
- The Introduction asks:
Does 3D vision (in computer vision, animal navigation, and human vision) rely on building an accurate 3D model of the environment? Or can it simply rely on a partial or distorted 3D model, or no model at all?
2. Sergey Levine and Dhruv Shah argue:
Robot navigation should be based on ‘traversability’ rather than an explicit 3D map of the environment. And they explain how a robot can learn this directly through trial and error (‘experiential learning’)
3. Ida Momennejad provides:
A rubric for NeuroAI that distinguishes between:
1. Human-like behaviour
2. Neural plausibility, and
3. Engineering goals
And uses 3D navigation by AI as a key example
4. Jenny Read develops:
A new model of stereo vision for the praying mantis, explaining how an insect with much fewer neurons than humans is able to extract 3D information from stereo
5. Kate Jeffery explores:
The rotational and translational symmetry in navigation cells (place cells, head-direction cells, and grid cells), when this symmetry breaks down, and what this means for how space is processed in the brain
6. Edward Horrocks, Isabelle Mareschal, and Aman Saleem ask:
How are vision and motion integrated? Combining insights from humans and monkeys (self-motion affects the perception of optic flow) with their work on locomotion signals in mouse visual areas
7. Andrew Glennerster argues that:
Human 3D vision doesn’t rely on building a 3D map of the environment, but instead if better thought as a collection of sensorimotor contingencies that map how our actions lead to changes in the 2D retinal image
8. Paul Linton argues for:
A new approach to stereo vision, and therefore visual scale and visual shape, arguing that we’ve wasted the last 200 years trying to shoe-horn stereo vision into Kepler and Descartes triangulation-based account
9. Fulvio Domini argues for:
A rejection of probabilistic approaches to 3D vision, arguing that rather than trying to estimate the true 3D layout, human 3D vision instead prioritises maximising whatever 3D vision signal exists
10. Dhanraj Vishwanath argues that:
3D vision relies on 3 distinct encodings:
1. 3D shape
2. egocentric distance, and
3. object-to-object distance
And explores how these 3 different encodings are prioritised in different viewing conditions
11. Sarah Creem-Regehr, Jeanine Stefanucci, and Bobby Bodenheimer explore why:
Distances are underestimated in VR, and the ways in which distance perception in VR can be improved, from visuomotor experience in VR, to the properties of the headset
12. Anna Rzepka, Kieran Hussey, Margaret Maltz, Karsten Babin, Laurie Wilcox, and Jody Culham:
Observers rely more on the familiar size of objects in VR than they do in the real world, questioning whether we can equate studies in VR with the real world
13. Robert Whitwell, Mehul Garach, Mel Goodale, and Irene Sperandio:
Defend the dissociation between perception and action in the Ebbinghaus illusion, where the illusion affects visual perception but not action
(vs. the suggestion that the dissociation is merely due to differences in eye movements in the two tasks)
14. Antonella Maselli, Eyal Ofek, Brian Cohn, Ken Hinckley, and Mar Gonzalez-Franco:
Displace the virtual location of the hand in VR, and find participants are more effective at making corrections towards the body midline than away from it
15. Michael Morgan reviews:
The complexity of the stereo computations required to see an object coming fast towards you, like a snowball!
16. Ewa Niechwiej-Szwedo, Linda Colpa, and Agnes Wong review:
The importance of stereo vision for reaching and grasping in children, with the aim of better treating stereo vision deficits that affect half a billion people worldwide
17. Ione Fine and Woon Ju Park ask:
How are people born blind able to cross a busy road using sound? Remarkably, their spatial processing of sounds recruits a brain region (hMT+) typically associated with the visual perception of motion
Royal Society Meeting
This volume builds on Royal Society meeting ‘New Approaches to 3D Vision’ in November 2021
Abstracts and recordings of talks are available on the meeting website:
As well as YouTube playlist:
Finally, a big thank you to my co-editors:
Michael Morgan FRS, Jenny Read, Dhanraj Vishwanath, Sarah Creem-Regehr, and Fulvio Domini, as well as The Royal Society
