Day 3: How do neurons detect and assemble edges? — Understanding the Visual Cortex

Moving deeper into the book "Principles of Neural Science" — starting with Chapter 23 on intermediate-level visual processing
Chapter distinguishes low-level vs intermediate visual processing: computation of local orientation is low-level, contour integration is intermediate-level
Intermediate-level visual processing is concerned about which boundaries & surfaces belong to a specific object and its background
Think of it like the internal layers of a CNN — edges have been determined, now only the contours need to be detected
"Intermediate-level visual processing thus involves assembling local elements of an image into unified percept of objects & backgrounds"
Although determining these things is highly complex — combining different parts of the image has an astronomical number of solutions. Each relay in the visual circuitry of the brain basically has some form of "perception" while looking at an object, and hence we see edges/boundaries when there are none. There's slight ambiguity, and three signals help resolve it:

1) Visual cortex is context-dependent: what we perceive is based on where we are seeing it — hence we don't like "weird" things at weird locations
2) The functional properties of neurons in the visual cortex can be altered by visual experience
3) Visual processing is subject to cognitive functions such as attention, expectation, etc.

Hubel & Wiesel discovered two types of cells in V1: simple and complex (which do the abstractions).

Edge-detecting neurons fire only when an edge is at a certain angle (say 45°) — similar to kernels in CNNs. Then deeper layers handle contours
Simple cells are ON & OFF cells: when a light bar enters the receptive field (what neurons see in their local path), they go either ON or OFF
Complex cells are not so rigid — they are always active and don't have an ON/OFF switch, which is why the book says complex cells might be the second step, i.e. a combination of simple cells

Where edge detection happens in the visual pathway is evolution-based — different species arrange cells differently.

In mice, edges are detected even before the signal goes to cortex (at an extremely early stage)
In cats, edge detection is moved up — edges are processed at the intermediate stage with simple cells, then go into complex cells
In primates/humans, it moved much further up — we skip simple cells altogether and just process things with complex cells
This is an evolutionary advantage: as edge detection happens nearer to the cortex (inside the cortex), we get much more flexibility because the cortex is plastic & connected to millions of things — hence we can "understand & relate" what we see