Expanation of the Predictive Coding Animation

Predictive coding is a theory of cognition in which the brain is constantly generating and updating a mental model of sensory input. The model is broadcast through the network of sensory processing brain regions. In each region, the model being propagated is compared to the sensory input and if they do not match, a Prediction Error is sent back up the network and the model is revised.

This animation is a work of science fiction. It is portraying a dramatically simplified model of what is happening in the brain - this is plausible. Whether it is more correct or less, something like this is certainly happening in the brain.

This animation picks a few regions out of a massive network connecting 100+ brain regions and shows how predictive coding might work. These regions are: Cortical Functional Areas (8/100+) - Visual Cortex V1 - Primary Visual Cortex V2 - Visual Area 2 MT - Motion Processing IT - Object Processing V4 - Color Processing Cortical Functional Areas (8/100+) - Auditory Cortex STS - Superior Temporal Sulcus BA41 - Auditory Cortex BA42 - Brodmann Area 42 Thalamic Nuclei (2/13) LGN - Thalamic Relay MGN - Thalamic Relay   Script: 0 = "Driving down the road" 1 = "Listening to some tunes" 2 = "Towards a green light" 3 = "The Light turns yellow" 4 = "The song changes" 5 = "The Light turns red" 6 = "You notice Rte 20 sign" 7 = "The Light turns green" 8 = "A Red Ferrari ZOOMs by"

In this little model, the only place to see any negotiation is in V1 and V2. And this is oversimplified, too. The regions bounce back and forth between the model coming down and the sense data coming up. I'm pretty sure that the regions negotiate locally and change incrementally.

The model flows down from the hierarchy at a high ratio of down information to up information. This model shows 4 paths down for every path up. That is probably an under estimate.

To repeat: The model is in the Global Workspace and is broadcast throughout the nervous system. These signals serve as predictions for the sensory input. When the prediction does not match the sensory input, a prediction error signal is sent back thru the network to update the model. The model and the sensory input match, the network is relatively quiescent. Sending only the prediction errors, instead of the full sensory input, saves a massive amount of energy.

I know almost nothing about the Audio Cortex, except that A1 includes BA41 and BA42. STS is just a context free grab of a region that seems to be more active when human voices are heard and it has some interconnections to the Visual Cortex. https://en.wikipedia.org/wiki/Brodmann_areas_41_and_42

The V1 images are little bitmaps of pixels that are meant to represent the first stage of visual processing. V2 images are basically random lines and circles - enough to show the contents of the region are changing. There is a lot of processing that goes on in V1 & V2. One thing is to recognize lines.

Predictive Coding, specifically, is the coding of the prediction and prediction error signals. See, image from XX

The functions of these regions is proposed because they consume slightly more oxygen when a task involves that function.

Even as the prediction errors update the model, if no salient events occur, we can continue on autopilot without having to engage consciousness.

The prediction error are the red circles traveling upwards. In this animation they are represented by 6 pulses sent from the LGN or MGN

When we see a red Ferrari zooming past, the information is reassembled from a map of the colors, a map for motion processing and a map for object identification. These are Retinotopic Maps. Christof Koch uses the red Ferrari zooming past to illustrate the Binding problem in his Glossary of Terms. Related Pages: Script for Animation, Explanation, Zoomable, Theory <> Gentle Intro to Predictive Coding <> More GEEK Art <> More Notes