When we are thirsty, drinking water should be rewarding. That’s how we experience it as humans. And after all, we are restoring the bodily homeostasis function that regulates hydration.
The Mouse Mystery
Figure below: Scientists at Caltech conducted a study to see why animals find drinking water rewarding. They recorded large spikes of dopamine release when thirsty mice drank both water and a salty saline solution. This proved that mice found both of these liquids rewarding. But strangely, when these scientists injected water directly into the digestive tracks of thirsty mice, they found no changes in dopamine levels, even though the injected water would also have hydrated the thirsty animals.
figures by Jovana Andrejevic
This phenomenon might seem contradictory at first but is elegantly explained by a concept defined by the Xzistor Concept brain model called a ‘Satiation Event’.
Let’s briefly look at what is meant by a Satiation Event.
As the artificial agent (robot) becomes thirstier over time, meaning the simulated water level in the blood is effectively dropping, the agent will enter the Deprivation phase of the Thirst UTR curve. When the agent tries to drink something (again simulated), it will be a strong indicator that the fluid is a ‘legitimate’ hydration source if the water level in the blood suddenly increases (or a derivative marker shows a similar trend) and the UTR value suddenly starts to drop. This sudden drop in the UTR curve is indicated by the Satiation phase of the UTR curve in the figure below. This will signify that a Satiation Event (apex of the curve) had taken place.
It is crucial to the agent’s survival in its environment to learn from the Satiation Event that had taken place when the fluid (water) was consumed for the first time. The model will prominently flag this Satiation Event the moment it takes place and save all available information around it for future use: What did the agent do? What did the agent do just before that? What did the water source look like? How did it feel to the touch? Where was it found? How was it retrieved and manually handled? This will be important information for the agent’s survival in future and stored as a set of associations as part of learning. The Xzistor agent will learn based on the reward state generated by consuming the fluid, allowing it to navigate back to this fluid source in future when thirsty.
The Mouse Mystery explained.
The Satiation Event explains how any reward provided to the brain too long after the correct drinking behavior had been performed (e.g. 10 minutes later) will reinforce the wrong behaviors – not the moving to the faucet and sipping/swallowing sequence. Therefore the biological brain has no choice but to reward the ‘swallowing’ behavior (assuming it will lead to hydration) and deliberately assign no reward to the actual restoration of hydration into the blood stream that is expected to happen a few minutes later. The biological brain’s effective UTR curve will still broadly resemble the Xzistor Concept curve, except that the Satiation Event is moved to an earlier (pre-emptive) point and triggered by a ‘swallow sequence’ rather then a ‘lowering of osmolality’. Some mammals will closely resemble the Xzistor UTR curve while other mammals like mice, humans, etc. will require the Satiation Event to be moved earlier and triggered by the ‘swallow sequence’ to ensure learning and ‘valuing’ of this behavior as important to survival.
No more Mouse Mystery!
Another interesting insight provided by the Xzistor Concept brain model.
