Why would a rat brain that has learnt to avoid a feeding lever delivering a distasteful salty mixture, instantly forget all it has learnt, and joyfully consume the nasty mixture when it is provided with a new very strong desire for salt?
The experiment is described here:
Instant Transformation of Learned Repulsion into Motivational “Wanting”
The Xzistor model can explain this bizarre instantaneous override of reinforced (learnt) behaviour using only Control Theory.
Here is how:
We can design an Xzistor agent with an innate aversion to Strong Salt. All that is required is a homeostatic control loop with Strong Salt as a negative feedback control variable. The Xzistor model calls this a Body UTR (Urgency To Restore) mechanism. The model also explains how artificial emotions can be created based on whether homeostasis is diverging (negative) or being restored (positive). These emotions will be created by unique representations placed in the body map of the agent, as if they are part of the body sensory states, but representing the extent to which homeostasis is diverging or being restored rather than originating from body sensors. They will therefore be ‘felt’ as if coming from inside the body.
Aided by a reflex that makes the Xzistor agent retreat (repulse) from the Strong Salt source, it will quickly learn through reinforcement learning that it should avoid the Salt Source and the Strong Salt emotions will come to represent avoid or approach states.
The Xzistor model decrees that every Body UTR like this Strong Salt avoidance control loop, will also always activate the autonomic Body UTR (basically modeling the autonomic nervous system) so that it acts in concert with this Strong Salt control loop i.e. if the Strong Salt homeostasis is disturbed (salt is ingested), the autonomic Body UTR will also rise, and as the Strong Salt ingestion is halted and homeostasis is restored, the autonomic Body UTR will fall (calmed). This autonomic Body UTR generates its own unique ‘felt’ emotions modelled as a ‘stress emotions’ and ‘nausea emotions’ akin to how it happens in the human body (we feel stress as ‘butterflies in the belly’ and nausea as the urge to ‘vomit from the stomach’).
These autonomic Body UTR responses (or emotions) might be mild, but they play an important role in how the Xzistor agent will memorise this experience i.e. how the associations will be formed so that they can be used in future to decide what actions to perform. This is because this autonomic Body UTR will be re-activated when Strong Salt memories are recalled – note that the Strong Salt Body UTR or its emotions cannot be re-evoked form merely recognising or recalling related experiences, only the autonomic ‘stress’ and ‘nausea’ emotions can be regenerated. But just thinking about the Strong Salt lever will cause the Xzistor agent to feel actual stress and actual nausea again.
Let’s go back and look what happened when the Xzistor agent experienced the Strong Salt lever for the first time. It immediately felt the unique negative emotions (aversive) and learnt to avoid it – and of course feeling better as it moved away and learning that it is better to avoid the Strong Salt lever (operant learning).
How strong was this association that formed in the mind of the agent?
Not very strong. Although the agent is programmed to dislike it, a bad taste will not create a massive aversive state with high autonomic stress and nausea (like we could get if the agent experienced severe pain, heat or fear levels). For this discussion, lets assume a 5% stress level and a 10% disgust level was stored as part of the association. The newly formed association will include the visual image of the Strong Salt lever, the taste of the salty mix and the smell of the salty mix, and the retreat actions.
Now we have an association in the Association Database (model memory) that will regenerate the stress and nausea when thinking about or observing the Strong Salt lever, or tasting or smelling salt. If the agent actually do touch the lever again and tastes the salt, this association will also be ‘recognised’ and ‘re-activated’ causing a compulsion to perform the learnt retreat actions.
Let’s now reboot the Xzistor brain!
Now we reboot the Xzistor agent brain by building in a massive Strong Salt Body UTR – also using salt as the control variable but we design his Body UTR to crave (try to maintain) very high levels of salt. Without salt, the agent goes into homeostatic deficit (the model calls this Deprivation) and feels new very strong negative emotions.
This control loop is so strong that it will override any other Body UTRs in the agent like hunger, thirst, cold, warm, pain, fatigue, etc.
We now activate this craving for Strong Salt and release the Xzistor agent into the area where the Strong Salt lever is located, and where it can easily be observed visually.
What happens now in the brain of this agent?
We know the Xzistor brain model will always first collect information on all the Body UTR loops and decide which one is the strongest (called the Prime UTR by the model). Let’s assume this Strong Salt craving we have given the agent is REALLY strong and its unique Deprivation emotion states come in at the 95% level.
The brain is now told ‘you need to taste salt as an absolute priority’. Salt! You need salt! Now!
The agent brain now needs to do a search of all associations in the Association Database to see what associations are linked to salt (this is called Threading and when the brain really needs to narrow down the search to focus on useful results, it is called ‘directed’ Threading).
So, what is Threading looking for in the Association Database?
Salt. The taste sensation (representation of course) of salt.
Will this need for salt be recognised by the Association Database?
Yes, salt has been encountered before. We have an association about encountering the Strong Salt lever before and the agent brain will make the match, and ‘recognise’ the Strong Salt lever.
As it recalls this association – what happens next?
The autonomic stress and nausea states are immediately re-generated from memory and presented to the executive part of the brain which is tasked with deciding what the agent should do next. The model collects all emotions coming from all the Body UTRs and those coming from recalled memories (called Brain UTRs) and combine all the emotional feelings (positive and negative) and, through an adjudication process, decide which is the absolute strongest Body or Brain UTR emotion that should be addressed first. The agent still feels all the other emotions, but will only act on the strongest (most urgent).
The Strong Salt craving emotions (at 95%) comes out as the Prime UTR that needs immediate action. But it will also cause a strong activation of the autonomic UTR states – say stress at 45% and nausea at 20%. This will totally override the effect on the autonomic UTR from the recalled Strong Salt lever association (at stress 5% and nausea 10%) which will have no chance to compete against the urgent needs of the new Strong Salt craving.
The only contribution this recalled association can and will make, is to link the taste of salt to the visual image of the lever – as for the rest it is to weak to trigger an avoidance action in the brain. And the best guess action for the Xzistor agent to solve its very strong Strong Salt craving would be to navigate to the lever that has been associated with salt in the past. (Navigating to the Strong Salt lever will be an approach action sequence learnt as part of reinforcement learning e.g. navigating to other levers like a sugar disposing lever to find Satiation).
And will the horrible salt mix actually taste good?
Yes, external observers without this type of craving might think it must taste horrible to the agent – but the Xzistor agent has been programmed to crave this Strong Salt taste. So as it ingests the salt, the Body UTR Deprivation will flip to Satiation and this will trigger a very strong positive emotion. This will further be boosted by strong Satiation (calming) of the autonomic stress and nausea states (even activation of the Xzistor limbic system model – where other Satiation states are momentarily activated (co-opted) to increase overall hedonic satisfaction level).
Conclusion:
We will thus see that where an Xzistor agent has learnt to avoid a salty mix feeder, if we give it a strong enough new craving for salt, it will actually use past learning about the visual image and location of a salt source, albeit reinforced as mildly discouraging (aversive), to immediately guide it to that salt source. And it will derive huge hedonic pleasure from ingesting the nasty salt mix – as this will reset its homeostatic imbalance, the basis of its emotions (feelings)!
