Background & Aims
Endogenous pain modulation is thought to encompass a crucial evolutionary purpose in guiding behavior away from harm [4]. This is best exemplified when looking at pain through the lenses of reinforcement learning (RL) theory [7]. Namely, RL has proven successful in explaining behavior in tasks assessing pain modulation, with the pain modulation being linked to RL prediction errors [3]. However, adaptive behavior requires more than learning through error correction. Biological environments encompass different forms of variability that result in different forms of uncertainty that need to be accounted for by decision-makers [2]. While learning through error correction has been linked to pain modulation, no prior study has tried to concomitantly assess the effect of uncertainty processing mechanisms on pain modulation. The present study aimed to investigate how both mechanisms of learning and uncertainty processing are linked to endogenous pain modulation from rewards and punishments.
Methods
We used a modified version of the Wheel of Fortune task (c.f.[1,3,5]) where players actively gamble to obtain pain relief as a reward, and the avoidance of pain increase as a punishment. Those active gambles are contrasted with control trials, where participants are passively subjected to random changes in nociceptive input. The task was given a probabilistic-reversal outcome-schedule. Such schedules are traditionally used to assess learning and uncertainty in decision-making. In total, 30 healthy human volunteers completed the task. During the task, participants received thermal stimulation on the volar forearm. In the first step of the data analysis procedure, we assessed the effect of objective task properties on pain modulation with a linear mixed effects approach. In the second step, computational models were fitted to the observed choice-outcome distribution. Both model-free RL models, as well as two inferential Bayesian learners, were considered in the procedure.
Results
The number of trials since the last contingency reversal significantly increased pain inhibition from pain relief as a reward, indicating that the longer participants spent within a stable reward contingency, the stronger the pain inhibition was. The model that provided the best fit for the data was a Bayesian inferential Hidden Markov Model [6]. The model assumed that, as decision-agents, participants build, update, and utilize an inferential belief about the statistical properties of the environment. The model also assumed this updating process to be different whether the agents are subjected to pain as punishment or reward in a trial. From this model, we extracted trial-by-trial values of uncertainty (belief entropy), which showed a significant negative correlation with the extent of pain inhibition. Similarly, we extracted trial-by-trial prediction errors from the best fitting RL model, but found no significant correlation between prediction errors and pain modulation.
Conclusions
Present results indicated that behavior under acute experimental pain was best explained by a Bayesian inferential Hidden Markov Model rather than simpler, model-free RL models. This indicates that processes of representational inference, i.e. the ability to build and utilize an internal representational model of the statistical properties of the environment, may be of interest to understanding adaptive behavior in acute pain situations. Contradicting previous results [3], we found no significant association between RL prediction errors and the extent of pain modulation from rewards and punishments. However, objective and model-based predictors of uncertainty showed a significant association with the pain modulation. Pain inhibition was stronger when belief certainty was higher. This potentially indicates that certainty in the statistical properties of the rewarding environment is a necessary component for pain relief, as a reward, to induce pain inhibition in situations of acute pain.
References
[1] Becker S, Gandhi W, Elfassy NM, Schweinhardt P. The role of dopamine in the perceptual modulation of nociceptive stimuli by monetary wins or losses. Eur J Neurosci 2013;38:3080–3088.
[2] Dayan P, Yu A. Uncertainty and learning. IETE J Res 2003;49:171–181.
[3] Desch S, Schweinhardt P, Seymour B, Flor H, Becker S. Evidence for dopaminergic involvement in endogenous modulation of pain relief. eLife 2023;12:e81436.
[4] Fields HL. A motivation-decision model of pain: the role of opioids. Proceedings of the 11th world congress on pain. IASP press Seattle, 2006. pp. 449–459.
[5] Florin E, Koschmieder KC, Schnitzler A, Becker S. Recovery of Impaired Endogenous Pain Modulation by Dopaminergic Medication in Parkinson’s Disease. Mov Disord 2020;35:2338–2343.
[6] Schlagenhauf F, Huys QJM, Deserno L, Rapp MA, Beck A, Heinze H-J, Dolan R, Heinz A. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. NeuroImage 2014;89:171–180.
[7] Seymour B. Pain: A Precision Signal for Reinforcement Learning and Control. Neuron 2019;101:1029–1041.
Presenting Author
Fabrice Hubschmid
Poster Authors
Fabrice Hubschmid
B. Sc., M. Sc.
Heinrich-Heine University Düsseldorf
Lead Author
Simon Desch
Department of Experimental Psychology, Heinrich Heine University Düsseldorf, Germany
Lead Author
Esther Florin Dr.
Heinrich-Heine University Düsseldorf
Lead Author
Susanne Becker
Heinrich Heine University Düsseldorf, Institute of Experimental Psychology
Lead Author
Topics
- Other