EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Intrinsic rewards explain context-sensitive valuation in reinforcement learning

Gaia Molinaro iD, Anne G E Collins iD

PLOS Biology · 17.07.2023 · 21 (7) · e3002201

DOI: 10.1371/journal.pbio.3002201 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Biology, online erschienen am 17.07.2023. Verfasst von Gaia Molinaro und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America. Hauptthemen laut MEDLINE: Reinforcement, Psychology, Reward. Die Arbeit wurde bisher 30-mal zitiert (Crossref) und nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: University of California Berkeley, Foundation for the National Institutes of Health, National Science Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

When observing the outcome of a choice, people are sensitive to the choice’s context, such that the experienced value of an option depends on the alternatives: getting $1 when the possibilities were 0 or 1 feels much better than when the possibilities were 1 or 10. Context-sensitive valuation has been documented within reinforcement learning (RL) tasks, in which values are learned from experience through trial and error. Range adaptation, wherein options are rescaled according to the range of values yielded by available options, has been proposed to account for this phenomenon. However, we propose that other mechanisms—reflecting a different theoretical viewpoint—may also explain this phenomenon. Specifically, we theorize that internally defined goals play a crucial role in shaping the subjective value attributed to any given option. Motivated by this theory, we develop a new “intrinsically enhanced” RL model, which combines extrinsically provided rewards with internally generated signals of goal achievement as a teaching signal. Across 7 different studies (including previously published data sets as well as a novel, preregistered experiment with replication and control studies), we show that the intrinsically enhanced model can explain context-sensitive valuation as well as, or better than, range adaptation. Our findings indicate a more prominent role of intrinsic, goal-dependent rewards than previously recognized within formal models of human RL. By integrating internally generated signals of reward, standard RL theories should better account for human behavior, including context-sensitive valuation and beyond.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Reinforcement, PsychologyRewardHumansLearningMotivation

Autor:innen und Institutionen

  • Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America.
  • Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America.

ORCID-Profile: Gaia Molinaro, Anne G E Collins

Förderung

  • University of California Berkeley
  • Foundation for the National Institutes of Health
  • National Science Foundation

Zitieren

APA

Molinaro, G., & Collins, A. G. E. (2023). Intrinsic rewards explain context-sensitive valuation in reinforcement learning. PLOS Biology, 21(7), e3002201. https://doi.org/10.1371/journal.pbio.3002201

Vancouver

Molinaro G, Collins AGE. Intrinsic rewards explain context-sensitive valuation in reinforcement learning. PLOS Biology. 2023;21(7):e3002201. doi:10.1371/journal.pbio.3002201

BibTeX
@article{Molinaro2023,
  author = {Molinaro, Gaia and Collins, Anne G E},
  title = {Intrinsic rewards explain context-sensitive valuation in reinforcement learning},
  journal = {PLOS Biology},
  year = {2023},
  volume = {21},
  number = {7},
  pages = {e3002201},
  doi = {10.1371/journal.pbio.3002201}
}
Kostenlos starten →