Skip to main content

Inferring relevance in a changing world

Author(s): Wilson, Robert C.; Niv, Yael

To refer to this page use:
Abstract: Reinforcement learning models of human and animal learning usually concentrate on how we learn the relationship between different stimuli or actions and rewards. However, in real-world situations “stimuli” are ill-defined. On the one hand, our immediate environment is extremely multidimensional. On the other hand, in every decision making scenario only a few aspects of the environment are relevant for obtaining reward, while most are irrelevant. Thus a key question is how do we learn these relevant dimensions, that is, how do we learn what to learn about? We investigated this process of “representation learning” experimentally, using a task in which one stimulus dimension was relevant for determining reward at each point in time. As in real life situations, in our task the relevant dimension can change without warning, adding ever-present uncertainty engendered by a constantly changing environment. We show that human performance on this task is better described by a suboptimal strategy based on selective attention and serial-hypothesis-testing rather than a normative strategy based on probabilistic inference. From this, we conjecture that the problem of inferring relevance in general scenarios is too computationally demanding for the brain to solve optimally. As a result the brain utilizes approximations, employing these even in simplified scenarios in which optimal representation learning is tractable, such as the one in our experiment.
Publication Date: 24-Jan-2012
Electronic Publication Date: 24-Jan-2012
Citation: Wilson, Robert C, Niv, Yael. (2012). Inferring Relevance in a Changing World. Frontiers in Human Neuroscience, 5 (10.3389/fnhum.2011.00189)
DOI: doi:10.3389/fnhum.2011.00189
EISSN: 1662-5161
Type of Material: Journal Article
Journal/Proceeding Title: Frontiers in Human Neuroscience
Version: Final published version. This is an open access article.

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.