Learning in noise: Dynamic decision-making in a variable environment
Abstract
In engineering systems, noise is a curse, obscuring important signals and increasing the uncertainty associated with measurement. However, the negative effects of noise are not universal. In this paper, we examine how people learn sequential control strategies given different sources and amounts of feedback variability. In particular, we consider people's behavior in a task where short- and long-term rewards are placed in conflict (i.e., the best option in the short-term is worst in the long-term). Consistent with a model based on reinforcement learning principles (Gureckis & Love, 2009), we find that learners differentially weight information predictive of the current task state. In particular, when cues that signal state are noisy, we find that participants' ability to identify an optimal strategy is strongly impaired relative to equivalent amounts of variability that obscure the rewards/valuations of those states. In other situations, we find that noise and variability in reward signals may paradoxically <i>improve</i> performance by encouraging exploration. Our results demonstrate how experimentally-manipulated task variability can be used to test predictions about the mechanisms that learners engage in dynamic decision making tasks.
Highlighted Figures
Keywords
Bibtex entry:
@article{gureckis2009learning,
abstract = {In engineering systems, noise is a curse, obscuring important signals and increasing the uncertainty associated with measurement. However, the negative effects of noise are not universal. In this paper, we examine how people learn sequential control strategies given different sources and amounts of feedback variability. In particular, we consider people's behavior in a task where short- and long-term rewards are placed in conflict (i.e., the best option in the short-term is worst in the long-term). Consistent with a model based on reinforcement learning principles (Gureckis & Love, 2009), we find that learners differentially weight information predictive of the current task state. In particular, when cues that signal state are noisy, we find that participants' ability to identify an optimal strategy is strongly impaired relative to equivalent amounts of variability that obscure the rewards/valuations of those states. In other situations, we find that noise and variability in reward signals may paradoxically <i>improve</i> performance by encouraging exploration. Our results demonstrate how experimentally-manipulated task variability can be used to test predictions about the mechanisms that learners engage in dynamic decision making tasks.},
author = {Gureckis, T.M. and Love, B.C.},
journal = {Journal of Mathematical Psychology},
number = {3},
pages = {180--193},
publisher = {Academic Press},
title = {Learning in noise: Dynamic decision-making in a variable environment},
volume = {53},
year = {2009}}QR Code:
Download SVG