Modeling Exploration/Exploitation Behavior and the Effect of Individual Differences

Abstract

Searching is a ubiquitous behavior. In this thesis I focus on two very important types of search behavior: exploration and exploitation. Exploration means looking around in the environment for resources, while exploitation means stopping exploration and making use of resources already found. In the first part of the thesis, I focus on human’s exploration/exploitation strategies in a resource-collecting setting of a card search task, where individuals make a series of decisions either to explore to find a new resource or exploit a previously-discovered one, accumulating values from both as they search. The search environment changes across three experiments, by varying the resource type and the certainty of search duration. Through comparing participants’ search patterns with the optimal strategies and modeling their behavioral data, I show that the resource type could influence human’s search patterns significantly, but not the certainty of search duration. In addition, participants mainly use some linear threshold to make exploration/exploitation decisions in the card search task. In the second part of the thesis, I focus on the connection between impulsivity of individual differences and exploration/exploitation. Both search and impulsivity decisions require individuals to deal with time and rewards, and thus individual differences in impulsivity might lead to individual differences in search behavior. I investigate the influence of impulsivity on search behavior from risk seeking and lack of persistence perspectives in the card search task. Experiment 4 and 5 indicate that risk seeking mainly affects the initial exploration stage of the trial while lack of persistence largely influences the consecutive exploitation phase. This shows that the performance in a simple card search task can be related to multiple facets of impulsivity such as risk seeking and lack of persistence.