Written by Franz-Xaver Neubert
The study of the human brain has always been largely influenced by the scientific tools we have at hand. However, it is no less influenced by the ideas, terminology and metaphors that are in place for describing the problems and findings in research; in some ways, these might be viewed as tools in their own right.
Famously, Galen used the metaphor of a Roman fountain when talking about the brain. Descartes thought that many processes that we take to be mental are actually physical and can be ascribed to a complex hydraulic system of fluids flowing through the nerves and the nervous system. Jacques de Vaucanson’s mechanical duck inspired philosophers to envisage human behaviour as being driven by mechanical processes. Thomas Hobbes also proposed that ideas and associations resulted from minute mechanical motions in the head. Galvani and Volta’s pioneering research on electricity in the 1790s made scientists think of electricity as a driving force for mental processes, and the first microscopic images of neurons and their axons fuelled colourful allusions to the telegraph and telephone.
Today, we often speak about mental processes using “computer” metaphors. We speak of “processing information in different modules” or latent variables “computed” by the brain. Some scientists think of mental processes as the application of logical or mathematical rules to symbols. The influence of technology on how we think about mental processes might not be a one-way street. It is likely that neuroscientific discoveries have provided inspiration for engineers. They inform computer science and robotics of the solution of problems that humans are particularly good at solving (e.g. spotting a face in the crowd). They might also shape how we think about the workings of, for example, machines (think of the “memory” metaphor with respect to computers’ RAM and ROM).
The way that neuroscience, until recently, thought about decision-making was still deeply rooted in rather simple “mechanical” concepts of brain function. Neurophysiologists thought about decision processes, in the tradition of Charles Scott Sherrington, as largely reflexive – a very complex but essentially predetermined and “mechanical” response to a stimulus. However, inspiration from economics has brought about a fundamental turn in decision neuroscience.
The way that philosophers and economists talk about decisions differs greatly from that of neurophysiologists4. In microeconomics a choosing agent is thought to have an internal representation of values and preferences that are revealed by their choice behaviour. The idea of expected utility has extended this to include representations of outcome probabilities: stating that choosing agents should always prefer actions that maximise expected utility, i.e. the “best” combination of subjective values and probabilities in a set of lotteries. This is utility theory.
Although it is apparent that economics proposes “representations” of outcome probabilities and subjective values, economists were largely agnostic about how this might actually be implemented in (human) brains. Recently a “neuroeconomic” turn started to combine neurophysiological models of “representation” and “processing” with economical ideas about choice, preference, subjective value, risk, ambiguity, delay discounting and others. This reconciled a multitude of previous theories and findings in neurobiology and inspired a large set of exciting new questions.
One of the first major discoveries emerged from studies on the representation of value in the brain. In 1997 Schultz, Montague, and Dayan published a collection of experiments demonstrating that the firing of dopamine neurons in the primate midbrain signals “reward prediction errors” – the discrepancy between expected and actually received return11. This was taken as evidence in support of reinforcement learning theories and suggested that the brain’s internal motivational representations might yield to economic analysis.
At about the same time, Knutson and colleagues showed the first signs of robust haemodynamic activation of the ventral striatum in human subjects incentivised with money, a non-primary reinforcer with no intrinsic relevance for survival8. This suggested that the brain evaluated and encoded both primary and secondary rewards in similar ways, perhaps even in a “common currency”.
In another early set of studies, Platt and Glimcher recorded neuronal activity from the lateral intraparietal area (LIP), a cortical area involved in connecting visual stimulation to attention and orienting movements of the eyes10. In their experiments, monkeys choose freely between two targets associated with a fluid reward. The two options differed in size, or probability that the reward would be received, across blocks of trials. Studying an agent’s free choices in this way was a novel approach as previous neurophysiological experiments – largely influenced by “Sherringtonian” views of the nervous system, i.e. linking stimuli with responses – had mainly studied experimenter-determined behaviours in response to physical cues. The authors reported that the firing rates of LIP neurons representing a specific eye movement scaled with the product of reward size and reward probability, i.e. the ‘‘expected value’’ of this eye movement. Consistent with this notion, firing rates predicted monkeys’ choices: higher firing rates were observed with higher reward probability. This work drew attention to the idea of utility theory being a useful tool for studying the processes by which the brain organises action.
Crucially, all of these early studies suggested that economic variables such as expected reward, subjective value or utility were in fact being represented in the (human) brain, or at least affected how information about stimuli and responses were encoded. If the brain were just a simple stimulus-response mapping device, as suggested by earlier neuroscientists and biologists, then the encoding of a specific visual stimulus or a certain eye movement should not be affected by economic variables such as subjective value or probability. However, Platt and Glimcher showed that the representation of eye movements or their respective targets in LIP did exactly that: their neural responses varied largely with their utility and the monkey’s preferences.
These early discoveries inspired decision- neurophysiology and a new field called “neuroeconomics” emerged5. Soon researchers were trying to find neural correlates for other economic variables and phenomena – such as risk, ambiguity, delayed reward and counterfactual values. Naturally, they also cared about the exact “mechanistic” implementation of these processes – and this might in turn prove to be of interest for behavioural economics.
How is this implemented in the brain?
With this curiosity for “neural implementation”, neuroeconomists started to wonder how, for example, the human brain carries out value comparison. In the case of eye movements, they put forward sophisticated models of evidence accumulation in LIP neurons, together with dynamically varying degrees of interaction between topographically organised representations of the different options and adjustable thresholds for determining choice. It was also becoming apparent that signatures for such value comparison processes could be found in several regions – among them not only LIP but also the frontal eye field and the superior colliculus2, 6. This was taken by some to suggest that value comparison might not be achieved by a single brain region but might rather be thought of as a network process.
The description of concrete, “mechanistic” ways of how value comparison is implemented in the brain might explain some puzzling micro-economic phenomena. One example is the “distractor effect” on value comparison, which is thought to be a failure of optimal decision-making arising from the presence of irrelevant and non-choosable “distractor options”. Surprisingly, choosing between two options in the presence of a third, very poor alternative is more difficult than in the presence of a very good alternative. Chau and colleagues1 linked this phenomenon with brain imaging data to suggest that value difference signals in the ventromedial prefrontal cortex (vmPFC) reflected this phenomenon. Furthermore, they used a biophysical model initially proposed by Wang and colleagues12 to explain how such a paradoxical effect might emerge as a result of how value comparison is neurally implemented in the brain. They argued that value comparison is carried out by different pools of neurons, which represent the value of their respective option through self-excitation, and a mutual inhibition of the other neural pools via excitatory connections to an additional inhibitory neural pool. A low-value option would thus not only yield very low firing in the pool that represents its value, but also decrease the overall inhibitory activity in the system, which would then render comparison of the two other options more difficult.
Such studies may prove relevant for behavioural economics in turn. Studying the neural implementation of valuation and choice in the brain may allow for a mechanistic explanation of these deviations from “optimal” economic behaviour. Future research will help explain, for example, the problem of “internal reference points”. It is widely known in psychology that humans often compare offers to internal reference points and choices are largely influenced by these offsets. This behaviour is not easily reconciled with economic predictions of rational decision-making but might be explained by theories of efficient value encoding in the cortex. In short, such theories hold that human brains – with their limitations in energy and time resources – can only master the infinite problem space of valuation in a complex and changeable world by using short-cuts and simplifications and deducing structural regularities.
Another puzzle that may be solved by taking a closer look at how choices are made in the brain is the “curse of choice” phenomenon. Humans are known to be less decisive and unhappier with their eventual choices when offered a bigger range of options. Understanding how value representation and comparison for multi-option decisions is implemented in the neural tissue might lead us to understand this paradoxical effect.
More complex decisions
In the early years, neuroeconomics focused on studying these relatively simple decisions. The fact that even these simple choices were not “reflex-like” but relied on the representation of “economic variables” was encouraging. However, they hardly lay at the core of human decision-making. As such, more recently neuroeconomists have started to apply their approach to study more elaborate decision processes. Some studies now focus on real-life decisions with multiple alternatives and changing contexts. Others have started to investigate decisions in competitive interactive settings or “games”.
During mixed-strategy interactive games (such as work-or-shirk or rock-paper-scissors) humans tend to exhibit “unpredictable” behaviour in equilibrium state. For example when playing many rounds of rock-paper-scissors they choose a stable and predictable “mix” of rock, paper and scissors (e.g. 33% of trials rock, 33% paper, 33% scissors) in equilibrium state, but their specific next choice is still completely unpredictable (otherwise their opponents would adapt). These “random” choices are nevertheless often self-reported as deliberate. Similar behaviour is observed in monkeys. This “deliberately unpredictable” behaviour exhibited by humans and monkeys has been linked with the small random fluctuations in the firing rates of neurons suggesting that these constitute the neural implementation of “rationally unpredictable behaviour” in competitive games hypothesised by decision theorists3.
Another example of how neuroeconomics, or the neural implementation of economical behaviour, might help to explain behavioural phenomena that may at first surprise axiomatic decision theory – the normative account of how rational choices ought to be made – comes from the studies of value representation. Humans (and other primates) are much more impulsive than would be deemed “rational” by economists when it comes to choosing between immediate rewards and delayed gratification. Perhaps even more surprisingly they largely differ in their “impulsiveness”. Kable and Glimcher7 showed that individual behavioural patterns of temporal discounting, i.e. trading higher rewards received later against lower rewards received sooner, was reflected in patterns of neural value coding in a region called the medial prefrontal cortex (mPFC). One subject (a MD/PhD student) had a very flat discounting function (they were very ready to accept bigger delays for rewards of similar size) and the BOLD response in their mPFC decreased only very slowly with increasing delays.
Another subject (a professional skydiver) steeply discounted temporally delayed rewards, and this behavioural pattern was also reflected in the way their mPFC encoded rewards, in that activity in this region decreased sharply with increasing delays. These observations suggest that these preference profiles relate to more stable personality traits, which in turn might lead to certain career and life decisions, although this is probably a mutual interaction. One thing is clear: stable personality traits and life histories are intricately tied to individual brain structure and function, as well as -on a behavioural level- the choices we make. Studying the implementation of economical processes in the brain will thus help us to understand choice behaviour even at the level of the individual subject. It is likely to illuminate some highly irrational aspects of inter-temporal decision-making. For example humans discount delayed rewards very steeply for short delays but less steeply for very distant time points. This may relate to how outcomes are evaluated within different time horizons. Moreover, humans often reconsider or overturn their own choices later in time. A better understanding of how value-comparison systems interact with executive control systems in the brain could solve this puzzle.
Despite being simplistic and somewhat outdated, the Cartesian and Sherringtonian view of choice had one crucial advantage: it was very precise and mechanistic in how decision processes were implemented in the nervous system (as predictable stimulus-response connections). It therefore rendered choice a predictable process carried out by the brain. The neuroeconomic turn allowed the reconciliation of neurophysiological and economic views of choice: it took the mathematical frameworks of economics to obtain a similarly precise and mechanistic account of how economic decisions were implemented in the brain. Combining these distinct fields of study and merging the two points of view, neuroeconomics promises to bring forward our understanding of human decision making, by studying its neural implementation and thus explaining behavioural deviations that axiomatic decision theory has thus far lacked the fine-grained capacity to elucidate.
1 B. K. Chau, N. Kolling, L. T. Hunt, M. E. Walton, and M. F. Rushworth, ‘A Neural Mechanism Underlying Failure of Optimal Choice with Multiple Alternatives’, Nat Neurosci, 17 (2014), 463-70.
2 Long Ding, and Joshua I Gold, ‘The Basal Ganglia’s Contributions to Perceptual Decision Making’, Neuron, 79 (2013), 640-49.
3 Michael C Dorris, and Paul W Glimcher, ‘Activity in Posterior Parietal Cortex Is Correlated with the Relative Subjective Desirability of Action’, Neuron, 44 (2004), 365-78.
4 Paul W. Glimcher, Decisions, Uncertainty, and the Brain : The Science of Neuroeconomics (Cambridge, Mass. ; London: MIT Press, 2003), pp. xx, 375 p.
5 Paul W. Glimcher, Neuroeconomics : Decision Making and the Brain (Amsterdam ; London: Elsevier Academic Press, 2009), pp. xviii, 538 p.
6 Timothy D Hanks, Charles D Kopec, Bingni W Brunton, Chunyu A Duan, Jeffrey C Erlich, and Carlos D Brody, ‘Distinct Relationships of Parietal and Prefrontal Cortices to Evidence Accumulation’, Nature, 520 (2015), 220-23.
7 J. W. Kable, and P. W. Glimcher, ‘The Neural Correlates of Subjective Value During Intertemporal Choice’, Nat Neurosci, 10 (2007), 1625-33.
8 Brian Knutson, Charles M Adams, Grace W Fong, and Daniel Hommer, ‘Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens’, J Neurosci, 21 (2001), RC159.
9 I. Levy, S. C. Lazzaro, R. B. Rutledge, and P. W. Glimcher, ‘Choice from Non-Choice: Predicting Consumer Preferences from Blood Oxygenation Level-Dependent Signals Obtained During Passive Viewing’, J Neurosci, 31 (2011), 118-25.
10 M. L. Platt, and P. W. Glimcher, ‘Neural Correlates of Decision Variables in Parietal Cortex’, Nature, 400 (1999), 233-8.
11 W. Schultz, P. Dayan, and P. R. Montague, ‘A Neural Substrate of Prediction and Reward’, Science, 275 (1997), 1593-9.
12 Xiao-Jing Wang, ‘Neural Dynamics and Circuit Mechanisms of Decision-Making’, Current opinion in neurobiology, 22 (2012), 1039-46.