Task 5 - Values

Deciding on preferences on objects in the environment based on their value.

It codes for association between certain objects in the environment and their reward

Decision making --> preferences --> reward

3 parts: medial, central and lateral

1. Lateral PFC

Involved in state representation

2. Orbitofrontal cortex (OFC)

Involved in value representation

3. Anterior Cingulate Cortex (ACC)

Involved in outcome evaluation

Encodes the relevant states of the environment, including rules guiding goal-directed behaviour

• Neurons here accumulate sensory information and transform perceptual information into motor outputs.
• It also encodes other aspects of the environment, representing the environmental state.

→Required to make optimal actions, yielding the most desirable outcome.

→Involves task rules, relevant cues, determining action-outcome associations, and accumulation of sensory input and transforming this into motor output.

Assigns object representations of the environment to their corresponding values of the associated expected outcome.

• Represents the relationship between expected outcomes (one value being interdependent to another).

-Allows one to choose objects as a goal, and guides behaviour to obtain these ( = object selection).

→Ventral PFC might be specialized for predicting the values of rewards expected from different objects in one’s environment

• ACC might play a role in choosing appropriate actions when the environment is uncertain or dynamic.

→ACC may be involved in combining information about the costs and benefits associated with alternative actions.

-Anterior cingulate cortex may encode and update utilities associated with different actions (= action value functions).

-Medial PFC might be specialized for updating the desirability of alternative actions

• preferences

Certain actions based on potential costs or benefits and related to environmental states.

Want to improve/maximize your situation.

e.g. more food or more sleep? what is more beneficial? what will you choose based on values?

• LPFC represents utility of certain environmental states

Rushworth model: 3 steps

1. Value assignment (LOFC --> vmPFC --> ACC)

The value of the expected outcome associated with certain objects is assigned.→E.g. While in a store and purchasing fruit, you may co

nsider how much you value bananas, and how much you value pineapples.

-Lateral OFC (lOFC)

is involved in the assignment, store and updating of values based on reward and punishment.

→Represents the nature, identity and type of expected reward

associated with a choice

2. Value comparison (vmPFC/mOFC)

Comparing the values that were assigned to the different objects available in the environment

E.g. Do I value the banana more, or do I value the pineapple more?

-vmPFC and mOFC play a role here. The aforementioned processes happen in this area, based on the object-outcome information stored in the lOFC.

→Eventually a single representation dominates vmPFC/mOFC

3. Action value comparison (ACC/dmPFC)

Which response should be made to obtain the reward being chosen in the previous step.

→E.g. I decided that I value the banana more. Now what motor response should occur for me to take the banana?

-Anterior Cingulate Cortex (ACC) and dorsomedial PFC (dmPFC)

play a role here: Reflect the value of the goal that is currently being pursued, and represent values of both options that are being considered

Expected value for object selection

• Lee: LPFC = state representation, encodes environment and cues for optimal action,
• Rushworth: lOFC = updating value representations

Value evaluation for alternative actions

• Lee: OFC = value representation --> values of alternative options in making optimal choices
• Rushworth: vmPFC/mOFC --> comparing different outcomes of expected options

ACC

• Lee: outcome evaluation = assigns values and updates them
• Rushworth: OFC and ACC interact through distinct, successive steps

The reward value and pleasantness of a certain food decreases when having eaten it to satiety.

-Helps in regulating food intake

find the neural basis of responses to food odour and how these responses are regulated through sensory-specific satiety.

• Used fMRI to measure responses to the odour of banana and vanilla. Participants then ate banana to satiety, and fMRI was once more conducted in response to both odours.

→Results:Average pleasantness ratings decreased for banana after it was eaten to satiety

• Activity in the orbital frontal cortex (OFC) decreased in response to banana odour after being eaten to satiety, but not to odour of vanilla (vanilla was not eaten).
• As this occurred for banana but not for vanilla, it shows that this is related to the sensory-specific satiety effect

Lee (3 way division model) says OFC is involved in object selection

OFC represents the value that one assigns to each of the objects in the environment and keeps updating them depending on personal preference OR on how much/what you ate lately

• value of object can increase/decrease relative to one another

LOFC is involved in learning and assigning values to different stimuli and updating these values

• LOFC also represents the nature, identity or type of the expected reward associated with a choice.

→Lesions in LOFC lead to deficits in assigning precise values to stimuli

• Medial OFC involved in representing the expected reward of different options, or comparing reward goals and choosing one of them to subsequently drive behaviour

• Reference dependence
• Risk aversion over gains
• risk seeking over losses
• loss aversion
• probability weighting

Basically you want to win as much as possible while losing as little as possible so you have to weigh the probabilities of choosing something over something else.

• LOFC = Values and rewards type info assigned to different options
• vmPFC/mOFC = value comparison
• ACC = Action value comparison

vmPFC damage would mean that one cannot compare and choose appropriately between different objects, based on their expected outcomes

→The person is then impaired in setting an appropriate goal and guiding behaviour accordingly.

→Such a general deficit in goal-related behaviour would therefore impact several areas of one’s life and behaviour.

• Phineas Gage → damage to mPFC →personality changed Shows that orbitofrontal cortex damage might spare most complex abilities (language, memory,motor skills, ...) but impair social behaviour.
• Shows that personality may be a complex function of several mental processes, such as goal-directed cognition

vmPFC encodes value for mulitple classes of goods:

• before scan, participants rated willingness to pay 80% probability of receiving certain stuff > snacks, trinkets, money (assign value to goods)
• Inside scanner, participants choose between expected price of all stuff or 80% chance of receiving each items

vmPFC activation = common evaluation system exists

Are value-related representations similar for different types of rewards?

• 2 categories of items: snacks and engaging activities
• in scanner showing people words related to snacks and engaging activities
• after scan, asked participant what they prefered
• significant correlation
• activities are also related to value

Participants received 50 euros, choose between keeping 20 and the lottery (lose all in 66% of cases, 33% win all)

• one trial says 'keep 20', other says 'lose 30' = the same

Framing effect: when showing gain framing, gamble less. When showing losing frame, gamble more.

• Is in line with prospect theory

Right amygdala reacts to sure gain and gamble loss same way, gamble gain and sure outcome loss also in same way.

• gains and losses not treated equivalently in brain

Inconsisent choices: ACC, conflict between emotional and analytical systems

• OMPFC: rationality, react less to framing, evaluates and integrates emotional and cognitive information, underpinning more rational behaviour

1. vmPFC(value)
2. OFC(value)
3. DLPFC
4. insula
5. M1 (choice)
6. posterior parietal cortex (sensory)

Involved structures: dmPFC, posterior cingulate cortex/precuneus and angular gyrus

• Known for being active when a person is not focused on the outside world and the brain is at wakeful rest (daydreaming, mind-wandering)
• Also activates when one thinks about self or others, remembers the past, and plans for the future.
• Deactivates more when a task becomes more difficult

Ventromedial prefrontal cortex is the affective node

• Core region involved in goal-oriented decision making; involved in self-referential information processing.

Posterior cingulate cortex is the memory-related node

The default mode network is involved in self-referencing, and the “value for the self”.

→Important for knowing personal preferences, which may determine the utility of an object

The affective node (vmPFC) is de-activated during task execution.

→The mPFC deactivates less during self referential judgements, and thinking about others results in a strong deactivation of the mPFC.

-AsmPFC is active during everyday activities, and is concerned with self referential judgements

• When one must perform tasks that do not require the self, the mPFC deactivates, and activity shifts toward more relevant/necessary brain areas for the task at hand

Lee article:

• OFC lesions in monkeys lead to impaired ability to adjust decision-making strategies and impaired ability to assign appropriate values to objects in environments.

→Relates to Rushworth’s lateral OFC, stated to be involved in value assignment.

Rushworth’s article:

-Makes a division between selection of objects and selection of actions.

→Lesions of lateral OFC or vmPFC/mOFC disrupt stimulus-reward but not action-reward associations.

→ACC lesions disrupt action-reward but not stimulus-reward associations.

• ACC is involved in action-reward and lOFC or vmPFC/mOFC is involved in stimulus-reward.