March 14, 2008

Artifacts of Decision Making in the Brain

Filed under: ML, Neuroscience — krsrk @ 8:56 am

We take decisions all the time from the most simple(food) to the most complex(invest, career, friends).

Ever wondered

  • Where in the brain does this all happen?
  • How do we make these decisions?
  • How good are we at making these decisions?
  • and many more…

One of the decision making theories is based on learning to decide better simply based on rewards and punishments. It has been extensively studied in machine learning, statistics, control communities under the guise of Reinforcement Learning. Temporal Difference learning is one of the simplest models of reinforcement learning used to explain learning from rewards. The learning in the model is based on difference in the expected future reward and actual reward, Temporal Prediction Error.

Decision making behavior of animals was extensively studied and models of their behavior were proposed. Conditioned response to an object in the presence/absence of reward was one of the widely studied behaviors. When animals were repeatedly presented by the object followed by reward, animals started expecting reward on presenting the object.

Models of animal behavior were developed conditioned on just the object and actions taken by the animal. Rescorla-Wagner(RW) model was one of the models used to explain the conditioned reward animal behavior. This model is based on increasing/decreasing the association strength between reward and preceding object presentation. Come to think of it temporal difference learning is very similar to RW model and maybe its offshoot.

Significant amount of work in understanding decision making regions of the brain started after the seminal paper by Wolfram Schultz, Peter Dayan and Read Montague which showed that Dopamine neurons(DA) in Ventral Tegmental Area(VTA) encode the temporal prediction error.

This has led to a new field of research – Neuro-economics – for studying the decision-making aspects of the brain.

Schultz, W., Dayan, P., Montague, P.R. (1997) A Neural substrate of prediction and reward. Science, 275: 5306, 1593-1599. link


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: