Descisions From Description and Experience

One of the interesting lines of recearch that I have recently become more involved with is the distinction between making decisions based on description and making them based on experience. In this research, these are referred to as Decisions from Experience (DfE) and Decisions from Description (DfD). In the past, I have studied differences and similarities between learning and decision making, which is as similar area of research in cognitive psychology. In the future I will be exploring whether these differences between decisions made from description and experience also occur in large language models, but for this post I will be focusing on the background of these differences.

One of that classical results inf DfE and DfD research is the so-called ‘Experience-Description Gap’. When I was first learning about this I took the gap to refer to a difference between how peoples decisions differ from optimal based on whether they are making those decisions in the setting of experience or description. For example, a bias that may impact someone’s decision making to be suboptimal in a description based task would be the following: say you are presented with two dice, one with the numbers [1,2,3,4,5,6], and the other with the numbers [1,1,1,1,1,20] on it. If you were told that you could roll either of these dice and recieve a dollar payout of the die roll, which would you choose? In this case, we may see that the so-called optimal decision, from the perspective of expected utility, is the second die which has a mean value of ~4.2, compared to the mean of 3.5 on the left die. However, many people will prefer the left die due to the lower variance in outcomes. Specifically, the second die has a high likelyhood of an outcome of 1, while the other die has evenly distributed outcomes. In this case we could explain the behavior of someone who chooses the left die to roll by saying that they are ‘risk-averse’ because they made a suboptimal decision (in a sense).

There are many possible interacting phenomenon when an actual human in an experimental setting, or in real life, makes decisions based on the description of possible events that may occur. Does the person only need one dollar for a snack in the vending machine outside, and not care about the other outcomes? Is the person rich and thus doesn’t care about outcomes less than $10? Or is the person a gambler and wants to experience the thrill of rolling the right die because they place an external utility on the feeling of not knowing the outcome, beyond the actual roll of the die itself? These and many more are all possible explanations of one specific instance of someone choosing in this task. However, the reason we are interested in explaining trends of behavior like risk-aversion is that they tend to be relatively consistent across populations and decision scenarios, meaning that they may be useful ways of understanding some of the motivations that people have when making decisions. In this case, the decider seems to be acting in a way that they add an aditional utility onto choices with more sure outcomes, compared to ones with less sure outcomes. This is a reasonable conclusion to make given the literature on decision making from description, that this risk-aversion is at least someone responsible for some decisions that humans make.

One of the key points of the usefulness of this explanation of behavior among all of the possible explanations we could concieve of is that it is relatively consistent. What we mean by that is if you design some experiment where a preference for options with lower variance could be expressed by the behavior of individuals, we would expect some percentage of the population under consideration to express that risk-averse behavior. By testing the level of risk-aversse, risk-seeking, and risk-neutral behavior under a variety of different conditions, it becomes less likely that there are unconsiered factors impacting all the decisions made in these conditions, and more likely that there is some degree to which risk is affecting the decisions being made.

One issue with the existence of risk-aversion in decision making more generally occurs when we transition into the setting of decision making from experience. We can similarly conceptualize this type of learning task by using a simple example: imagine you are presented with two dice, one with the letters [a,b,c,d,e,f] and the other with the letters [a,a,a,a,a,t], and you are told that after rolling a letter, you will recieve a corresponding dollar payment based on the letter you roll. Immediately (without considering the previous example) we wouldn’t have strong evidence for preferring one of the dice over the other. So we might begin the task by choosing a die at random, and rolling it. If for instance we roll the right die and roll an ‘a’ after which we recieve $1, we may want to try the left die to see what happens with those letter, after we roll a ‘d’ we might recieve $4.

Now, lets consider someone who has rolled the left and right dice a few times and represend their experience as a vector of the action they selected, the letter that they observed, and the payment they recieved [[‘Left’, ‘a’, $1], [‘Right’, ‘a’, $1], [‘Left’, ‘f’, $6], [‘Right’, ‘t’, $20], [‘Right’, ‘a’, $1]]. Now, the averages of the results when selecting the left die is 3.5, and the average when selecting right die is 7.66. Additionally, we can compute the variances of these results to see that the left die has an outcome variance of 12.5 and the right die has a variance of 120.33. If we assigned some penalty to the higher variance outcomes, we might expect that someone would prefer the left die under some conditions since it has a lower variance. However, research from decisions from experience has found that we can typically expect a reversal of the trend in which many people prefer the right die when making decisions from experience, even though it could be described as being risk-seeking which is less commonly observed in decisions from description.

The surprising reversal of outcome variance preferences between the description and experience settings has many possible explanations. One possibility is that these decision differences result of an overattention to rare events when they are experienced, and an underattention to rare events when they are described. If we take the above example, we can see that someone who weighs lower likelihood outcomes as being even less likely would produce risk-averse behavior in the description setting. Specifically, we can see that the left die has an uniform likelihood of all outcomes, while the right die has a 83.33% chance of rolling a 1 and a 16.66% chance of rolling a 20. If we adjust these probabilities to an altered percieved probability of, say, 90% of rolling a 1 and a 10% chance of rolling a 20, then the percieved expected utility would be (0.9 * 1) + (0.1 * 20) = 2.9 which is less than the expected utility of the left die at 3.5. Meanwhile, if we assigned a higher probability to experienced outcomes when we would reverse this bias.

While this explanation provides neat and tidy predictions and somewhat of an explanation of the behavior, it doesn’t fully satisfy us, as the specific reason why people would differentially weight these outcomes in the two cases is not yet described. There has been work in explaning the sources of this behavior more explicitly, and these will be related to my interest in exploring whether large language models display the same reversal of expectations between the two conditions. There are some major issues with applying this directly onto LLMs, specifically that they have a much different way of representing experience as humans do. Thus, it isn’t immedietly clear whether and to what extent LLMs may be similar to humans in this regard. I will continue to update posts of this as my experimentation progresses with LLMs.