# Handicapping Study #### Key Finding The study showed that more information does not necessarily improve the *quality* of a decision, but it will improve one's *confidence* in the decision. #### More Information I came across this study via Adam Robinson in Tribe of Mentors. It can be summarized as follows: * A group of 8 handicappers were presented with a list of 88 variables taken from a chart of horses past performance * The handicappers were asked to select 5 variables from the 88 that they would like to have access to in predicting the winner of the horse race. They were also asked to rate how confident they were in their prediction. * With 5 variables, the handicappers were 17% accurate in their predictions. They rated themselves (on average) as 19% confident. So, given 5 variables confidence and decision making accuracy were nearly equal. * The, the handicappers were given 10, 20, and 40 variables of their choosing and then asked to make predictions of the winner of the horse race. * What happened was the accuracy has flat-lined at ~17%. On the other hand confidence nearly doubled and increased to 34%.  * What we learn from this is that beyond a certain minimum amount of information, additional information only feeds [Confirmation Bias](Confirmation%20Bias.md). #### Musings This is related to [redunancy-in-data](redunancy-in-data.md) and [Dimensionality-Reduction](Dimensionality-Reduction.md). If we think of our decision (output variable) as a function of the information we have (input variables), likely a decision is only heavily influenced by a handful of input variables. In a sense we can classify our input variables into three buckets: 1. **Correlated (potentially via causation) with our output/decision.** For example, consider the decision of changing careers. An input variable here may be `freedom provided by new career`. This variable is directly tied to your resulting decision. If freedom and autonomy are important to you, then you likely will factor this in to your decision in changing careers. 2. ** Correlated with our output/decision, but also correlated with another input variable that is more appropriate and actionable in relationship to the output.** Continuing our previous example, let us consider two new input variables: `paid time off` and `vacation days`. These two variables are indeed correlated with the decision we are trying to make. However, they are essentially redundant; one almost completely encodes the other. If we mistakenly treat them as two unique pieces of information it may leave us feeling *more confident* in our decision (since two piece of information in favor of a decision feels more powerful than only one), when in reality it was only one piece of information to being with. Worth noting is that both of these variables are in reality inputs to `freedom provided by new career`. In a sense this is problem of dimensionality reduction: several variables can be combined/reduced to yield the single, most information `freedom provided by new career` 3. **Not correlated to the output/decision.** This can either be *noise* or a variable that simply has not relation to the output variable. For instance, if the new career has a plethora of jobs in downtown Manhattan but you don't care one way or the other whether you work in a city, this doesn't correlate with the output. #### Resources * Slovic comments on the study in this article:  * [Adam robins commentary](https://ma.tt/2017/11/adam-robinson-on-understanding/) (excerpt from Tribe of Mentors) --- tags: #scientific-study #biases #decision-making links: [Decision by Design MOC](Decision%20by%20Design%20MOC.md) [Stop-FLOP-or-Know](Stop-FLOP-or-Know.md) [When-to-Decide](When-to-Decide.md) created: 2020-11-25 modified: 2020-11-25