10.1 Confirmation Bias

Humans tend to seek out and evaluate evidence in such a way as to reinforce their existing opinions, rather than testing them against new information or alternative views. Even scientists inevitably fall prey to confirmation bias. Students are motivated to innovate ways to minimize this bias in their personal judgments.

The Lesson in Context

This lesson continues the lessons on cognitive heuristics and biases, reminding us that even careful scientists can fall prey to such human tendencies. 10.2 Blinding will introduce techniques to counter these tendencies. In this lesson, we let students fall into the trap of biased assimilation by asking them to evaluate the same news article that is labelled leftwing for one group and rightwing for the other.

Earlier Lessons

2.2 Systematic and Statistical Uncertainty

Both biased assimilation and selective exposure introduce a systematic bias in our perception and assessment towards a direction that confirms our existing beliefs.

3.2 Calibration of Credence Levels

Actively open-minded thinking aims to measure one's avoidance of confirmation bias. It assesses whether one actively seeks out arguments contrary to their prior beliefs and evaluates evidence objectively independently of their beliefs.

9.1 Heuristics

Selective exposure together with the availability heuristic can exacerbate the bias in one's belief.

Later Lessons

10.2 Blinding

Blind analysis and preregistration are techniques to reduce motivated reasoning during the data collection and analysis process.

Takeaways

After this lesson, students should

Be wary of one's own tendency towards motivated reasoning and confirmation bias.
Be able to explain and differentiate the two mechanisms for confirmation bias, selective exposure and biased assimilation.
Be able to identify cases of selective exposure.
Be able to identify cases where biased assimilation is likely.
Be able to explain how confirmation bias can lead to errors.
Know how to reduce confirmation bias by actively seeking out and examining counter-evidence to their own beliefs.

Confirmation Bias

Also known as myside bias. Seeking or otherwise favoring evidence consistent with what is already believed or what is being tested.

Selective Exposure

Selectively seeking or exposing oneself to evidence that is likely to conform to prior beliefs or working hypotheses.

Biased Assimilation

Systematically favoring or discounting evidence to render that evidence more compatible with pre-existing beliefs or working hypotheses.

Social Media

Modern social media offer a perfect environment for selective exposure, where one only chooses to view content created by those that already confirm their preexisting worldview or beliefs. Content moderation algorithms of major websites exacerbate the problem by reinforcing one's choice in the type of "desirable" content, leading to the "filter bubble".

Bug Fixing

As a counterexample, when testing whether a web form taking an email address has been coded correctly, a programmer not only tries "sensesensibilityscience@berkeley.edu" to see if it works, but also non-email address strings like "https://berkeley.edu", "laf1209#$%^&", "Robert; DROP TABLE Students" to see if it breaks. The programmer actively seeks to disconfirm their hypothesis that "the web form is coded correctly".

Hubble Parameter

The measured value of the Hubble parameter (which describes the speed of the expansion of the Universe) published over time. The few early measurements are clearly too far away from the currently accepted value and overconfident, but they confirmed the accepted value at that time. As soon as one deviating value was published, a slew of publications finally came to support this new value.

Perfect Prediction Scam

This is a scam someone can run by starting with a large sample of people and claiming to have a method that perfectly predicts the outcome of a contest (for explanation purposes, say a football game). You send your prediction to a group of people: to half you say team A will win, and to the other half you say team B will win. After the game ends, half of the group will think your method worked. You repeat the process with that smaller group who now thinks your algorithm works, shrinking the group each week, until a small group of people think your method is always correct (confirmation bias, because each week they get new evidence that makes them more and more sure of themselves). When you are ready to end the scam, you ask them for money in return for a prediction of the final game.

The Perfect Prediction Scam

The Skeptic's Dictionary explanation of the scam.

Media and the 2016 Election

The media's reporting on the 2016 election is a good example of confirmation bias on the media's end. According to the linked article, the polls were far less sure about Clinton's victory than most media outlets reported: "Journalists just didn't believe that someone like Trump could become president [...] they cherry-picked their way through the data to support their belief."

The Media Has A Probability Problem

FiveThirtyEight on how outlets misread the 2016 election forecasts.

Salem Witch Trials

The entire logic of the Salem Witch Trials is riddled with confirmation bias. Once someone had been suspected of being a witch, finding further evidence was not difficult — much of it based on little more than a hunch. Once the idea was planted that someone was a witch, it was very hard to see evidence to the contrary. This is why so many of the trials were such a sham.

Salem witch trials — Spectral evidence

Wikipedia.

Palm Reading and Psychics

Confirmation bias is part of how palm readers and other psychics operate. A psychic starts by sharing general statements about you; as they make more and more correct-seeming statements, you begin to believe them more. Eventually, if the psychic says something incorrect, you can easily dismiss it because everything else has seemed so accurate.

Palm Readings: How Pseudoscience Works

PhillyVoice.

Being smart or knowledgeable shields you from confirmation bias.

Just being "smart" is not enough to avoid confirmation bias; indeed, sometimes cognitive effort and fluency actually exacerbate confirmation bias. One also needs to actively be looking for flaws in their own reasoning through practices such as actively open-minded thinking discussed in 3.2.

'Since even smart people get fooled by confirmation bias, there's nothing we can do about it.'

This assumption is also false since there are practices like the aforementioned that have been shown to actively reduce confirmation bias.

After this lesson, students should

Attitudes
1. Be wary of one's own tendency towards motivated reasoning and confirmation bias.
Concept Acquisition
1. Confirmation bias: Seeking or otherwise favoring evidence consistent with what is already believed or what is being tested.
  1. Selective exposure: Selectively seeking or exposing oneself to evidence that is likely to conform to prior beliefs or working hypotheses.
  2. Biased assimilation: Systematically favoring or discounting evidence to render that evidence more compatible with pre-existing beliefs or working hypotheses.
Concept Application
1. Recognize instances of "selective exposure" in scientific, policy, and everyday contexts.
2. Recognize instances of "biased assimilation" in scientific, policy, and everyday contexts.
3. Explain how different forms of confirmation bias (i.e., selective exposure, biased assimilation) can lead to errors.
4. In a given instance of confirmation bias, identify the following components:
  1. Prior beliefs/working hypothesis affecting reasoning,
  2. Evidence/interpretations that would be favored,
  3. Evidence/interpretations that would be missing/discounted, and
  4. How the confirmation bias might affect the resulting conclusions.

Additional Content

You must be logged in to see this content.