9.2 Biases

Learning Goals

[Link to PlayPosit]

[Link to instructional video]

More details

After this lesson, students should

1. Learn that we use heuristics as a shortcut in everyday decision making.
2. Recognise that while heuristics are useful and necessary, they can lead us astray by introducing biases in our decision making.
3. Be aware of cognitive biases and where they arise.
4. Learn the basics of Bayesian reasoning.

Definitions

• Base Rates

  The base frequency of a given attribute in a whole population.

  • Base Rate Neglect

    People frequently overlook the importance of base rates when calculating the probability of an event based on probabilities that seem more relevant to the specific case.

  • Bayes' Rule

    [math]\displaystyle{ \text{(Probability that a positive test is accurate)} = \text{(Base probability of positivity)} \times \frac{\text{(True positive rate of the test)}}{\text{(False positive rate of the test)}} }[/math]

• Representativeness Heuristic

  Cases in which how representative something is of a category or outcome is used as a proxy /for evaluating how likely the category membership or outcome is (not taking base rates into account).

• Conjunction Fallacy

  The tendency to neglect that something is less likely to be part of a subset of a set than a set itself. In reality, [math]\displaystyle{ A }[/math] is always more likely to be true than [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] because if [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] is true then [math]\displaystyle{ A }[/math] must be true. This usually happens as a consequence of the representativeness heuristic by means of [math]\displaystyle{ B }[/math] being representative of the set in question.

• Availability Heuristic

  Cases in which people use how readily something comes to mind as a proxy for an estimate of its probability.

Common Misconceptions

• Heuristics cause us to make wrong judgements so they're bad and we should stop using them.

  Heuristics can sometimes lead us towards fallacies. But, that does not mean that they are useless! Heuristics still tend to be better than making decisions arbitrarily. And we don't always have the time or means to fully analyze every decision.

Context

Humans make many decisions on a daily basis, often in the absence of complete information or under the constraints of time and mental capacity. We use heuristics as useful shortcuts for quick decision making, which may introduce bias in our conclusions. The purpose of the lesson is not to cast doubt on our use of heuristics, but to recognise the limitations of quick human judgments, where they may arise, as well as their consequences. This parallels 2.1 Senses and Instrumentation and 2.2 Systematic and Statistical Uncertainty, where the limitations of instruments are discussed and quantified, without rejecting the validity and usefulness of instruments altogether.

Before

2.1 Senses and Instrumentation

Senses and instrumentation are inherently imperfect, but imperfect tools can still be useful in obtaining partial knowledge. Similarly, heuristics are flawed, but they make extremely useful tools when time, knowledge, and mental resources are limited.

2.2 Systematic and Statistical Uncertainty

The use of heuristics can often introduce bias in our judgments—tendencies to make one decision more often than another, paralleling the idea of systematic uncertainty in instrumental measurements.

After

8.2 Biases

This lesson focuses on heuristics that affect our judgments of frequencies—how often things occur or likelihoods of events. The next lesson discusses biases in decision making that stem from a self-centred view of the world—an overemphasis on "me" and "now".

10.1 Confirmation Bias

We single out confirmation bias into its own topic, as it permeates scientific and group decision making, affecting both our sense of the prevalence of events around us as well as the importance of "me" and "now".

Recommended Outline

Before Class

• [Any essential logistical things that need to be done for this class]
• Prepare a seating chart.
• Review PlayPosit and discussion questions and ask faculty, Gabriel, or Emlen any questions you have.
• (Optional) Prepare a presentation.

During Class

• (5 min) Come up with some fun way to assign the roles of spokesperson and notetaker (e.g. earliest birthday in the year, lives furthest from campus). Remind them of the responsibilities of these roles.
• (25 min) Go through the KOALA-21 example and exercise. Note that this has many sub-steps and is worth reviewing how you'll present it.
• (6 min) Guide the students through the Jessie Again prompt and let them work through the problem.
• (17 min) Have the students answer the conjunction fallacy problem and present the corresponding examples.
• (16 min) Have the students answer the availability heuristic problem and present the corresponding examples.
• (6 min) Have the students discuss bounded rationality in small groups.
• (5 min) Collect questions for plenary.

After Class

• [Any essential logistical things that need to be done as followup for this class]
• Collect answers from notetakers for the forum / plenary.

Lesson Content

Clicker Question

You're sitting in your room cramming for an exam when your roommate decides to throw an impromptu party and invites people from all over the university. Realizing that you're not going to get any work done, you decide to make the most of it and start mingling with the crowd. In the process, you strike up a conversation with someone named Jessie who starts droning on and on about rockets. They go on for so long that you start to lose interest and begin thinking about what sort of person Jessie really is.

Here's some helpful numbers:

• There is one engineering major for every three non-engineering majors at your college.
• Half of all engineering majors like rockets.
• A sixth of all non-engineering majors like rockets.

Which is more likely?

1. Jessie is an engineering major.
3. Jessie is a non-engineering major.
4. It's equally likely that Jessie's an engineering or non-engineering major.

KOALA-21

We introduce and practice Bayes' rule in this activity.

Instructions

• (2 min) Have the students answer the clicker question that's listed just below these instructions.
• (1 min) Remind the students of what true/false positive/negative rates mean. Emphasize that [math]\displaystyle{ \text{(true positive rate)} = 1 - \text{(false negative rate)} }[/math], and [math]\displaystyle{ \text{(true negative rate)} = 1 - \text{(false positive rate)} }[/math].
• (3 min) Show and explain the following diagram.
• (10 min) POSSIBLE, NOT explaining odds this year: [[[[Give the following explanation to the students"
  • Introduce the idea of odds, as opposed to probabilities. Examples: 50% probability of a coin flip translates to 1:1 odds; 90% probability translates to 9:1 odds. Conversely, a:b odds translate to a/(a + b) probability. Bayes' rule is simplest when phrased in terms of odds.]]]]
  • The main equation we will use is

    [math]\displaystyle{ \text{(Probability that one actually has the disease)} = \text{(Prior probability of having the disease)} \times \frac{\text{(True positive rate)}}{\text{(False positive rate)}}. }[/math]

  • In the case of KOALA-21,

    [math]\displaystyle{ \text{(Probability that the KOALA has the disease)} = (1/99) \times \frac{1-0.09}{0.10} \approx 1/9.9. }[/math]

  • We just wrote this as a ratio. But, as odds, we represent it as 1:9.9.
  • The actual probability of having the disease is [math]\displaystyle{ 1/(1+9.9) \approx 0.092 }[/math].
• (5 min) Have the students do the discussion in small groups.

Clicker Question

Suppose there is an epidemic of KOALA-21 breaking out among koalas in New South Wales, which about 1% of the koalas have contracted. A test for KOALA-21 was developed, whose false positive rate is 9% and false negative rate is 10%. If a particular koala has tested positive of KOALA-21, what is the actual probability that it really has KOALA-21?

1. Between 0-25%
2. Between 25-50%
3. Between 50-75%
4. Between 75-100%

Discussion Question

Have students work in small groups to work out this problem, following the KOALA-21 example above. Give assistance where needed.

1. Based on a daily case count of 73,000 (November 2021) and a 14-day recovery period, it can be Fermi estimated that the prevalence of Covid-19 is 0.3%. For the commonly used PCR test, the true positive rate (sensitivity) is 98%, and the true negative rate (specificity) is 80%. If you do one PCR test and get a positive result, what are the actual odds that you have Covid?
2. From the odds you obtained from the first test, if you do another PCR test and get a positive result, what are now the actual odds that you have Covid?
3. Why is it often recommended for first-time positive patients to test for Covid a second time?

1. Kuhn argues that scientists do not seek out evidence to "refute the theories embedded in their paradigm". This would be an example of which heuristic? (Biased Assimilation)
2. Read the following transcript from 11.1 inclusive leadership:

   But how do you actually become an inclusive leader? What are the keys to inclusive

leadership? This is some great research done by Juliet Bourke and Andrea Espedido. And they recognize certain keys to becoming a more inclusive leader. The first key is simply an awareness of bias. We all have our own biases. So it's a cognizance of our personal blind spots. It's being able to recognize the flaws in a system, recognize that the system isn't meritocratic, but still putting an effort to try to pursue a more meritocratic system. It's not pretending we don't have biases, because we all do, but it's recognizing what our personal blind spots may be, and trying to counteract them.

Select one or two biases from our section and discuss how they might contribute to inhibiting inclusivity as a leader? How might we approach them to reduce this burden?

Jessie Again

Now that we have some practice working out Bayesian odds with diseases, it's time to try and figure out who Jessie really is. Recall our "disease" formula but somewhat generalized.

[math]\displaystyle{ \text{(Odds that a positive test is accurate)} = \text{(Base odds of positivity)} \times \frac{\text{(True positive rate of the test)}}{\text{(False positive rate of the test)}} }[/math]

We're using the term "test" here very broadly. For example, our original conversation with Jessie counts! In that case, we were using Jessie's interest in rocketry as a "test" for whether or not they're an engineering major. If Jessie is interested in rockets and is also an engineering major then we have a true positive. But if Jessie isn't an engineering major then it's a false positive.

Here are the values from the original problem. Given this information, what are the odds that Jessie is an engineering major?

1. There is one engineering major for every three non-engineering majors at your college.
2. Half of all engineering majors like rockets.
3. A sixth of all non-engineering majors like rockets.

Conjunction Fallacy

1. Andy is a junior at Berkeley. His favorite book is Howard Zinn's "People's History of the United States." He's passionate about politics, and he regularly attends local protests. He is most likely to be:
   1. A computer science major.
   2. A computer science and also a political science major (double major).
   3. A computer science major who is a member of the Berkeley College Democrats.
   4. A computer science and also a political science major (double major) who is a member of the Berkeley College Democrats.

Conjunction fallacy is often made due to our use of the representativeness heuristic—how representative something is of a category or outcome is used as a proxy /for evaluating how likely the category membership or outcome is (not taking base rates into account). Present one or more of the following examples (source).

1. One group of Dutch local politicians is asked how likely they think their municipality will make the headlines of all major newspapers next year, while another group is asked how likely they think their municipality will make the headlines next year due to a terrorist attack on King's Day. (For context, the terrorist attack on King's Day in 2009, in which a car drove into a crowd at the royal parade, killing 8, is a salient event to the Dutch public.) Do you expect the first group or the second group to rate their event to be more likely?
2. One group of participants assesses the likelihood of an earthquake hitting California next year and causing a massive flood, while the other group assesses the likelihood of a massive flood somewhere in North America next year. Do you expect the first group or the second group to rate their event to be more likely?

Availability Heuristic

1. In the English language, are there more words that have 'K' as the first letter OR as the third letter?
   1. More words with 'K' as the first letter.
   2. More words with 'K' as the third letter.

Present one or more of the following examples:

1. Is it more likely that one dies due to a shark attack or due to falling airplane parts?
2. Repeated vivid stories about a type of events in the media inflate people's perception of the rates or likelihood of such events.
   1. Vivid descriptions of crimes committed by immigrants skew public perception about immigration as a threat to public safety.
   2. Mass murders and terrorist attacks provide more salient memories, compared to domestic homicides, but are actually less common. The public, however, is typically more concerned with terrorist attacks than domestic homicides.
3. "Gratidude journals" encourage one to record only the positives of one's daily life, providing an abundance of examples of good days compared to bad days. We can use availability heuristic to our advantage to improve our mood.

Bounded Rationality

1. "Cognitive heuristics are a hindrance to rational reasoning. They lead to poor judgment and harmful cognitive biases in decision making, and we should strive to avoid them." Do you agree with this statement? Discuss in small groups.

Collect Questions for Plenary

(5 min) Collect remaining questions from the students for faculty in plenary (can be questions for clarification, extension, discussion, etc.), and add [ here].