|
|
| Line 5: |
Line 5: |
| {{Navbox}} | | {{Navbox}} |
|
| |
|
| == Useful Links == | | == The Lesson in Context == |
|
| |
|
| * [[:File:Confirmation Bias Cards.pdf|Confirmation Bias Cards]]
| | <!-- Always begin section with a description of this lesson in relation to the course as a whole. --> |
| * [https://docs.google.com/forms/d/19MqAkCwXoT6OV_AZ0yakpC8pZs_FTYYpTbJPpF7jfVk/edit?usp=drivesdk Abstract Evaluation Google Form Template]
| | This lesson continues the lessons on cognitive [[9.1 Heuristics|heuristics]] and [[9.2 Biases|biases]], reminding us that even careful scientists can fall prey to such human tendencies. [[10.2 Blinding]] will introduce techniques to counter these tendencies. In this lesson, we let students fall into the trap of biased assimilation by asking them to evaluate the same news article that is labelled leftwing for one group and rightwing for the other. |
| * [[:File:Gun Control Abstracts.pdf|Gun Control Abstracts]]
| |
| * [https://docs.google.com/document/d/1LeAiHBYuHHcwGbpfF4sXwykTYqiSLe2vnrCRchNZgyo/edit?usp=sharing Three Column Overview of the Week]
| |
| * [https://docs.google.com/presentation/d/1ZLF0UNxQ1ua3S01bL0PoqhUjsqF03OZPOATDoTyZWEI/edit?usp=drivesdk Lesson Slides (2023 Master)]
| |
| * [https://sensesensibilityscience.berkeley.edu/topic/19 Website Page]
| |
|
| |
|
| === Readings and Assignments ===
| | <!-- Expandable section relating this lesson to earlier lessons. --> |
| | {{Expand|Relation to Earlier Lessons| |
| | {{ContextLesson|2.2 Systematic and Statistical Uncertainty}} |
| | {{ContextRelation|Both biased assimilation and selective exposure introduce a systematic bias in our perception and assessment towards a direction that confirms our existing beliefs.}} |
| | {{ContextLesson|3.2 Calibration of Credence Levels}} |
| | {{ContextRelation|Actively open-minded thinking aims to measure one's avoidance of confirmation bias. It assesses whether one actively seeks out arguments contrary to their prior beliefs and evaluates evidence objectively independently of their beliefs.}} |
| | {{ContextLesson|9.1 Heuristics}} |
| | {{ContextRelation|Selective exposure together with the availability heuristic can exacerbate the bias in one's belief.}} |
| | }} |
| | <!-- Expandable section relating this lesson to later lessons. --> |
| | {{Expand|Relation to Later Lessons| |
| | {{ContextLesson|10.2 Blinding}} |
| | {{ContextRelation|Blind analysis and preregistration are techniques to reduce motivated reasoning during the data collection and analysis process.}} |
| | }} |
|
| |
|
| * [https://www.youtube.com/watch?v=xURCmzmQG8s The Gardener and the Carpenter]
| | == Takeaways == |
|
| |
|
| ==== Lecture Video ====
| | <tabber> |
|
| |
|
| <youtube>https://youtu.be/pyB3WZFTf08</youtube>
| | |-|Learning Goals= |
| | |
| == Learning Goals ==
| |
|
| |
|
| After this lesson, students should | | After this lesson, students should |
| | <!-- Learning goals are written as a numbered list. --> |
| # Be wary of one's own tendency towards motivated reasoning and confirmation bias. | | # Be wary of one's own tendency towards motivated reasoning and confirmation bias. |
| # Be able to explain and differentiate the two mechanisms for confirmation bias, selective exposure and biased assimilation. | | # Be able to explain and differentiate the two mechanisms for confirmation bias, selective exposure and biased assimilation. |
| Line 32: |
Line 40: |
| # Know how to reduce confirmation bias by actively seeking out and examining counter-evidence to their own beliefs. | | # Know how to reduce confirmation bias by actively seeking out and examining counter-evidence to their own beliefs. |
|
| |
|
| === Definitions ===
| | |-|Definitions= |
|
| |
|
| * '''Confirmation Bias'''
| | <!-- Definitions must be written with the Definition and Subdefinition templates. The first Definition should have the "first=yes" flag at the end. --> |
| *: Also known as '''myside bias'''. Seeking or otherwise favoring evidence consistent with what is already believed or what is being tested.
| | {{Definition|Confirmation Bias|Also known as myside bias. Seeking or otherwise favoring evidence consistent with what is already believed or what is being tested.|first=yes}} |
| ** '''Selective Exposure'''
| | {{Subdefinition|Selective Exposure|Selectively seeking or exposing oneself to evidence that is likely to conform to prior beliefs or working hypotheses.}} |
| **: Selectively seeking or exposing oneself to evidence that is likely to conform to prior beliefs or working hypotheses.
| | {{Subdefinition|Biased Assimilation|Systematically favoring or discounting evidence to render that evidence more compatible with pre-existing beliefs or working hypotheses.}} |
| ** '''Biased Assimilation'''
| | <br /> |
| **: Systematically favoring or discounting evidence to render that evidence more compatible with pre-existing beliefs or working hypotheses.
| |
|
| |
|
| === Examples ===
| | |-|Examples= |
|
| |
|
| * Modern social media offer a perfect environment for selective exposure, where one only chooses to view content created by those that already confirm their preexisting worldview or beliefs. Content moderation algorithms of major websites exacerbate the problem by reinforcing one's choice in the type of "desirable" content, leading to the "filter bubble".
| | <!-- Example formatting is still experimental. --> |
| * As a counterexample, when testing whether a web form taking an email address has been coded correctly, a programmer not only tries "sensesensibilityscience@berkeley.edu" to see if it works, but also non-email address strings like "https://berkeley.edu", "laf1209#$%^&", "Robert; DROP TABLE Students" to see if it breaks. The programmer actively seeks to disconfirm their hypothesis that "the web form is coded correctly".
| | '''Social Media''' |
| * The measured value of the Hubble parameter (which describes the speed of the expansion of the Universe) published over time. The few early measurements are clearly too far away from the currently accepted value and overconfident, but they confirmed the accepted value at that time. As soon as one deviating value was published, a slew of publications finally came to support this new value. [[File:H0 over Time.jpeg|center|400px]]
| | : Modern social media offer a perfect environment for selective exposure, where one only chooses to view content created by those that already confirm their preexisting worldview or beliefs. Content moderation algorithms of major websites exacerbate the problem by reinforcing one's choice in the type of "desirable" content, leading to the "filter bubble". |
| | {{Line}} |
| | '''Bug Fixing''' |
| | : As a counterexample, when testing whether a web form taking an email address has been coded correctly, a programmer not only tries "sensesensibilityscience@berkeley.edu" to see if it works, but also non-email address strings like "https://berkeley.edu", "laf1209#$%^&", "Robert; DROP TABLE Students" to see if it breaks. The programmer actively seeks to disconfirm their hypothesis that "the web form is coded correctly". |
| | {{Line}} |
| | '''Hubble Parameter''' |
| | : The measured value of the Hubble parameter (which describes the speed of the expansion of the Universe) published over time. The few early measurements are clearly too far away from the currently accepted value and overconfident, but they confirmed the accepted value at that time. As soon as one deviating value was published, a slew of publications finally came to support this new value. [[File:H0 over Time.jpeg|thumb]] |
|
| |
|
| === Common Misconceptions ===
| | |-|Common Misconceptions= |
|
| |
|
| * ''Being smart or knowledgeable shields you from confirmation bias.''
| | <!-- Misconceptions must be written with the Misconception template. The first Misconception should have the "first=yes" flag at the end. --> |
| *: Just being "smart" is not enough to avoid confirmation bias; indeed, sometimes cognitive effort and fluency actually exacerbate confirmation bias. One also needs to actively be looking for flaws in their own reasoning through practices such as actively open-minded thinking discussed in [[3.2 Calibration of Credence Levels|3.2]].
| | {{Misconception|[Description of misconception.]|[Resolution.]|first=yes}} |
| * ''Since even smart people get fooled by confirmation bias, there's nothing we can do about it.''
| | {{Misconception|[Description of misconception.]|[Resolution.]}} |
| *: This assumption is also false since there ''are'' practices like the aforementioned that have been shown to actively reduce confirmation bias.
| |
|
| |
|
| == Context ==
| | </tabber> |
|
| |
|
| This lesson continues the lessons on cognitive [[9.1 Heuristics|heuristics]] and [[9.2 Biases|biases]], reminding us that even careful scientists can fall prey to such human tendencies. [[10.2 Blinding]] will introduce techniques to counter these tendencies. In this lesson, we let students fall into the trap of biased assimilation by asking them to evaluate the same news article that is labelled leftwing for one group and rightwing for the other.
| | == Useful Resources == |
|
| |
|
| === Before ===
| | <tabber> |
|
| |
|
| : '''[[2.2 Systematic and Statistical Uncertainty]]'''
| | |-|Lecture Video= |
| :: Both biased assimilation and selective exposure introduce a systematic bias in our perception and assessment towards a direction that confirms our existing beliefs.
| |
| : '''[[3.2 Calibration of Credence Levels]]'''
| |
| :: Actively open-minded thinking aims to measure one's avoidance of confirmation bias. It assesses whether one actively seeks out arguments contrary to their prior beliefs and evaluates evidence objectively independently of their beliefs.
| |
| : '''[[9.1 Heuristics]]'''
| |
| :: Selective exposure together with the availability heuristic can exacerbate the bias in one's belief.
| |
|
| |
|
| === After ===
| | <br /><center><youtube>[id]</youtube></center><br /> |
|
| |
|
| : '''[[10.2 Blinding]]'''
| | |-|Discussion Slides= |
| :: Blind analysis and preregistration are techniques to reduce motivated reasoning during the data collection and analysis process.
| |
|
| |
|
| == Recommended Outline == | | {{LinkCard |
| | |url=[address] |
| | |title=Discussion Slides Template |
| | |description=The discussion slides for this lesson. |
| | }} |
| | <br /> |
|
| |
|
| === Before Class ===
| | |-|Handouts and Activities= |
|
| |
|
| * Print out abstracts, enough to give one to each student.
| | {{LinkCard |
| * Prepare a seating chart.
| | |url=[address] |
| * Be ready to present a link or a QR code to the Google Form.
| | |title=[Title] |
| * Be ready to create a quick histogram from the Google Form, comparing average ratings of the four articles. Have on hand the key to which abstract is which, so you can label the graph during the big reveal.
| | |description=[Description.]}} |
| * Review PlayPosit and discussion questions and ask faculty, Gabriel, or Emlen any questions you have.
| | {{LinkCard |
| * (Optional) Prepare a presentation.
| | |url=[address] |
| | |title=[Title] |
| | |description=[Description.]}} |
| | <br /> |
|
| |
|
| === During Class ===
| | |-|Readings and Assignments= |
|
| |
|
| * (5 min) Come up with some fun way to assign the roles of spokesperson and notetaker (e.g. earliest birthday in the year, lives furthest from campus). Remind them of the responsibilities of these roles.
| | {{LinkCard |
| * (10 min) [[#Selective Exposure|Both Sides/Selective Exposure Activity]].
| | |url=[address] |
| * (30 min) [[#Biased Assimilation|Biased assimilation activity]].
| | |title=[Title] |
| * (20 min) [[#Blickets|Blickets]]
| | |description=[Description.]}} |
| * (10 min) Go through the [[#Overview Discussion Questions|discussion questions]].
| | {{LinkCard |
| * (5 min) Use any remaining time to let people work on the project.
| | |url=[address] |
| | |title=[Title] |
| | |description=[Description.]}} |
| | <br /> |
|
| |
|
| === After Class ===
| | </tabber> |
|
| |
|
| * Collect answers from notetakers for the forum / plenary.
| | == Recommended Outline == |
|
| |
|
| == Lesson Content == | | === Before Class === |
|
| |
|
| === Selective Exposure ===
| | [Any steps that need to be taken before class.] |
|
| |
|
| This activity is based on an [https://www.exploratorium.edu/middleground/play/my-side-bias exhibition at the Exploratorium]. It's intended to demonstrate the phenomenon of selectively deciding what information we expose ourselves to. Even if the students don't, in aggregate, end up with any systematic bias based on their beliefs in what arguments they choose to read, they will likely still "feel" the effects to some degree during the process and in their questioning of why they chose what arguments to grab.
| | === During Class === |
|
| |
|
| ==== Preparation ==== | | {| class="wikitable" style="margin-left: 0px; margin-right: auto;" |
| | |[n] Minutes |
| | |[Activity.] |
| | |- |
| | |[n] Minutes |
| | |[Activity.] |
| | |} |
|
| |
|
| # Print 3 copies of this PDF.
| | == Lesson Content == |
| # For each side (for/against) of each topic, prepare one envelope to hold all its arguments.
| |
| # Clearly write the topic on the front of the envelope, as "Arguments for ...." and "Arguments against..."
| |
| # Cut out the arguments and place them in their respective envelopes.
| |
| | |
| ==== Instructions ==== | |
| | |
| # Multiple topics on which students may already have an opinion:
| |
| ## Should the US have universal health care?
| |
| ## Should you eat more blueberries for your health?
| |
| ## Should parents receive vouchers to pay for private schools?
| |
| ## Should people reduce their social media use?
| |
| # Each topic has multiple arguments for/against, printed redundantly (students are told this).
| |
| # Two envelopes are prepared for each topic, containing the arguments for/against, respectively, brightly marked ARGUMENTS FOR/ARGUMENTS AGAINST on the outside.
| |
| # Students are asked to queue and pick a topic and one argument from either of the envelopes for that topic.
| |
| # They can choose to queue again for another argument.
| |
| # At the end, a quick show of hands for whether students initially picked an argument from the side they already agree with. For those that went back to queue again, show of hands for whether they picked from the side they agreed with or not.
| |
| # After the discussion questions, collect the arguments and put them back in their respective envelopes.
| |
| | |
| ==== Discussion Questions ====
| |
| | |
| As a whole class, discuss:
| |
| # Why did you choose to open the envelope that you did? (Pick students who agreed/disagreed with the envelope they chose.)
| |
| # Which form of confirmation bias is this activity trying to demonstrate, selective exposure or biased assimilation?
| |
| # Have you committed this form of confirmation bias in your daily life? What could you do to mitigate it?
| |
| | |
| Collect the arguments and place them back in their respective envelopes.
| |
| | |
| === Biased Assimilation ===
| |
| | |
| This activity follows the classic study on biased assimilation (Lord, Ross, & Lepper, 1979), where subjects are given identical abstracts of a scientific study, but with conclusions reversed for half of the subjects, and are then asked to assess the validity of the supposed study. Biased assimilation is demonstrated when subjects rate abstracts with belief-congruent conclusions higher than ones with belief-incongruent conclusions.
| |
|
| |
|
| ==== Preparation ==== | | === Activity 1 === |
|
| |
|
| The abstracts (conditioned on a randomly assigned A/B/C/D for each student) and the following questions have been made into a [https://docs.google.com/forms/d/19MqAkCwXoT6OV_AZ0yakpC8pZs_FTYYpTbJPpF7jfVk/edit?usp=drivesdk Google form] ahead of time, and its link put into a QR code. {{Todo|This needs to be updated every year.}}
| | [Brief description of and motivation for the activity] |
| | | {{BoxCaution|[Common misconception or thing to look out for.]}} |
| # What's your position on the effect of gun control laws on gun violence?
| | {{BoxWarning|[Thing you really need to look out for!]}} |
| ## I think gun control laws have no effect on gun violence
| | {{BoxTip|title=[Title]|[Useful tip, guideline, or other background.]}} |
| ## I think gun control laws reduce gun violence
| |
| # Which abstract did you read?
| |
| ## A
| |
| ## B
| |
| ## C
| |
| ## D
| |
| # How convincing do you think the evidence in this study is? [Please try to evaluate the evidence independent of your agreement with the conclusion.] (1-10 scale)
| |
| # Briefly, say why you did or didn't find the evidence convincing.
| |
|
| |
|
| ==== Instructions ==== | | ==== Instructions ==== |
|
| |
|
| # Go around the room and assign A, B, C, or D to each of the students.
| | {| class="wikitable" style="margin-left: 0px; margin-right: auto;" |
| # (10 min) Ask students to scan a QR code to open the Google Form. In the form, they must choose the letter they are assigned to read the [[:File:Gun Control Abstracts.pdf|respective abstracts]].
| | |[n] Minutes |
| # (10 min) The GSI reveals how the abstracts were created (one supporting each position, with flipped abstracts modified to support the opposite conclusion; see bolded abstracts below). Then GSI makes a quick graph of the results and the students discuss how they reflect biased assimilation. If it works as expected, belief-congruent abstracts will be rated higher than belief-incongruent abstracts.
| | |[Activity.] |
| # (10 min) The GSI shows students the results from the original study (Lord, Ross, & Lepper, 1979) and leads a brief discussion on the implications for biased assimilation in incorporating other evidence.
| | |- |
| [[File:Lord Ross Lepper.png|center|500px]] | | |[n] Minutes |
| | |[Activity.] |
| | |} |
|
| |
|
| === Blickets === | | ==== Discussion Questions ==== |
| | |
| This is another demonstration that adults tend to seek confirmation of their initial hypothesis, rather than consider unlikely alternative explanations as children do. If running low on time, skip this to reach the final discussion questions. {{Caution|small=right|Start at about a half hour in. <br/ ><br/ > Don't make a moral values judgment over whether exploring or exploiting is universally preferable. Sometimes taking advantage of your choices is good.}}
| |
| | |
| <youtube>https://youtu.be/xURCmzmQG8s?t=1800</youtube>
| |
| | |
| ==== Length of Video ====
| |
| | |
| You can show different amounts of the video depending on how much time you have and what you want to include. The version included in the slides has a fairly long cut of the video (19:20). If you're running out of time and want to stop it at 7:41, 13:45, or 15:58, that's okay.
| |
| # 7:41 long. Has description of blicket detector and shows one video of a kid messing around with it. Gives no explanation of why kids might be better at blicketing.
| |
| # 13:45 long. Has everything the previous one has. Also has Alison hypothesizing some reasons kids might be better and another experiment showing them "intelligently imitating" adults. (AKA useful to learn "good enough" solutions instead of yolking about to find your own.)
| |
| # 15:58 long. Has everything the previous one has. Also has experiment where kids find everything a toy can do through play and none of it through parenting.
| |
| # 19:20 long. Has everything the previous one has. Also frames this in terms of "explore/exploit" and "high/low temperature search."
| |
| | |
| ==== Jargon ====
| |
| | |
| This video uses the terms "conjunctive" and "disjunctive." Students won't be tested on these, but they're useful for understanding the video.
| |
| * '''Conjunctive'''
| |
| *: Logical "and." Both components need to be true for the proposition to be true.
| |
| *:* ''S'' = ''P'' and ''Q''
| |
| *:* ''S'' is true if and only if both ''P'' and ''Q'' are true. Otherwise ''S'' is false.
| |
| * '''Disjunctive'''
| |
| *: Logical "or." ''Only one'' component needs to be true for the proposition to be true.
| |
| *:* ''S'' = ''P'' or ''Q''
| |
| *:* ''S'' is true if ''P'' is true, or if both are true. Otherwise ''S'' is false.
| |
| | |
| ==== Blickets Discussion Questions ====
| |
| | |
| # When are the kids less likely to fall victim to confirmation bias than adults? Why might this be rational or not?
| |
| #: {{Answer|The kids practice confirmation bias when given specific instructions.The follow the "good enough" solution rather than playing around and finding the optimal one.}}
| |
| # What are some circumstances where we should play around and search for solutions like the kids do?
| |
| #: {{Answer|The obvious answer here is science! We want our models of the world to be as good as possible.}}
| |
| # Does this tell us anything about how to structure problems so as to reduce confirmation bias?
| |
| #: {{Answer|This requires setting up incentives (like scientific optimism and not punishing wrong results) to keep hunting for better solutions. It also means making "good enough" solutions less tempting for us to snatch. This can be solved through some of the blinding techniques we cover next lesson.}}
| |
| | |
| === Overview Discussion Questions ===
| |
| | |
| Discuss the following questions in small groups, and then spend a few minutes to share with the whole class.
| |
| # (5 min) Give two or more examples of ''selective exposure'' in a scientific, policy, or everyday context. Explain how the selective exposure leads to error in this particular context.
| |
| # (5 min) Give two or more examples of ''biased assimilation'' in a scientific, policy, or everyday context. Explain how the selective exposure leads to error in this particular context.
| |
| | |
| <!-- == Overflow ==
| |
| | |
| <div class="toccolours mw-collapsible mw-collapsed" style="overflow:auto;">
| |
| <div style="font-weight:bold;line-height:1.6;">Extra content that's not currently part of the official lesson plan.</div>
| |
| <div class="mw-collapsible-content">
| |
| | |
| === Wason Selection Task ===
| |
| | |
| [https://www.youtube.com/watch?v=t7NE7apn-PA Demo video] (from beginning to ~3:10, then ~5:30 to ~6:30)
| |
| | |
| This is a classic demonstration of humans' performance in deductive reasoning. In the context of confirmation bias, the predominance of a certain wrong answer demonstrates our tendency to seek to confirm a hypothesis, rather than find ways to potentially falsify it. {{Caution|Reveal this takeaway only after the following activity has been done.}}
| |
| | |
| Make visual slides for this activity for best effect.
| |
| | |
| ==== Card Puzzle ====
| |
| | |
| * Cards have a letter on one side and a number on the other.
| |
| * '''Rule:''' If a card has a vowel on one side, it has an even number on the other side.
| |
| * Which card(s) would you turn over: E, 7, 4, M?
| |
| * You may give the following options for a quick show of hands:
| |
| *# E only
| |
| *# E and 4 only
| |
| *# E and 7 only
| |
| *# E, 7, and 4 only
| |
| *# E, 7, 4, and M
| |
| {{Answer|E and 7, although most people choose to turn over E and 4, or just E.}} {{Caution|Don't reveal the answer until after the bartender question below.}}
| |
| | |
| ==== Bartender ====
| |
| | |
| * Cards have an age on one side and what the person is drinking on the other side.
| |
| * '''Rule:''' If someone is drinking alcohol, they must be over 18.
| |
| * Which card(s) would you turn over: beer, 16-yo, 25-yo, apple juice?
| |
| * You may give the following options:
| |
| *# Beer
| |
| *# Beer and 16-yo
| |
| *# Beer and 25-yo
| |
| *# Beer, 16-yo, 25-yo
| |
| *# Beer, 16-yo, 25-yo, and apple juice
| |
| {{Answer|Beer and 16-yo, and most people should get this right.}}
| |
| | |
| Now reveal the answers to the above two questions. The tendency to choose E and 4 (specifically 4) for the first task shows our tendency to confirm the rule, rather than seek out ways to falsify it. Humans are much better at the second task, which is logically identical to the first, because it is easier to test membership in a category for which we have clear intuitions. Initially Cosmides and Tooby, evolutionary psychologists, thought that it was because humans evolved a "rule violating" module in the brain, but in fact it turns out we're pretty good at this task if there's any intuitive category to look for, e.g. "gliders" = "things that fly without engines" and "underage drinkers" = "people under 21 drinking alcohol."
| |
| | |
| === Changemaker ===
| |
| | |
| #{{Changemaker|What relevance does confirmation bias play in Vice Provost García Bedolla's Fireside Chat? }}{{Changemaker|(Hint. The following passage can help you if you are stuck): "I think, lifting up, what I was just saying about trusting your gut? I think often we, when we are in spaces, it's easy to go along with what's dominant, or it's easy to do what everyone else is doing, or it's easier to be part of the crowd. And I think often we see opportunities or we see things that don't feel quite right and doing something about that or going down that road feels really risky. But what I like to tell my students is any trend, if you can see it it's over. And so in fact, building something new may be very difficult, but that's where the real payoff comes. And to just trust that you, we stand on the shoulders of giants and that you are here representing lots of people like you. And so, if you see something wrong or see a place where change needs to happen or see something that you think is important to either change or do or influence, that you trust yourself, that you're right in that analysis and that you are willing to take the risk."
| |
| }}
| |
| | |
| </div></div> -->
| |
|
| |
|
| {{NavCard|prev=9.2 Biases|next=10.2 Blinding}} | | [Question 1] |
| [[Category:Lesson plans]] | | {{BoxCaution|[Possible misconception that may need to be corrected and clarified.]|small=right}} |
| | {{BoxAnswer|[Intended answer to the above question.]}} |
| | {{Line}} |
| | [Question 2] |
| | {{BoxCaution|[Possible misconception that may need to be corrected and clarified.]|small=right}} |
| | {{BoxAnswer|[Intended answer to the above question.]}}{{NavCard|prev=[Previous Lesson]|next=[Next Lesson]}} |
| | <includeonly>[[Category:Lesson plans]]</includeonly> |
