10.2 Blinding

Learning Goals

[Link to PlayPosit]

[Link to instructional video]

More details

After this lesson, students should

1. Explain why blind analysis might be needed, by explaining the errors that might arise in its absence.
2. Recognize when blind analysis is being used and explain what function it serves. Identify situations and decisions that would call for blind analysis.
3. Be able to evaluate techniques (e.g., registered replication, adversarial collaboration, peer review)
   1. for ability to address confirmation bias, and
   2. in comparison to blind analysis.
4. Propose how to use blind analysis for simple studies.

Definitions

• Preregistration

  A research group publicly commits to a specific set of methods and analyses before they conduct their research.

• Registered replication

  One or more research groups commit to a specific set of methods and procedures to verify the result of an earlier work (typically with the input of the original research team). Results are publicized regardless of outcome.

• Adversarial collaboration

  Scientists with opposing views agree to all the details of how data should be gathered and analyzed before any of the results are known.

• Peer review

  New results are evaluated by other experts in the same field to determine whether they are valid. This only reduces confirmation bias if reviewers don’t share the same biases.

Examples

• Fermilab's muon g−2 experiment performed highly precise measurements of the magnetic dipole moment of muons to test the theoretical predictions of the currently accepted model of elementary particles. Blinding is done by injecting a secret code into all of the data that would undergo analysis, so that the scientists involved would not make specific choices in the analysis in a way that makes the final value agree with the theoretical prediction. The secret code was kept in a physical locker, the opening of which was highly publicised in the announcement event. Once the data was "unscrambled", the result shows that there is indeed a sizeable deviation of the measured value from the theoretical prediction. (Short video about this process)
• The black hole image obtained by the Event Horizon Telescope (EHT) is constructed from incomplete partial images taken by a network of telescopes around the globe. Scientists on the EHT team had to "fill in" the missing parts of the image. The

Common Misconceptions

• Students may confuse blind analysis with a double blind experiment. The latter is used primarily in treatment testing in conjunction with a placebo, such that the patient is prevented from knowing whether they received the real treatment or placebo, and the doctor is also prevented from knowing this fact in order not to inadvertently reveal this fact to the patient through subtle signs. The former type of blinding applies to the analysis process once the data has been collected. In the case of treatment testing, blind analysis may be employed whether or not double blinding is.

Context

[Explanation of how the current topic into the larger context of the course by explaining how the previous topic(s) relate to the current topic and leaving cliffhangers for future topics where relevant throughout the lesson]

Before

X.X Lesson Name

Explanation.

• Itemized explanation.

After

X.X Lesson Name

Explanation.

• Itemized explanation.

Recommended Outline

Before Class

• [Any essential logistical things that need to be done for this class]
• Prepare a seating chart.
• Review PlayPosit and discussion questions and ask faculty, Gabriel, or Emlen any questions you have.
• (Optional) Prepare a presentation.

During Class

• (5 min) Come up with some fun way to assign the roles of spokesperson and notetaker (e.g. earliest birthday in the year, lives furthest from campus). Remind them of the responsibilities of these roles.
• ([#] min) [Description of some module from #Lesson Content]
• (5 min) Collect questions for plenary.

After Class

• [Any essential logistical things that need to be done as followup for this class]
• Collect answers from notetakers for the forum / plenary.

Lesson Content

Clicker Question

1. Question
   1. First option
   2. Second option
   3. Third option
   4. Fourth option

Tube Measurement

Video link: https://youtu.be/RO4-wA8-Bhw

[Brief description of and motivation for the activity]

Discussion Questions

1. Question 1
   1. Subquestion a

   2. Subquestion b

Collect Questions for Plenary

(5 min) Collect remaining questions from the students for faculty in plenary (can be questions for clarification, extension, discussion, etc.), and add [ here].