5.1 False Positives and Negatives
From Sense & Sensibility & Science
More actions
Learning Goals
[Link to PlayPosit]
[Link to instructional video]
More details: [link to topic on SSS website]
After this lesson, students should
- Understand that there is always the inevitability of a trade-off—for a given test, one can reduce the risk of false positives by increasing the risk of false negatives, and vice versa. You make this trade-off when you determine what the threshold is (e.g. convict someone if you’re 99% or 70% certain that they’re guilty).
- Understand that whether false positive or negative is preferable depends on a human values judgement on the consequences of either: (e.g. “innocent people shouldn’t be in jail”) and risks (e.g dangerous people committing more crimes).
Definitions
- False Positive/Type I Errors
- A test yields a positive result, but in fact the condition is not present (e.g. your test says you have COVID, but you don’t)
- False Negative/Type II Errors
- A test yields a negative result, but in fact the condition is present. (e.g. your test says you don’t have COVID, but you do)
- Threshold
- The cut-off quantity chosen for an indicator variable to decide when a given instance will be supposed positive vs. negative.
Unless the indicator variable is a perfect indicator of the underlying condition, there will be some probability of false positives and/or false negatives for any given threshold. Choosing a threshold is thus partly a matter of deciding one’s relative tolerance for false positives and false negatives; a higher threshold will lead to more false negatives, a lower threshold to more false positives.
- The cut-off quantity chosen for an indicator variable to decide when a given instance will be supposed positive vs. negative.
Examples
- Incoming Asteroid: "The oncoming asteroid has only a 1% chance of hitting Earth. But if it does, life on Earth will be destroyed. It'll be expensive to stop the asteroid, but the risk is bad enough it's worth it."
- Cancer Treatment: "I asked them to imagine that they faced a choice between two types of radiation therapy for early-stage breast cancer. The first treatment would leave them with a 15% chance of local recurrence and a 10% chance of moderate or severe breast fibrosis. The second treatment would leave them with only an 8% chance of local recurrence but a 30% chance of moderate or severe fibrosis. The radiation oncologists raised their hands in almost equal numbers for the two treatments. Some believed the higher risk of fibrosis was unacceptable, given the treatability of most local recurrences, whereas others believed the trauma of recurrence outweighed the discomfort of fibrosis. But sometimes physicians' values differ in important ways from those of many patients. When such value judgments are incorporated into professional treatment guidelines, without any explicit acknowledgment that a reasonable patient might choose an alternative course of treatment, they take potential choices away from patients." source
Common Misconceptions
- There have been many people who still caught the virus even after a "negative" test result. These tests are just unreliable.
- Most medical tests will inevitably produce false positive or false negative results at a certain rate. Although no conclusion can be totally accurate, it is possible to quantify the rate of such errors, so that the test result can be used to assess the probabilistic risks and odds of having a disease. This alludes to 6.1 Probabilistic Thinking.
Context
[Explanation of how the current topic into the larger context of the course by explaining how the previous topic(s) relate to the current topic and leaving cliffhangers for future topics where relevant throughout the lesson]
Before
- X.X Lesson Name
- Explanation.
- Itemized explanation.
After
- X.X Lesson Name
- Explanation.
- Itemized explanation.
Recommended Outline
Before Class
- [Any essential logistical things that need to be done for this class]
- Prepare a seating chart.
- Review PlayPosit and discussion questions and ask faculty, Gabriel, or Emlen any questions you have.
- (Optional) Prepare a presentation.
During Class
- (5 min) Come up with some fun way to assign the roles of spokesperson and notetaker (e.g. earliest birthday in the year, lives furthest from campus). Remind them of the responsibilities of these roles.
- ([#] min) [Description of some module from #Lesson Content]
- (5 min) Collect questions for plenary.
Lesson Content
Clicker Question
- Question
- Option 1
- Options 2
Activity 1: Name
[Brief description of and motivation for the activity]
| Common misconceptions and any useful tricks, tips, guidelines, or other background |
Instructions
Discussion Questions
- Question 1
- Subquestion a
| Intended answer to the above question. |
| Possible misconception that may need to be corrected and clarified. |
- Subquestion b
Collect Questions for Plenary
(5 min) Collect remaining questions from the students for faculty in plenary (can be questions for clarification, extension, discussion, etc.), and add [ here].