Topics and lesson plans

These are all the lesson plans used in the full fourteen week version of the course that's run at UC Berkeley. These include lecture videos as well detailed content that can be used in smaller discussion sections.

List of Lesson Plans

Week 1: The Purpose and Limitations of Science

Week 2: What we can Measure

Week 3: Statements of Uncertainty

Week 4: Dealing with Noise

Week 5: Trade-offs and Progress

Week 6: Establishing Causation

Week 7: Types and Consequences of Causation

Week 8: Describing and Simplifying Reality

Week 9: Heuristics and Biases

Week 10: Catching and Countering Biases

Week 11: Science Gone Wrong

Week 12: Group Evaluations

Week 13: Better Group Decision Making

Week 14: The Outlook and Potential of Science

1.1 Introduction and When Is Science Relevant
- Democracy vs. epistocracy
- Facts vs. values
1.2 Shared Reality and Modeling
- Shared reality
- Raft vs. pyramid
- Evaluation of models
- Science vs. decree
- Scientific realism vs. anti-realism
- Operationalism, conventionalism, and realism
2.1 Senses and Instrumentation
- Validation of instruments through interactive exploration, triangulation with other instruments, and comparison with direct senses
2.2 Systematic and Statistical Uncertainty
- Measurement proxies as sources of systematic and statistical uncertainty
3.1 Probabilistic Reasoning & 3.2 Calibration of Credence Levels
- The value of partial and probabilistic information
- Words of estimative probability (probably, likely, definitely, etc.)
- [math]\displaystyle{ p }[/math]-values and statistical significance
- Error bars and confidence intervals
- Strategies to improve calibration of credence levels (feedback, AOT, growth mindset, etc.)
4.1 Signal and Noise & 4.2 Finding Patterns in Random Noise
- Signal-to-noise ratio
- [math]\displaystyle{ p }[/math]-hacking
- Look elsewhere effect
- Gambler's fallacy
- Hot-hand fallacy
- File drawer effect
- HARKing (hypothesizing after results are known)
- Effect size (as distinct from statistical significance)
5.1 False Positives and Negatives
- Thresholds between positive and negative detections
- Trade offs between false positives and negatives
5.2 Scientific Optimism
- Iterative progress
6.1 Correlation and Causation
- Causation as correlation under intervention
- Randomized controlled trials
- Different directions of causation
- Spurious correlations
6.2 Hill's Criteria (Causation in the Messy Real World)
- Natural experiments
- Hill's criteria for causation
- Causal networks
7.1 Causation, Blame, and Policy
- Singular and general causation
- Acts of omission vs. commission (and the omission bias)
- Status quo bias
7.2 Emergent Phenomena
- Global effects that arise through the interaction of small pieces (rather than general causation)
- Explanation at different scales
- Scientific reductionism
8.1 Orders of Understanding
- Orders of magnitude
- Multiple causes of comparable importance
- Orders of importance of causes
- Refinement of models using higher order descriptions
- Scale of impact of policies
8.2 Fermi Problems
- Fermi problems
9.1 Heuristics
- Base rate neglect
- Representativeness heuristic and conjunction fallacy
- Availability heuristic
- Bounded rationality
9.2 Biases
- Fundamental attribution error
- Conformity
- Obedience
- Temporal discounting
10.1 Confirmation Bias
- Selective exposure
- Biased assimilation
- Strategies for reducing confirmation bias (AOT, etc.)
10.2 Blinding
- Techniques for blind analysis
- Preregistration
- Registered replication
- Adversarial collaboration
- Peer review
11.1 Pathological Science
- The spectrum of poor research
- Langmuir's pathological science indicators
11.2 When Is Science Suspect
- The validity and reliability of social science metrics
- External validity
- The difficulty of creating and applying social science metrics in cross-cultural contexts
- How researchers' biases and backgrounds shape their research agendas
12.1 Wisdom of Crowds and Herd Thinking
- The Wisdom of Crowds effect removing independent biases via error cancellation
- How humans can engage in herd thinking to cluster around biased answers to group questions
13.1 Denver Bullet Study
- The Denver Bullet Study process of integrating factual analysis from experts with value ratings from different stakeholders
13.2 Deliberative Polling
- The deliberative polling process of having the general public make decisions as though they were experts
14.1 Scenario Planning
- The Scenario Planning process of predicting and responding to possible futures
- What makes good drivers to use in Scenario Planning

Anatomy of a Lesson Plan

Work in Progress

This section is currently being written or undergoing major revision.

Each lesson plan has the following components:

Useful Links
Readings and Assignments
Learning Goals
Context
Recommended Outline
Lesson Content
Overflow (this section may sometimes be missing)

Useful Links

This section contains links to the prerecorded lecture videos (called "PlayPosits") that the students watch prior to discussion sections (called "labscussions"), the slides to use during labscussion, a "three column" overview of the week, and a link to Boxthe "website page". There may also be other links to handouts, videos, activities, papers, or other things relevant to the lesson.

Three Column Overview of the Week

This is a bullet point summary of everything in the PlayPosit videos, labscussion sections, and plenary session for the week. It's mainly intended for faculty to get a sense of what's being taught, but it may be helpful to others as well.

Lesson Slides

See the section on the anatomy of labscussion slides for more info.

These are the slides for this lesson. For most lessons you can use these slides directly as is. For a handful of lessons there will be different versions of the slides for each section. In those cases, separate links for each section will be made available. If you make any changes to the lesson slides, please duplicate the slides and leave them in the same Google Drive folder with your section number(s) in parentheses at the end of the name. This format is important for record keeping purposes.

A student-viewable version of the lesson slides will be linked in the syllabus after each lesson as many of the activities rely on the students being unaware of certain elements of them.

Website Page

You should generally ignoreBox the website page link. It's kept for historical reasons.

This website contain out of date content and may not necessarily align with the versions of the lessons presented on this wiki. The wiki is the most up to date version of the content. However, the information Boxon the website may still be a useful reference if you want a slightly different perspective on the material.

Readings and Assignments

This section has links to relevant readings and videos that can be assigned to students. In most cases, this also includes a supplemental lecture video presented by one or more professors that have previously taught the course. In the UC Berkeley version of the course, the students watch these videos prior to section.

Learning Goals

This is what we intend the students to take away from the lesson. The learning goals begin with the abstract ideas we want to walk away with followed by any specific definitions we would like the students to learn. These learning goals will be made available to the students and are included in the slides for every lesson. This is the content that is in scope for the quizzes and final exam.

This section also has "examples" and "common misconceptions" the the students often have or end up with. Not all the examples come up within the lessons normally. Nevertheless, you are welcome to use them. You are welcome to add any other examples you stumble upon or misconceptions you find your students falling victim to.

Context

This section contains general information about how this lesson fits into the broader picture of the course as well as which other lessons tie into it.

Recommended Outline

This includes any steps that the instructors or students need to take prior to labscussion section as well as any wrap-up that needs to be done after. The "during class" part of this section has recommended minute-by-minute timelines so that you get through the content. Feel free to adjust this timeline to fit the needs of your section.

Lesson Content

This is the actual meat of the lesson. It has detailed explanations for each activity in the order that they're done in labscussion.

Anything in a blue box is a useful note or tip for the instructor.

Anything in an orange box is a warning or note of caution for the instructor.

Anything in a green box or written in text of this color is the answer to some question.

Overflow

This section contains activities and discussions that are not currently being used but that we want to maintain a record of regardless. Many of them were used in previous iterations of the course.

Anatomy of Labscussion Slides

The slides for each labscussion are color coded. The "gold-blue" slides represent the "introduction" slides and typically have content you can quickly brush through.

The remaining slides cycle through "red", "blue", and "green" slides. These always progress in the same order. A change in color corresponds is a visual cue that shows the labscussion is moving on to a new activity. All the slides for the same activity are of the same color. The colors keep cycling if there are more than three activities in a lesson. The section's agenda is also shown whenever there's a transition to a new activity.

Introductions

Every slide deck begins with a title card that introduces the lesson. Followed by the lesson's agenda, learning goals, and definitions. Note that the lessons are (typically) not designed to have much time for review at the start. So, you don't have to spend time going over the learning goals and definitions in each section. They're mainly included so that you can flip back to them during class if it comes up and so that the students can reference them easily after class.

⟨

⟩