12.2 Grill the Guest

Useful Links

• Three Column Overview of the Week
• Question Collection Google Form
• Lesson Slides (2023 Master)

Readings and Assignments

• [PlayPosit Video]
• Toddlers Use Speech Disfluencies to Predict Speakers' Referential Intentions (2023 Paper)

Learning Goals

After this lesson, students should

1. Be able to critically examine a piece of contemporary research and pose meaningful questions to gauge the validity and limitations of its results.
2. Be able to flexibly invoke and synthesize course concepts in a real world dialogue on science and society.

Context

This is the first of several lessons involving guests. We typically invite a social scientist to share a recent published study of theirs, to be read by students ahead of time and discussed in groups during section. Questions and comments generated during the section will then be collected, a few of which will be posed to the guest during the live plenary. Students are encouraged to base their questions on particular course concepts introduced thus far.

Recommended Outline

Before Class

• Someone invites a guest speaker at least one month in advance, consulting the whole teaching staff for ideas.
• Send the article by the guest to students several days in advance and remind them to read it before the discussion section. A briefly annotated version should be made to make it easier for students to skim it and pick out the important information.
• Prepare a seating chart.
• Review PlayPosit and discussion questions and ask faculty, Gabriel, or Emlen any questions you have.
• (Optional) Prepare a presentation.

During Class

• (7 min) Short review of the paper.
• (43 min) Go through the first round of discussion.
• (30 min) Go through the second round of discussion.

After Class

• Pose any remaining questions to the guest in writing and send a thank you email.
• Collect answers from notetakers for the forum / plenary.

Lesson Content

Paper Review

Do a short review of the paper. The students should have read the annotated version of it. But, it's likely that at least some students haven't fully grasped it yet.

Discussion Round 1

Not every student may have read or fully understood the paper. So, the first discussion should help ensure that everyone walks away with its main questions, methods, and ideas.

Instructions

In small groups, the students spend 15 minutes discussing all of the following questions and reaching a consensus on them. Then, regroup as a whole class and go through all the questions again. This should take four minutes per question.

Questions

1. What was the research question of this paper?
2. What was the main hypothesis of this paper? What was at least one competing (alternative) hypothesis?
3. What type of experiment(s)/study did this paper conduct?
4. What was the central conclusion of this paper? What were the sources of evidence to support the conclusion in this paper?
5. On a scale of one to ten, how strong do you think the central conclusion in this paper was supported? Did you find any weaknesses or concerns about this paper?
6. What aspects of this paper did you find particularly important or exciting?
7. How do you feel the findings of this paper could impact public policy decisions or communications with the public?

List of Course Concepts

1. The design, purpose, and evaluation of models
2. Senses, instrumentation, and the role of interactive exploration
3. Sources of systematic and statistical uncertainty, their consequences, and how we reduce them
4. Credence levels and their calibration
5. Signal and noise
6. The human tendency to find patterns in random noise
7. The consequences of false positives and negatives
8. Scientific optimism
9. Incremental progress
10. Causation in terms of intervention and randomized controlled trials
11. Hill's criteria and other non-RCT suggestions of causation
12. Singular and general causation
13. Emergent forms of causation and how simple rules/effects can lead to large scale behavior
14. Orders of understanding
15. Fermi problems
16. Representativeness heuristic, conjunction fallacy, and availability heuristics
17. Base rate neglect
18. Fundamental attribution error
19. Conformity, obedience, and authority biases
20. Confirmation biases
21. Blind analysis and pre-registration
22. Measurement of human groups
23. Wisdom of crowds
24. Herd thinking

Discussion Round 2

The second discussion is for the students to develop questions to ask the speaker.

Preparation

Assign numbers to each of the small groups. Generally, each group will be assigned two course concepts. Make sure that the groups and concepts are divided up so that each concept from the list is represented.

If they find applications of their assigned course concepts quickly, invite them to consider how other course concepts might be applied to the paper, whichever they want.

Instructions

1. (2 min) Split the students into twelve small groups and provide them with their concepts. The students need to think about/discuss how this course concept may or may not have been incorporated into the paper and its content.
2. (20 min) The students then need to draft a question asking about the incorporation (or lack thereof) for each of their course concepts.
3. (8 min) Each group selects a spokesperson to submit their questions to a google form.