12.2 Grill the Guest
From Sense & Sensibility & Science
More actions
Useful Links
- Three Column Overview of the Week
- Question Collection Google Form Template
Google form link must be updated every year. - Question Card Template
- Lesson Slides (2023 Master)
Readings and Assignments
- [PlayPosit Video]
Add PlayPosit video if/when it exists. - Toddlers Use Speech Disfluencies to Predict Speakers' Referential Intentions (2023 Paper)
Paper must be updated every year.
Learning Goals
After this lesson, students should
- Be able to critically examine a piece of contemporary research and pose meaningful questions to gauge the validity and limitations of its results.
- Be able to flexibly invoke and synthesize course concepts in a real world dialogue on science and society.
Context
This is the first of several lessons involving guests. We typically invite a social scientist to share a recent published study of theirs, to be read by students ahead of time and discussed in groups during section. Questions and comments generated during the section will then be collected, a few of which will be posed to the guest during the live plenary. Students are encouraged to base their questions on particular course concepts introduced thus far.
| We have a different guest and paper every year. So, the substance of this lesson can change quite substantially. |
| Blind analysis is a common and easy topic to criticize on, however the students should be reminded that it is not yet widespread in many fields of science, and perfectly valid scientific results can still be obtained without it. |
Recommended Outline
Before Class
- Someone invites a guest speaker at least one month in advance, consulting the whole teaching staff for ideas.
- Send the article by the guest to students several days in advance and remind them to read it before the discussion section. A briefly annotated version should be made to make it easier for students to skim it and pick out the important information.
- Prepare a seating chart.
- Review PlayPosit and discussion questions and ask faculty, Gabriel, or Emlen any questions you have.
- (Optional) Prepare a presentation.
During Class
- (7 min) Short review of the paper.
- (43 min) Go through the first round of discussion.
- (30 min) Go through the second round of discussion.
After Class
- Pose any remaining questions to the guest in writing and send a thank you email.
- Collect answers from notetakers for the forum / plenary.
Lesson Content
Paper Review
Do a short review of the paper. The students should have read the annotated version of it. But, it's likely that at least some students haven't fully grasped it yet.
Discussion Round 1
Have the students discuss and come up with answers to the following questions in small groups:
- What was the motivation for this study?
2023 Answer: - Prior research with adults suggests that disfluencies help facilitate sentence comprehension by signalling the presence of something new to the discourse.
- Wanted to know if toddlers did something similar and if it plays any role in their language development.
- What are the hypotheses of this study?
2023 Answer: - Whether children use indicators of processing difficulties in order to anticipate likely referents for an upcoming noun.
- What experiment(s) did the researchers perform to test these hypotheses?
2023 Answer: - Two experiments main experiments involving toddlers of different ages.
- First: 2 years 4 months - 2 years 8 months.
- Second: 1 year 4 months - 2 years 2 months. (Done in two groups.)
- The children were seated on their parents' lap while having their eyes tracked.
- Participants were presented objects that were either novel or familiar with accompanying fluent or disfluent audio. The experiment tested how much attention the toddlers paid to the objects given the audio.
- Two experiments main experiments involving toddlers of different ages.
- What are the conclusions of this study?
2023 Answer: - That young children have learned that disfluencies contain information, attend to disfluencies in speech, and can make use of the information contained in disfluencies during comprehension in order to infer speaker intention.
Discussion Round 2
Have the students discuss and come up with answers to the following questions in small groups:
- What instruments were used?
2023 Answer: - Primarily a Tobii 1750 eye-tracker.
- The eye-tracker keeps track of what the toddlers are looking at. The children don't have to wear anything to be eye-tracked.
- It was calibrated by having the children fixate on a shrinking dot located successively at one of five different screen locations.
- What was measured by these instruments? What's the signal? What are some sources of noise?
2023 Answer: - The eye-tracker measured where the toddlers were looking on the screen.
- The software tracked the fraction of the time the child spent looking at the object after it was labelled.
- Signal: Where the toddler was looking.
- Noise: Anything that makes the tracker misidentify this.
- What is this used as a proxy for?
2023 Answer: - What the child was actually paying attention to.
- This, in turn, is used as a proxy for the toddler's language acquisition.
- Eye tracking can be used a proxy for many different things depending on the specifics of the experimental setup.
- It's not a perfect proxy. But, it's one of the few ways to get at what's going on in the heads of really young toddlers.
- What intervention was performed to demonstrate causation?
2023 Answer: - Participants were exposed to audio with and without disfluencies to see if their attention differed. The presence of the disfluency in the audio was the intervention.
- What are some confounding factors that the researchers controlled for? How did they control for them?
2023 Answer: - How interesting the objects and their labels were.
- Novel and known objects were paired by color/shape. Labels were paired to be have similar sorts of sounds that were as easy to learn.
- The length of the audio given the disfluency.
- An additional “look…” was added to the third presentation of the object in both fluent and disfluent audio to ensure the toddlers were prompted at the same amount of time before the window of interest.
- The use of “the” before disfluencies in the real world.
- Prior work shows that 81% of the time, “thee” is followed by a suspension of speech.
- Disfluencies were preceded by “thee.”
- Cues by the parents.
- Parents wore headphones playing music to mask auditory stimuli.
- They were instructed to keep their gaze downwards during the experiment.
- How interesting the objects and their labels were.
- What are some confounding factors that the researchers might not have controlled for?
- What was at least one alternative hypothesis for the study's findings?
Lesson Content Overflow
Discussion Round 1
Not every student may have read or fully understood the paper. So, the first discussion should help ensure that everyone walks away with its main questions, methods, and ideas.
Instructions
In small groups, the students spend 15 minutes discussing all of the following questions and reaching a consensus on them. Then, regroup as a whole class and go through all the questions again. This should take four minutes per question.
Questions
- What was the research question of this paper?
- What was the main hypothesis of this paper? What was at least one competing (alternative) hypothesis?
- What type of experiment(s)/study did this paper conduct?
- What was the central conclusion of this paper? What were the sources of evidence to support the conclusion in this paper?
- On a scale of one to ten, how strong do you think the central conclusion in this paper was supported? Did you find any weaknesses or concerns about this paper?
- What aspects of this paper did you find particularly important or exciting?
- How do you feel the findings of this paper could impact public policy decisions or communications with the public?
Most papers we consider have potential public policy implications. If, for some reason, the paper doesn't, then this question can be ignored.
List of Course Concepts
| Update list of course concepts. |
- The design, purpose, and evaluation of models
- Senses, instrumentation, and the role of interactive exploration
- Sources of systematic and statistical uncertainty, their consequences, and how we reduce them
- Credence levels and their calibration
- Signal and noise
- The human tendency to find patterns in random noise
- The consequences of false positives and negatives
- Scientific optimism
- Incremental progress
- Causation in terms of intervention and randomized controlled trials
- Hill's criteria and other non-RCT suggestions of causation
- Singular and general causation
- Emergent forms of causation and how simple rules/effects can lead to large scale behavior
- Orders of understanding
- Fermi problems
- Representativeness heuristic, conjunction fallacy, and availability heuristics
- Base rate neglect
- Fundamental attribution error
- Conformity, obedience, and authority biases
- Confirmation biases
- Blind analysis and pre-registration
- Measurement of human groups
- Wisdom of crowds
- Herd thinking
Discussion Round 2
The second discussion is for the students to develop questions to ask the speaker.
Preparation
Assign numbers to each of the small groups. Generally, each group will be assigned two course concepts. Make sure that the groups and concepts are divided up so that each concept from the list is represented.
If they find applications of their assigned course concepts quickly, invite them to consider how other course concepts might be applied to the paper, whichever they want.
Instructions
- (2 min) Split the students into twelve small groups and provide them with their concepts. The students need to think about/discuss how this course concept may or may not have been incorporated into the paper and its content.
- (20 min) The students then need to draft a question asking about the incorporation (or lack thereof) for each of their course concepts.
If it is difficult to relate to the concept, encourage the students to try to come up with a hypothetical situation where a concept could have related to the study. They can ask about whether such hypothetical scenarios have come up. - (8 min) Each group selects a spokesperson to submit their questions to a google form.
Google form link must be updated every year.