11.2 When Is Science Suspect

As scientific studies inform societal views and policy, it is important to recognize how science, particularly when groups in power study groups out of power, has been misused to perpetuate injustice. With the help of historical examples, we aim to instill in students a sense of social responsibility as future scientists and decision makers.

The Lesson in Context

After discussing how science may go wrong in its form in 11.1 Pathological Science, we now discuss how science may go wrong in its societal outcome. Through a discussion activity, students will appreciate the difficulty in measuring humans in a way that is absolutely free of confounds that have discriminatory implications, and thus learn the importance of social responsibility as a scientist and including a more representative array of voices in selecting scientific questions and designing measures.

Earlier Lessons

1.1 Introduction and When Is Science Relevant

• If science is to be used in societal decision making, scientists must be mindful of its potential to cause harm.

1.2 Shared Reality and Modeling

• When studying humans, it is often necessary to define terms operationally, such as "what behaviors constitute altruism." Such operationalizations may be a crude measure of a more realist human trait, if one even exists. Cultural and personal biases may easily cause such definitions to favor the researchers' own group at the expense of others.

10.1 Confirmation Bias

• It is tempting, though improper, for scientists to only publish results in a way that confirms the predominant belief. When it comes to human groups, this belief may be a mere stereotype.

Takeaways

Learning GoalsDefinitionsExamplesCommon MisconceptionsExpanded Learning Goals

After this lesson, students should

1. Recognize the potential to abuse science for social and political ends.
2. Show heightened caution in situations in which science involves the study of human groups and subsequent validation of societal power structures.
3. Recognize that you yourself are always involved in some social dynamic that may be relevant to the assessment of any particular study of human groups.

While it may be easy to spot cases where science has been used intentionally to validate preexisting societal power structures, the goal of this lesson is to bring attention to the possibility that one may inadvertently use science in such a way, through negligence or even with the best intentions, especially when it comes to the study of human groups.

Just World Fallacy

The tendency to believe that outcomes are deserved and existing social structures are justified.

Reliability

The extent to which some metric is consistently measurable.

Validity

The extent to which some metric or scientific concept reflects some real external thing.

External Validity

The extent to which the result of an RCT generalizes to the world at large. There are several reasons results might not generalize.

• Longevity of Effect

Lab studies tend to measure dependent variables immediately after the intervention, but often inferences are desired for long-term effects.

• Disruption Effect

Awareness of being in an experiment.

• Novelty Effect

Novelty of the context may alter the effects of the experiment.

The Flynn Effect

Scores from IQ tests are typically normalized such as to set the average score to be 100. But, throughout the 20th century, there was a consistent increase in how well people performed on IQ tests. This means that when newer test subjects take older tests, the average scores tended to be greater than 100.

Cognitive Capabilities by Ethnicity and Gender

For a long time many scientists asserted that women and ethnic minorities were less intelligent than men of European descent, using studies of cognitive performance normalized to the skills such men spent their time developing. This claim was used to justify withholding educational and vocational opportunities for both women and minorities, as well as institutional and legal power.

Genetic Description of Pellagra

In 1916, Charles Davenport asserted that the disease pellagra was genetic (racial). However, it is actually caused due to nutritional factors (a lack of vitamin B3).

The Hereditary Factor In Pellagra

The introduction from Davenport's book describing his racial theory of pellagra.

Representation and External Validity

Historically, psychological or medical studies are often performed on predominantly white, wealthy, young males, with conclusions that are nevertheless generalized to the culturally and genetically diverse global population. For example, while blacks and Latinos make up 30% of the US population, they represent only 6% of participants in federally funded studies. While the studies themselves may not be discriminatory explicitly or implicitly, the differing validity of their application to diverse populations may be a source of furthered inequality.

Diversity in Clinical and Biomedical Research: A Promise Yet to Be Fulfilled

A paper on this phenomenon.

Exemplary Quotes

“The data described in The Bell Curve shows that Black students perform more poorly on IQ tests than White students. But historically, tests like that have been used to justify existing power structures and racial oppression, so maybe we should think about that more carefully before we interpret it to mean that White students are smarter. The IQ tests were written by Whites, for students who had grown up in similar environments. Maybe there are cultural biases. And hang on, there's a lot of vocabulary on those tests; that requires education, and we know that there are systemic racial inequalities in the education system. That by itself could explain the difference.”

Only a racist scientist could produce a racially discriminatory study.

As an example, a perfectly well-intentioned scientist studying the level of altruism in people across ethnic groups may come up with an operational definition of altruism that is based on their own cultural understanding of the term and fails to include alternative expressions of altruism in other cultures.

After this lesson, students should

1. Attitudes
   1. Recognize the potential to abuse science for social and political ends.
   2. Show heightened caution in situations in which science involves the study of human groups and subsequent validation of societal power structures.
   3. Recognize that you are yourself always implicated in some social dynamic or other that may be relevant to the assessment of any particular study of human groups.
2. Concept Acquisition
   1. Historically, science has been abused for social and political ends; this is why we should show heightened caution (using all the tools of the course) whenever science is used to evaluate particular groups of people.
   2. The abuse of science for social and political ends is exacerbated by confirmation bias and badging.
   3. The abuse of science for social and political ends is particularly insidious when it involves the study of human groups and subsequent validation of societal power structures (typically by purporting to establish that some group(s) have lesser capacities than others).
   4. Just World Fallacy: The tendency to believe that outcomes are deserved and existing social structures are justified.
3. Concept Application
   1. Identify examples in which there exists the potential for abuse of science for social and political ends.
   2. Explain how confirmation biases and badging can exacerbate the abuse of science for social and political ends in particular cases.
   3. Explain why the potential to abuse science for social and political ends is particularly insidious when it involves the study of human groups and subsequent validation of societal power structures, using a real example.
   4. Explain how the validity of human classifications (e.g., race, gender) can be problematic and contribute to such abuses (e.g., because there isn't any there, there are a number of things there, there is one thing there but it doesn't have the significance you take it to).
   5. Provide historical examples of the abuse of science for social and political ends.
   6. Provide examples of cases in which the study of human groups is legitimate and beneficial.
   7. Distinguish legitimate inquiry into human groups from bad science.

Additional Content

You must be logged in to see this content.

---

⟨

All lesson plans

⟨

Previous

11.1 Pathological Science

⟩

Next

12.1 Wisdom of Crowds and Herd Thinking