12.1 Wisdom of Crowds and Herd Thinking

Explore ways that groups fall short of their optimal reasoning ability. There are better and worse ways to aggregate a group's knowledge.

The Lesson in Context

This lesson discusses when groups of people make better or worse judgments than individuals. It leads into the last part of the course, which revolves around group decision making, e.g. as a society.

2.2 Systematic and Statistical Uncertainty

• Individual judgments can deviate from the truth due to systematic bias and random fluctuation. Reducing shared bias and increasing sample size are ways to improve the effects of Wisdom of Crowds.

9.2 Biases

• Conformity to the group consensus, illustrated by the Asch experiment, and obedience to a perceived authority in a group, illustrated by the Milgram experiment, are psychological tendencies that affect group decision making, tending to increase herd thinking.

10.1 Confirmation Bias

• Confirmation bias can exacerbate other biases in group decision making, such as the motivation to make judgments that conform to the group consensus or agree with the opinion of the authority figure - again, increasing problematic herd thinking.

13.1 Denver Bullet Study

• When making group decisions using the Denver Bullet Study method, it is important to survey each expert (on matters of fact) and each stakeholder (on matters of value) individually and independently so as to reduce "herd thinking" effects.

13.2 Deliberative Polling

• In a somewhat opposite approach, deliberative polling encourages moderated discussion between all participants punctuated by dialogue with the relevant experts, before participants answer a poll to make informed decisions in society. Conformity with other participants as well as "obedience" to (in the sense of taking advice from) the expert panelists are essential features. The result tends to be a convergence of opinions towards moderation on divisive issues.

Takeaways

Learning GoalsDefinitionsExamplesCommon Misconceptions

After this lesson, students should

1. Not take for granted that consensus offers the best conclusions.
2. Take seriously (but not as absolute!) the consensus of a group which has reasoned about a question in a careful, appropriate way.
3. Take seriously (but not as absolute!) the average of a large group's independent estimates of a number, under appropriate conditions.
4. Identify shared biases in a given group which may increase the odds of problematic herd thinking rather than helpful wisdom of crowds.

Wisdom of Crowds

Sometimes groups make better judgments than individuals. This happens when:

• Judgments are genuinely independent, preventing herd thinking.
• Members of the group do not share the same biases.
• There are enough people in the group to balance out random biases or fluctuations (analogous to the need for an adequate sample size).
• Works especially well when estimating a quantity, where errors may be large but are not systematic.

Herd Thinking

Sometimes groups make worse judgments than individuals. This happens when:

• Judgments of individuals are influenced by the judgments of others, leading to groupthink and sometimes polarization.
• Members of the group share biases, which can be exaggerated by discussion and cannot be decreased by averaging judgments.

Meta-analysis

A statistical analysis that aggregates the results from many independent studies addressing the same question. Each individual study is expected to have some error independent of the other studies. By combining these results this aggregate statistical error can hopefully be eliminated.

Family Dynamics

Within a group like a family, the parents may have chosen each other partly due to shared opinions and beliefs, and taught these to their children. The experience of strong consensus and absence of dissent within the family can cause the members to become more confident and more extreme in their beliefs (herd thinking), even if these opinions are quite different from a broader consensus in society. Moreover, it can be difficult for members to break out, since disagreeing with the group can be seen as disloyal, unethical, defiant, and ungrateful.

---

Congressional Committees

In Congress, most committees include both Republicans and Democrats. Ideally, this leads to groups whose members have different biases, helping to prevent herd thinking and leading to more considered policy proposals.

---

Scientific Community

In science, the ideal is a large community of people with varied biases who pursue investigations independently, and share their findings and ideas periodically. These features help the enterprise of science take advantage of the virtues of wisdom of crowds.

Insofar as the members of the scientific community share the same biases - for example, that they are not demographically representative of humanity at large - this can reduce the efficacy. For this reason, representation in science is important not only for ethical reasons of equity, but also for epistemic reasons.

Everyone agrees, so it must be true.

Sometimes everyone in a group agrees due to shared biases, sometimes combined with a charismatic or dominating figure who has convinced everyone else. Consensus can be a good indicator of accuracy, especially when biases are distributed and independent judgments are included, but it isn't a guarantee, especially when the group is missing key information.

I trust my guess about how many pages are in this book more than the average of my guess with other people's guesses, because even if I don't have more information than other people, it's MY guess and you should stick with your own guess.

If the goal is accuracy, more information is generally better. When other people's guesses don't share biases, as is typical with numeric guesses, averages of many independent guesses tend to be more accurate than any given randomly selected guess. Of course, if one person is an expert in some way, it may be worth weighting their answer more highly or even trusting their expertise.

Useful Resources

Lecture VideoDiscussion SlidesHandouts and ActivitiesReadings and Assignments

Discussion Slides Template

The discussion slides for this lesson.

Estimates Survey Template

Google form for the wisdom of crowds estimates activity.

Estimates Spreadsheet Example

Google sheet for the wisdom of crowds estimates activity.

Reducing the Effects of Juror Bias

On jury bias and composition. Kaplan and Miller, 1978.

Effects of Victim Attractiveness, Care and Disfigurement on the Judgements of American and British Mock Jurors

On jury bias and composition. Kerr, Bull, MacCoun and Rathborn, 1985.

The Emergence of Extralegal Bias During Jury Deliberation

On jury bias and composition. MacCoun, 1990.

Bias in Jurors vs Bias in Juries: New Evidence from the SDS Perspective

On jury bias and composition. Kerr, Niedermeier and Kaplan, 1999.

The Social Psychology of the Wisdom of Crowds

Richard Larrick, Albert Mannes, and Jack Soll on the wisdom of crowds.

Recommended Outline

Before Class

Update the Google form in the warm-up polls.

During Class

5 Minutes	Introduce the lesson and go over the plan for the day. Make sure people have groups, spokespeople, etc.
30 Minutes	Have the class do the assorted warm-up polls.
40 Minutes	Walk the class through the various jury discussions.
5 Minutes	Do the group problem if time allows. This part is optional.

Lesson Content

Warm-up Polls

For this discussion section, students will start by filling out a Google form which asks for their best estimate of four quantities (should take 3-5 mins). The form should connect to a spreadsheet which shows how the class is or isn't aggregating live. Depending on the particular setup, this may take some spreadsheet finagling live during class. An example spreadsheet that was used in spring 2023 can be found below.

Estimates Survey Template

Google form for the wisdom of crowds estimates activity.

Estimates Spreadsheet Example

Google sheet for the wisdom of crowds estimates activity.

Wisdom of Crowds Estimates

This first poll demonstrates the wisdom of crowds and what happens when there's a systematic bias in people's judgments. It has the following four numerical questions.

1. What is the heaviest recorded weight for a feline as of 2013?

922 lbs

2. What was the average weight for American men in 1960?

166 lbs

3. What is the heaviest recorded weight for a rabbit as of 2013?

55 lbs

4. What was the average weight for American women in 2010?

163 lbs

Make sure to copy the Google form and provide it to the students with a link and/or QR code.

Make sure the students give their best guesses for each question without discussing with any classmates or looking any answers up. Don't reveal the answers until everyone's submitted their responses.

Famous Artist Poll

A painting made by... someone.

This poll demonstrates the power of herd thinking in amplifying shared biases. Students are shown the painting on the right and asked which of the following artists created it.

1. Claude Monet
2. Andy Warhol
3. Armand Guillaumin
4. Eugene Boudin

Note that the names are ordered this way so that Claude Monet gets asked before Armand Guillaumin. If the order is switched you may get an opposite effect where a slightly larger (or more confident) proportion of students voting for Armand Guillaumin inspires more students to herd with them. In this case, the power of expertise directing a group is demonstrated.

The poll is done in two rounds. In both cases, say the names in order and count how many students raise their hands for each one. Pause for a moment to give students a chance to get their hands up if they're slow (and to see what other people are doing).

1. Ask the students to make their best guess, but don't require students to raise their hand.
2. Force every student to make a guess. Tell the students that they all need to raise their hand for some choice even if they don't know the answer.

The correct answer is Armand Guillamin. However, it's likely the case that most people will pick Monet, because he is more widely known than the other options. This is an example of the negative outcome of herd thinking, where a group of people independently and collectively select the wrong answer because they all share the same prior bias (picking the famous painter). In the first case the bias only comes from people's tendency to pick familiar names. In the second, it may get amplified by seeing other more people raise their hands and then wanting to join in.

This activity often does not work as intended. Students usually understand what it's intending to convey, but it's hard to get them to fall victim to it within the classroom.

Activity 1

[Brief description of and motivation for the activity]

[Common misconception or thing to look out for.]

[Thing you really need to look out for!]

[Title]

[Useful tip, guideline, or other background.]

Instructions

[n] Minutes	[Activity.]
[n] Minutes	[Activity.]

Discussion Questions

[Question 1]

[Possible misconception that may need to be corrected and clarified.]

[Intended answer to the above question.]

---

[Question 2]

[Possible misconception that may need to be corrected and clarified.]

[Intended answer to the above question.]

---

[[[Previous Lesson]|⟨]]

[[[Previous Lesson]|Previous]]

[[[Previous Lesson]]]

[[[Next Lesson]|⟩]]

[[[Next Lesson]|Next]]

[[[Next Lesson]]]