I’m enjoying The Second Machine Age by Erik Brynjolfsson and Andrew McAfee, and will review it soon. Meanwhile, though, I was struck by their discussion of the effectiveness of crowdsourcing for problem-solving or innovation. They give examples such as InnoCentive and Kaggle. I was particularly struck by this comment:
“Another interesting fact is that the majority of Kaggle contests are won by people who are marginal to the domain of the challenge.”
For instance, the best forecasts of hospital readmission rates came from people not involved in healthcare, while a contest to develop a computerised system for grading student essays was won by someone whose only expertise was a free online AI course offered by Stanford.
This reminded me a bit of Philip Tetlock’s debunking of professional expertise, but more of Scott Page’s completely brilliant book The Difference (or its follow-up, Diversity and Complexity), about the value of diversity. Combining ideas that are ‘far apart’ in some way creates new ideas and insights that are more creative and innovative. Outsiders, the people in the margins, are exactly those who bring new ways of thinking.
Crowds are wise when they are composed of people with different experiences and views. If they are all the same, you get group-think and herding.