The Wisdom of Crowds
Matt Paulson
It only took me six years as a crowdsourcing practitioner, but I finally got around to reading “The Wisdom of Crowds,” one of the flagship books in our industry. It provided for powerful reflection on what we’re doing right, and perhaps more importantly, wrong.
In the book, author James Surowiecki builds the case for the efficacy of “intelligent groups,” or “…under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them.”
He opens the book with an example most of us know: the crowd that guessed the weight of an ox at a country fair within a pound. If you’re not familiar with it, here are the CliffsNotes: British scientist Sir Francis Galton went to a country fair in 1906. There was a contest where attendees could guess the weight of an ox. At the end of the day, Galton collected the guesses, and while no one got it exactly right, when he averaged all 787 of them, the mean guess was 1,197 pounds; the ox weighed 1,198 pounds.
He then goes into a variety of others — how polling the crowd in “Who Wants to Be a Millionaire” resulted in them getting the answer right 91% of the time, how Iowa betting markets are one of the most accurate predictors of who will win the next Presidential election, how sports gamblers can predict the outcome of a game, and even the stock market.
The premise of the book is not that crowds are always smarter than the individuals in them. It’s that, on average, they’ll give you a more accurate result more often than any one individual. However, there are five conditions that must be satisfied for this to be true.
- Cognitive diversity and diversity of opinion: The crowd must be made up of individuals with a diverse array of opinions and cognitive abilities.
- Independence: Individuals within the crowd must be able to make their own decision without any influence from the crowd or individuals within it.
- Decentralization: Individuals must be able to draw on local knowledge to inform their own choices.
- Aggregation: There must be a way to aggregate the individual choices into a collective decision.
- Trust: Individuals must trust that the collective group will be fair.
While reading, I was reflecting on our own software, specifically which stages satisfy all five of these criteria. What immediately stood out to me was that the Ideate stage, which is by far the most frequently used stage by our customers, actually doesn’t, specifically the voting portion. Open voting doesn’t satisfy the “Independence” criterion because users are influenced by others. That being said, ideation is obviously integral in the process of crowdsourcing; you need ideas to solve problems. Open voting is also a good filter when looking at the popularity of an idea among your crowd. However, what this book underscored is that an ideation stage alone can not bring you to the best collective decision, which, in the IdeaScale context, is essentially the best ideas to implement.
Fortunately, we have you covered here. There are other stage types that satisfy all five criteria. In other words, once you have your ideas; you can build out a funnel that will enable you to continue to leverage the crowds’ wisdom to arrive at the best collective decision.
One of the most used stage types is Review, i.e. ReviewScale or Assessment. This satisfies all five. (For the sake of argument, and space, we’ll ignore 3 and 5, since I’m going to assume your crowd is engaging online and trusts your program.) The key here is #1. Most customers assume that only experts should evaluate ideas. What “The Wisdom of Crowds” opines is that even your evaluator crowd should be diverse, both cognitively and of opinion. In other words, don’t only leverage your 1-5 SMEs. Engage hundreds in your audience and let the software aggregate their results.
Less frequently used are Pairwise and Estimate. However, these are a few of the most powerful ways to go from an unmanageable mass of ideation data into a more actionable set of ideas based on crowd wisdom. Consider these types as you build out your next funnel. Better yet, get in touch with your Innovation Strategist to discuss more. You can find more info on Pairwise here, and for a cool presentation on how to leverage the crowd to arrive at the potential ROI of ideas, check out our group’s from this year’s Open Nation.