The Crowd and Scientific Research

A new crowdsourcing project has been launched by the The Darwin Correspondence Project and CARET in order to gather more information for an experiment launched by Charles Darwin more than 140 years ago.  Back in 1868, Darwin began inviting subjects to view a series of photographs of a human face making various mechanically-manipulated expressions and then invited those respondents to classify those expressions as a particular emotion.  The goal of the experiment was to find out what was at the core of human expressivity and emotions. Are such things cultural? Are they innate? How do we recognize emotions? Now the folks at The Darwin Correspondence Project and CARET are using the same research model, but asking a far broader audience (the original sample size only included 24 respondents) to add to the database of information.

When I was in a college psychology course, it was suggested that there are as few as seven discrete human emotions. In Darwin’s experiment, they didn’t make respondents pick from a list of emotions (seven, or otherwise), the crowd can simply fill in the blank. However, with this crowdsourced version of the experiment, the text field pre-populates with a variety of other responses from previous survey-takers based on the few letters that you type in. You can essentially see every response as you’re selecting your own. As I went through the rather fun (and sometimes creepy) experiment, I wondered if this difference might impact the data in interesting ways.

Which led me to a recent study analyzing the data quality of results gathered from online crowdsourced platforms (Mechanical Turk in specific). The report analyzes Mechanical Turk’s population of participants and says that is certainly at least as representative as any focus group’s population might be, but that one potential disadvantage to responses from the crowd is that unsupervised subjects (read: most subjects that are responding via the internet, really) tend to be less rigorous in their responses and less attentive in general. Still, all things considered, I think that Darwin would appreciate the wealth of information now available from the crowd.

What other ways can crowdsourcing skew an experiment? What are the advantages to research via the internet? What did you think of Darwin’s images?


Leave a Reply