Well, it turns out even psychologists have long been suspicious of them.
After all, most experiments use undergraduates as their guinea pigs. And 18- to 21-year-olds are hardly representative of the rest of the population.
Plus, they're problematic for other reasons: As Dr. Joseph Henrich and his colleagues at the University of British Columbia stated in a 2010 Behavioral and Brain Sciences paper, they are WEIRD: Western, Educated, Industrialised, Rich and Democratic.
Enter the Internet.
A new innovation, the Economist reports, is upending the way psychology experiments are conducted, overturning some conventional wisdom from previous psychology experiments and even changing who conducts psych experiments -- and that is crowdsourcing.
How the Internet is upending psych experiments
The Internet has connected people to help accomplish many projects -- from popular sites such as Kickstarter, which brings funding to ventures, and Wikipedia, which compiles users' knowledge to create an encyclopedia, to lesser-known sites such as oDesk, which connects contractors with would-be employers.
Then, there are sites that allow psychologists to troll for experiment subjects on the Internet, such as Mechanical Turk, the best-known of this lot. The more than half a million "Turkers," as the site's workforce is called, are only 40% American (still a lot, but hey, no longer a majority), a third Indian (the rest come from 100 other countries) and earn a median $1.40 per hour.
These subjects are more diverse than the WEIRD undergrads, though it is true they probably still don't accurately represent the general human population since they have volunteered to participate in these psych experiments for little money.
Overturning conventional psych wisdom
Still, psychologists are finding that using Turkers in experiments has already upended some conventional wisdom in the field:
- The trolley problem: Let's say a runaway trolley is on track to kill a group of people unless you were to push a single individual in front of it (and kill him or her). When presented to undergrads, most of them opted to kill the single individual to save greater people. A Harvard psych lecturer who is re-running this experiment with Turkers has found that only the atheists in the new subject group make the same choice. Those with religious beliefs make a very different choice. (Rand would not provide more details about his results, because they have not yet been published.)
- Punishing freeloaders: In experiments testing cooperation vs. punishment, players had money that they could either contribute to the group in order to increase the value of everyone's stake, or hold on to it themselves, which would then harm everyone. Studies conducted on WEIRD subjects have long showed that society rewards those who cooperate with the group and punishes those who refuse to do so.
In later versions of the experiment, researchers allowed subjects to punish the cooperators to see if they would want to take revenge for having been previously punished. When these experiments were conducted in non-Western countries and on Mechanical Turk, punishing people who had previously cooperated was almost as common as cooperation. These results, which were published last year in Nature Communications, disprove the notion that punishment evolved to encourage cooperation. Instead, as the Economist puts it, punishment "evolved as a self-interested weapon to fend off competitors, even when that competition is, in fact, a strategy of collaboration. In places where rules and institutions do not protect co-operators, freeloaders consistently dominate."
The future impact of crowdsourcing on psychology
Crowdsourcing is already changing the field by making new subjects across the world available to researchers. But it also promises to change the field in other big ways.
First, psychologists have the ability to run experiments quickly: Studies that once would have taken months or years can now be finished in a few days -- both because of access to subjects and because of funds. After all, Turkers willing to be subjects for less than $2 an hour cost much less than undergrad who would expect to make at least 10 times that.
Additionally, the site has now built in the ability to have 60 Turkers (a number that should increase in the future) interact in real time -- a new development that holds more promise for experiments.
Heck, even the field of psychology itself could be crowdsourced: "freelance" psychologists without a university or laboratory could use Mechanical Turk to design their own experiments. And some already have: Gabriele Paolacci, a marketing researcher at the Rotterdam School of Management and a former freelance psychologist, launched a blog called Experimental Turk to help draft guidelines for such freelance experiments.
Looks like diversity is coming not only to the psychology experiment subjects, but to the experimenters to themselves.
Related on SmartPlanet:
- Scientists' newest way to gather data? Enlist hikers
- Using the Internet affects your memory, study says
- Why we like to brag and overshare
- A piece of clothing that boosts your brain power
- What turns a man's mind to mush? A female observer
- Study finds hormone that predicts how long love will last
- Think time equals money? Then you won't enjoy it
- 'Safe' playgrounds could be stunting emotional development
- A pain-erasing pill is no longer the stuff of movies
via: The Economist