Business Brains

Crowdsourcing has a longer-term payoff than originally thought: study

Posting in Technology

Carnegie-Mellon University researchers find crowdsourcing initiatives have a lifecycle all their own, and their value keeps growing long after initial enthusiasm appears to have died out.

Is crowdsourcing a try-it-once fad, or a business idea that will stick over the long run? Typically, for those that have tried crowdsourcing, the number of ideas generated decline over time, and actual implementation rates for posted ideas end up being disappointingly low. But at this point the crowdsourcing process may be delivering more value than first realized, a new study out of Carnegie-Mellon University concludes.

CMU researchers have determined that the crowdsourcing process actually has a lifecycle all it's own, and it's natural for initial enthusiasm to dampen, and ideas to seemingly "dry up." As a result, organizers become discouraged and write off the concept as not working.

But under the covers, a learning process, coupled with market dynamics, are at work.  The study, led by Carnegie-Mellon University professors Kannan Srinivasan, Param Vir Singh and graduate student Yan Huang, finds that crowdsourcing is not a misguided fad, and that done right,  it creates more knowledgeable consumers and, in time, leads to more efficient, lower-cost generation of high potential ideas.

The researchers looked at data from Dell's crowdsourcing website, IdeaStorm.com, which opens up Dell's innovation process to customers. IdeaStorm assigns points to each idea posted, and participants are allowed to vote on each idea, which adds or subtracts points from the idea. Ideas with high ratings are then considered by the company's product development teams. Data for 12,000 ideas -- resulting in 400 product innovations -- was studied.

The study finds that as Dell's crowdsourcing effort matured, the initial rush of ideas of all stripes was replaced by a smaller, but steady, stream of higher-quality ideas that had a greater likelihood of adoption.

This counters the perception that the flow of ideas was slackening due to discouragement stemming from outsiders' unfamiliarity with products, services or cost structure. What actually is happening in crowdsourcing is a learning curve that participants undertake as they get a feel for the process. "Initially, individuals not only overestimate the potential of their ideas, but they also underestimate the cost of implementing their ideas. Hence, individuals tend to overestimate the probability that their idea will be implemented, and therefore, they initially post many ideas," the researchers state. "As individuals learn about the true cost structure of the firm and the potential of their own ideas the expected utility of idea posting for marginal idea contributors decreases. These learning processes cause the low potential idea contributors to stop posting ideas. Hence, the two learning processes perform a filtering function."

As a result, as the crowdsourcing process matures, the number of ideas will diminish, but the quality of the ideas will increase. So, it's natural that participation will drop off, but the efficiency of the crowdsourcing process increases, the researchers conclude.

The researchers urge that companies adopting crowdsourcing approaches put a system for peer evaluation in place, institute rapid company response to ideas that receive significant positive endorsement from the community of contributors, provide a way to help participants gauge costs, and ensure that contributors whose ideas are implemented are rewarded.

As Param Vir Singh explains it:

The initial field of ideas generated in a crowdsourcing effort tends to be overcrowded with ideas that are unlikely to be implemented as consumers overestimate the potential of their ideas and underestimate the cost of implementation. "However, individuals learn about their abilities to come up with high-potential ideas as well as the cost structure of a firm through peer voting and the firm's response to contributed ideas, and individuals whose ideas do not earn the favor of their peers or the backing of the firm drop out of the process while contributors of high-potential ideas remain active," he says.

In other words, as with many things, patience pays off with crowdsourcing.

Related:

Share this

Joe McKendrick

Contributing Editor

Joe McKendrick is an independent analyst who tracks the impact of information technology on management and markets. He is a co-author of the SOA Manifesto and has written for Forbes, ZDNet and Database Trends & Applications. He holds a degree from Temple University. He is based in Pennsylvania. Follow him on Twitter. Disclosure