Follow this blog:
RSS

Is ‘Big Data’ good or evil? New survey exposes growing debate

By | July 21, 2012, 5:13 PM PDT

The potential of ‘Big Data’ — those petabytes’ and exabytes’ worth of structured and unstructured information now being generated and gathered by users and systems across the globe — is bright, presenting new opportunities for growth and innovation. However, the world may still not be ready to effectively handle and understand all this data.

That’s the conclusion drawn from a recent survey of 1,021 Internet experts and other Internet users, published by the Pew Research Center and the Imagining the Internet Center at Elon University.

Tech experts believe the vast quantities of data — so-called “digital exhaust” — that humans and machines will be creating by the year 2020 could enhance productivity, improve organizational transparency, and expand the frontier of the “knowable future.” But they worry about “humanity’s dashboard” being in government and corporate hands and they are anxious about people’s ability to analyze it wisely.

A majority, 53%, felt positive about the enhanced capabilities Big Data would deliver, agreeing with the statement that “human and machine analysis of large data sets will improve social, political, and economic intelligence by 2020. The rise of what is known as Big Data will facilitate things like real-time forecasting of events); the development of “inferential software” that assesses data patterns to project outcomes; and the creation of algorithms for advanced correlations that enable new understanding of the world.”

However, that’s a slim majority. There’s another 39% of internet experts who express concern. They agreed with the counter-argument to Big Data’s benefits, which posited that “Human and machine analysis of Big Data will cause more problems than it solves by 2020. The existence of huge data sets for analysis will engender false confidence in our predictive powers and will lead many to make significant and hurtful mistakes. Moreover, analysis of Big Data will be misused by powerful people and institutions with selfish agendas who manipulate findings to make the case for what they want.”

As one of the study’s participants, entrepreneur Bryan Trogdon put it: “Big Data is the new oil,” observing that “the companies, governments, and organizations that are able to mine this resource will have an enormous advantage over those that don’t. With speed, agility, and innovation determining the winners and losers, Big Data allows us to move from a mindset of ‘measure twice, cut once’ to one of ‘place small bets fast.’”

The Pew/Elon report notes that user-generated information — such as that coming from social networks or transactions — “is only part of the story, a relatively shrinking part. Machines and implanted sensors in oceans, in the soil, in pallets of products, in gambling casino chips, in pet collars, and countless other places are generating data and sharing it directly with data ‘readers’ and other machines that do not involve human intervention.”

Government leaders, scientists, corporate leaders, health officials, and education specialists “are anxious to see if new kinds of analysis of large data sets can yield insights into how people behave, what they might buy, and how they might respond to new products, services, and public policy programs,” the report states.

Concerns voiced include privacy, as more and more data is collected about people—both as they knowingly disclose things in such things as their postings through social media and as they unknowingly share digital details about themselves as they march through life. “Not only do the advocates worry about profiling, they also worry that those who crunch Big Data with algorithms might draw the wrong conclusions about who someone is, how she might behave in the future, and how to apply the correlations that will emerge in the data analysis,” the report cautions. “There are also plenty of technical problems. Much of the data being generated now is unstructured and sloppily organized. Getting it into shape for analysis is no tiny task.”

Some opinions voiced:

Jeff Jarvis, professor, pundit and blogger: “Media and regulators are demonizing Big Data and its supposed threat to privacy/ Such moral panics have occurred often thanks to changes in technology…But the moral of the story remains: there is value to be found in this data, value in our newfound publicness. Google’s founders have urged government regulators not to require them to quickly delete searches because, in their patterns and anomalies, they have found the ability to track the outbreak of the flu before health officials could and they believe that by similarly tracking a pandemic, millions of lives could be saved. Demonizing data, big or small, is demonizing knowledge, and that is never wise.”

Sean Mead, director of analytics at Mead, Mead & Clark, Interbrand: “Large, publicly available data sets, easier tools, wider distribution of analytics skills, and early stage artificial intelligence software will lead to a burst of economic activity and increased productivity comparable to that of the Internet and PC revolutions of the mid to late 1990s. Social movements will arise to free up access to large data repositories, to restrict the development and use of AIs, and to ‘liberate’ AIs.”

Heywood Sloane, principal at CogniPower: “This isn’t really a question about the Internet or Big Data—it’s a question about who and how much people might abuse it (or anything else), intentionally or otherwise. That is a question that is always there—thus there is a need for a countervailing forces, competition, transparency, scrutiny, and/or other ways to guard against abuse. And then be prepared to misjudge sometimes.”

Perhaps David Weinberger of Harvard University’s Berkman Center put it the best perspective: “We are just beginning to understand the range of problems Big Data can solve, even though it means acknowledging that we’re less unpredictable, free, madcap creatures than we’d like to think.”
(Photo: Joe McKendrick)

Start your week smarter with our weekly e-mail newsletter. It's your cheat sheet for good ideas. Get it.

Joe McKendrick

About Joe McKendrick

Joe McKendrick is a contributing editor for SmartPlanet.

Joe McKendrick

Joe McKendrick

Contributing Editor, Business

Joe McKendrick is an independent analyst who tracks the impact of information technology on management and markets. He is the author of the SOA Manifesto and has written for Forbes, ZDNet and Database Trends & Applications. He holds a degree from Temple University. He is based in Pennsylvania.

Follow him on Twitter.

Joe McKendrick

Joe McKendrick

Joe McKendrick is an independent consultant and editor. Joe has performed project work for the following companies in the IT marketspace: IBM, Systinet/HP, Teradata. He has performed project work for the following organizations in partnership with Unisphere Research (Unisphere Media): IBM, Oracle Corp., International Oracle Users Group, Oracle Applications Users Group, Professional Association for SQL Server, International DB2 Users Group, International Sybase Users Group.

He writes for SmartPlanet and is not an employee of CBS.

If you liked this, don't miss...
3
Comments

Join the conversation!

Follow via:
RSS
0 Votes
+ -
evil
It's being gathered without properly informed voluntary consent (yah, I know it's redundant, but I'm emphasizing here).

Then it's being kept for longer than the owner would have ever wanted...

And it's being abused for purposes to which the owner would never have consented... if she had had the chance to specify her terms and conditions.

And it's being abused, being put to nefarious purposes beyond merely purposes the owner would not have readily agreed to.
Posted by Professor8
23rd Jul
0 Votes
+ -
Big data is a cause for optimism
It's surprising that elites would be so concerned about the problems that big data causes - especially when the reasons for concern are relatively unformed. "Engendering false confidence in our predictive powers" does not really seem like much of a concern when we are placing "small bets fast."

If the bets are small, we can course correct quickly - no reason to feel overconfident about our predictive powers - they will be easy to judge on their efficacy.

A bigger challenge is what to do about Black Swan events. This is already a problem for the financial industry, with 70% of trades performed by algorithms. All of the big data modeling goes in to determining what other algorithms are doing, which means that when something genuinely out of the ordinary happens, the whole system breaks down. The Flash Crash is a perfect example of this.

So, how to ensure this does not happen? At Janrain, as we see more customers storing social profile data, the question of what to do with it focuses more on how to keep humans, with the ability to interpret and judge, in the decision-making process where they can provide the most value. This will not change - in fact, the need for more and deeper understanding will be the biggest challenge of dealing with big data.
Posted by Jamie Beckland
23rd Jul
0 Votes
+ -
What is big data?
"Big" is just a kind of volume.
Posted by Daniel Isakevich
23rd Jul
Join the conversation
Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]

Join the SmartPlanet community and join the conversation! Signing up is fast and free. Don't wait -- we want to hear your opinion!