Data explosion: enough to fill DVDs stretching to the moon and back

IDC has just released estimates that this year, the Digital Universe — meaning every electronically stored piece of data or file out there — will reach 1.2 million petabytes, or 1.2 zettabytes, this year.

That’s up from a measly 800,000 petabytes in 2009. Despite an economic slowdown or meltdown in some quarters, the total amount of data still grew by 62%, IDC reports. To illustrate how much data this is, write John Gantz and David Reinsel, authors of the latest IDC report on the size of the Digital Universe, “picture a stack of DVDs reaching from the earth to the moon and back.” (About 240,000 miles each way.)

Still, this is only the beginning of a data explosion, Gantz and Reinsel observe — by 2020, the amount of data will have grown 44-fold from 2009, to 35 trillion gigabytes. Fueling the growth will be the evolution of all major forms of media – voice, TV, radio, print – from analog to digital.

“Our stack of DVDs would now reach halfway to Mars,” Gantz and Reinsel note. Most of this digital content is not unique — in fact, Gantz and Reinsel relate, “nearly 75% of our digital world is a copy – in other words, only 25% is unique.”

Of course, each and everyone of us contributes to the pile every day. Len Devanna, executive with EMC, sponsor of the study, puts it in perspective:

"In the past four weeks, I've personally generated about 5.3 GB in video, 1.6 GB in photos, 3 GB in audio  and let's say another 1 GB for emails, documents, Tweets, etc;. That's about 11 GB of new information that did not exist a short month ago. That's about 500 times larger than my first hard drive. Of course, these numbers represent just the stuff I know about. There's also the information inherently created as I simply go through life... Every time I make an online purchase, swipe my badge to enter work, get processed  going through a toll booth with my EZ Pass, etc;.  Each of these things leaves behind some digital footprint, and generates new information in the process."

IDC also estimates that at least 15% of the Digital Universe by 2020 will be managed or stored in the cloud — that is, “created in the cloud, delivered to the cloud, stored and manipulated in the cloud.” Gantz and Reinsel add that at least a third of all this data will “pass through the cloud” at some point in the lifecycle.

With all this data comes security headaches, of course. “By 2020, almost 50% of the information in the Digital Universe will require a level of IT-based security beyond a baseline level of virus protection and physical protection. That’s up from about 30% this year,” Gantz and Reinsel caution. “And while the portion of that part of the Digital Universe that needs the highest level of security is small – in gigabytes and total files – that portion will grow by a factor of 100.”

And, they add: “Not all data needs to be protected equally. A YouTube video of a cat doing tricks would seem to need less protection against hacking or corruption than a home-banking customer’s account balances.”

One bit of good news is the staffing and investment to manage this Digital Universe will only grow by a factor of 1.4. IDC estimates that in 2009, the world spent nearly $4 trillion on hardware, software, services, networks, and IT staff to manage the Digital Universe. “That spending is expected to grow modestly between now and 2020, which means the cost of managing each byte in the Digital Universe will drop steadily – an incentive to create even more information,” Gantz and Reinsel say.