Drowning in data: IDC predicts a plethora of petabytes

Stuart Corner
Tuesday, 04 May 2010 17:13

Business IT - Technology

Since 2007, on behalf of EMC, IDC has been attempting to estimate and forecast the amount of digital information created and replicated each year: it puts this at 1.2 zettabytes (10 raised to the power or 21 bytes) in 2010 and expects it to grow to 35 zettabytes by 2020. That's 35 trillion gigabytes.

If this were not problem enough, IDC predicts that this digital information will be produced in every smaller packages - files, images, record etc: the entities that actually give this data some meaning. It predicts that the number of 'data entities' to be managed will grow twice as fast as the number of bits, to be 67 times their current volume by 2020, some 25 quintillion 'data containers'. One quintillion being 10 to the power of 30.

The practical impact of these eye-watering statistics, IDC says, is that we will need new search and discovery tools. "Most of the Digital Universe is unstructured data (for example, images and voice packets). We will need new ways to add structure to unstructured data, to look inside the information containers and recognise content such as a face in a security video. In fact, the fastest-growing category in the Digital Universe is metadata, or data about data."

Deciding what to do with all this information, and securing it will present new headaches, IDC says. "We'll need to classify [information] by importanc: know when to delete it, and predict which information we will need in a hurry. How will we follow the growing number of government and industry rules about retaining records, tracking transactions, and ensuring information privacy?"


CONTINUED