A new survey conducted by IDC and commissioned by EMC has forecast that 2.8 zettabytes of information (that’s 2.8 billion terabytes) will be generated this year – soaring to 40 zettabytes by 2020. Alister Dias, EMC country manager, acknowledged that much of the data was replicated – but the sheer volumes that businesses and governments have to manage – and pay to store - remains mind boggling.
According to Mr Dias the IDC study estimated that overall investment in storage technologies would rise by only 40 per cent by 2020 despite the expectation that the volumes of data will grow more than tenfold. “Investment per gigabyte will fall from $2 per gigabyte to 20 cents,” as new technologies emerged, he said.
What is still missing however is extensive analysis of this massive data reservoir. In June the Economist Intelligence Unit released a report commissioned by Capgemini, which found that where big data analytics was deployed, a 26 per cent improvement in business performance resulted – with expectations that could rise to 41 per cent over the next three years.
As yet the data is being under exploited according to the IDC report.
He acknowledged that some organisations may be suffering “information overload paralysis” but argued that it was possible to start analysing big data in bite sized chunks.
Mr Dias claimed that currently only 3 per cent of useful data was being tagged and analysed. While he pointed to a handful of examples of big data analysis: “A bank uses big data to analyse unhappy Twitter comments before they complain to the bank; advertisers are analysing pay per view advertiser; airlines analyse frequent flyer information to enhance the experience,” Mr Dias said that there was much more value that could be extracted.
The survey also uncovered vulnerabilities associated with the data claiming that only about half of all the data which needed protecting – for privacy, security or copyright reasons– was being properly protected. Mr Dias said it was important that organisations consider issues such as data security, cloud computing and big data as inherently linked and develop intertwined policies for all three issues.
Another issue identified by the report was that the centre of the so-called “digital universe” was shifting.
According to Mr Dias when EMC and IDC started measuring the digital universe, 48 per cent of it emanated from the US and Europe, with only 20 per cent coming from so called emerging markets. Today emerging markets represent 36 per cent of the digital universe, and by 2020 are expected to represent 62 per cent of the market with China alone generating 22 per cent of the world’s data.