Information management vendor Veritas Technologies has released The Global Databerg Report based on a survey of 2,550 senior IT decision makers across 22 countries (including 100 in Australia) that was conducted by Vanson Bourne.
The survey revealed that 52% of all information currently stored and processed by organisations around the world is considered 'dark' data of unknown value. Another 33% of data is known to be useless as it is redundant, obsolete, or trivial (ROT).
At this rate, organisations around the world will waste $4.6 trillion cumulatively on the management of dark and ROT business data by the year 2020.
If the people surveyed are right, as little as 15% of stored data is actually business critical. If an organisation holds 1PB of data in total, it is probably spending more than $900,000 a year to store non-critical data.
In Australia, the figures are mixed: only 10% of data is known to be business critical, but 28% ROT data is better than the global average though still a cause for concern.
"Understanding and acknowledging that a data hoarding culture exists is a first step in addressing the problem," said Veritas senior vice president and APJ sales leader Chris Lin.
"There is an immediate need for organisations in Australia to take control of their databerg and identify business value and risk. Today, only 13% of Australian organisations do so.
"Data needs to be classified based on the organisation's data retention policy and there is a growing demand for an effective information journey for dark data to be implemented."
Other factors include the growing use of cloud (potentially adding to dark and ROT data), and employees using corporate facilities to store private data.
Australians are just above average in the latter regard, with 26.5% of us storing personal data on work devices, compared with 26% worldwide.
The problem is not just the cost of providing and managing this storage, but also the legal issues around data privacy (copies of legal and identity documents are being stored) and copyright (eg, unlicensed software).
Not understanding what is being stored on corporate resources is very risky and leaves no 'plausible denials' in the event of regulatory or criminal investigation, Veritas warned.
Publication of the Global Databerg Report follows the introduction of Veritas's Data Genomics Index, which showed that over 40% of stored data has not been touched in over three years, and is considered 'stale'.
Veritas demerged from Symantec last year.