For many years now IDC has published a biennial Digital Universe report, sponsored by EMC which has examined the exponential growth of digital data in the world. According to the most recent study, from June 2011, the amount of information created and replicated in the world surpassed 1.9 zettabytes (1.8 trillion gigabytes) last year, growing by almost nine times in just five years.
“Some aspects of big data are indeed new,” says IDC Australia’s Craig Stires. “And they are creating unprecedented levels of complexity for IT executives. As the variety of data sources increases rapidly, and the velocity at which data is generated also increases, IT executives are beginning to realise that these massive data sets cannot be processed, managed, and analysed using traditional databases and architectures.”
Stires says much data is structured, but that an increasing amount is unstructured, such as social media interactions, rich media files and geospatial information. “If your organisation is one of the growing number looking for deeper insight into customers and operations, then you understand that big data will be key.
“New possibilities are available for real-time analysis and response to customer interactions, whether through better customer service, or more targeted offers,” says Stires. “With technology barriers falling, organisations are given affordable access to infrastructure that scales up and out.
“They also now have access to software that offers high performance capabilities in traditional application, while extending its reach in video, voice, and social data insights. On top of this, services companies are providing roadmaps and best-practices to deliver business value from proven use cases.”
In line with this trend, IDC says CIOs and IT departments are looking at a major reassessment of both infrastructure, primarily leveraging high performance computing, in-memory technologies and massively parallel processing (MPP) architectures, and improved information management and business intelligence analytics.
“It's a great time to be mapping out how you can lead the transformation in your organisation. It's the right time to pay attention to the combination of increasing availability of experiences, end-user awareness, affordable technologies, and emerging vendor solutions. Making a critical, informed decision on your big data strategy today will be crucial to your organisations success and continued competitiveness tomorrow”.
IDC will discuss big data at a business analytics briefing in Sydney on Wednesday 26 September, at Sydney’s Shangri-La Hotel.