Monday, 01 August 2011 15:35

Big data coming to everyday enterprise apps


The term 'big data' is generally associated with science and a very few specialised areas of business. But one database company is predicting it will find a home in a much broader range of activities.

The emergence of 'big data' in areas such as astronomy, financial services and utilities is "changing profoundly" the world of database management, said Robert Nagel, vice president of software development at InterSystems. What's needed are systems that can handle the ingestion of massive amounts of well-structured data, persistence, rapid logging and checking, and then allowing analysis to begin while the data keeps accumulating.

Unlike traditional databases, InterSystems' Caché can ingest data at very high rates with low latency, and allow SQL processing while the data accumulates, he said.

Examples of its use include an ESA project to map one billion celestial objects that will require the processing of tens of thousands of 600-byte records each second. According to Nagel, Caché has been demonstrated to handle 90,000 records per second.

Another example is the third largest trading platform in the world (behind the New York Stock Exchange and NASDAQ), which runs at a large US investment bank. The bank previously used a system that was developed in house, but it sometimes failed near the end of the trading day, requiring a considerable recovery effort to identify which trades had been committed.

It was replaced with a Caché based system that delivered better than required latency, halved the infrastructure costs, and - most importantly -= removed the uncertainty associated with system failures.


And the move to smart electricity grids around the world requires the ability to collect and process large amounts of data for realtime analysis for demand and price management. InterSystems partners in Germany are using Caché for this sort of system, Nagel told iTWire.

Caché's approach also works with unstructured data, he said. Publishers are already using Caché to extract concepts from articles as they are being written, retrieve related previous stories for the writer's reference, and then automatically route the finished article to the appropriate editor with a recommendation of its importance. "That's going to be another growth area," said Nagel.

Similarly, the software is being used in contact centres to automatically route incoming emails to the most appropriate customer service agents.

Nagel suggests developers should look for opportunities to improve the ways businesses are run, by tapping into all the available data (structured and unstructured), allowing decisions to be made close to the point of action (to avoid delays), and to take advantage of "mass personalisation".

He thinks data-centricity and massive scalability are important for the next generation of enterprise applications, and will allow companies to run "in profoundly different ways".

"we're very excited about some of the successes customers have had with the [Caché] technology," he said.



You cannot afford to miss this Dell Webinar.

With Windows 7 support ending 14th January 2020, its time to start looking at your options.

This can have significant impacts on your organisation but also presents organisations with an opportunity to fundamentally rethink the way users work.

The Details

When: Thursday, September 26, 2019
Presenter: Dell Technologies
Location: Your Computer


QLD, VIC, NSW, ACT & TAS: 11:00 am
SA, NT: 10:30 am
WA: 9:00 am NZ: 1:00 pm

Register and find out all the details you need to know below.



iTWire can help you promote your company, services, and products.


Advertise on the iTWire News Site / Website

Advertise in the iTWire UPDATE / Newsletter

Promote your message via iTWire Sponsored Content/News

Guest Opinion for Home Page exposure

Contact Andrew on 0412 390 000 or email [email protected]


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments