Home Business IT Business Telecommunications Big data coming to everyday enterprise apps

Big data coming to everyday enterprise apps

The term 'big data' is generally associated with science and a very few specialised areas of business. But one database company is predicting it will find a home in a much broader range of activities.

The emergence of 'big data' in areas such as astronomy, financial services and utilities is "changing profoundly" the world of database management, said Robert Nagel, vice president of software development at InterSystems. What's needed are systems that can handle the ingestion of massive amounts of well-structured data, persistence, rapid logging and checking, and then allowing analysis to begin while the data keeps accumulating.

Unlike traditional databases, InterSystems' Caché can ingest data at very high rates with low latency, and allow SQL processing while the data accumulates, he said.

Examples of its use include an ESA project to map one billion celestial objects that will require the processing of tens of thousands of 600-byte records each second. According to Nagel, Caché has been demonstrated to handle 90,000 records per second.

Another example is the third largest trading platform in the world (behind the New York Stock Exchange and NASDAQ), which runs at a large US investment bank. The bank previously used a system that was developed in house, but it sometimes failed near the end of the trading day, requiring a considerable recovery effort to identify which trades had been committed.

It was replaced with a Caché based system that delivered better than required latency, halved the infrastructure costs, and - most importantly -= removed the uncertainty associated with system failures.


And the move to smart electricity grids around the world requires the ability to collect and process large amounts of data for realtime analysis for demand and price management. InterSystems partners in Germany are using Caché for this sort of system, Nagel told iTWire.

Caché's approach also works with unstructured data, he said. Publishers are already using Caché to extract concepts from articles as they are being written, retrieve related previous stories for the writer's reference, and then automatically route the finished article to the appropriate editor with a recommendation of its importance. "That's going to be another growth area," said Nagel.

Similarly, the software is being used in contact centres to automatically route incoming emails to the most appropriate customer service agents.

Nagel suggests developers should look for opportunities to improve the ways businesses are run, by tapping into all the available data (structured and unstructured), allowing decisions to be made close to the point of action (to avoid delays), and to take advantage of "mass personalisation".

He thinks data-centricity and massive scalability are important for the next generation of enterprise applications, and will allow companies to run "in profoundly different ways".

"we're very excited about some of the successes customers have had with the [Caché] technology," he said.



Did you know: 1 in 10 mobile services in Australia use an MVNO, as more consumers are turning away from the big 3 providers?

The Australian mobile landscape is changing, and you can take advantage of it.

Any business can grow its brand (and revenue) by adding mobile services to their product range.

From telcos to supermarkets, see who’s found success and learn how they did it in the free report ‘Rise of the MVNOs’.

This free report shows you how to become a successful MVNO:

· Track recent MVNO market trends
· See who’s found success with mobile
· Find out the secret to how they did it
· Learn how to launch your own MVNO service


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.