Home Business IT Business Telecommunications Big data coming to everyday enterprise apps

The term 'big data' is generally associated with science and a very few specialised areas of business. But one database company is predicting it will find a home in a much broader range of activities.

The emergence of 'big data' in areas such as astronomy, financial services and utilities is "changing profoundly" the world of database management, said Robert Nagel, vice president of software development at InterSystems. What's needed are systems that can handle the ingestion of massive amounts of well-structured data, persistence, rapid logging and checking, and then allowing analysis to begin while the data keeps accumulating.

Unlike traditional databases, InterSystems' Caché can ingest data at very high rates with low latency, and allow SQL processing while the data accumulates, he said.

Examples of its use include an ESA project to map one billion celestial objects that will require the processing of tens of thousands of 600-byte records each second. According to Nagel, Caché has been demonstrated to handle 90,000 records per second.

Another example is the third largest trading platform in the world (behind the New York Stock Exchange and NASDAQ), which runs at a large US investment bank. The bank previously used a system that was developed in house, but it sometimes failed near the end of the trading day, requiring a considerable recovery effort to identify which trades had been committed.

It was replaced with a Caché based system that delivered better than required latency, halved the infrastructure costs, and - most importantly -= removed the uncertainty associated with system failures.


And the move to smart electricity grids around the world requires the ability to collect and process large amounts of data for realtime analysis for demand and price management. InterSystems partners in Germany are using Caché for this sort of system, Nagel told iTWire.

Caché's approach also works with unstructured data, he said. Publishers are already using Caché to extract concepts from articles as they are being written, retrieve related previous stories for the writer's reference, and then automatically route the finished article to the appropriate editor with a recommendation of its importance. "That's going to be another growth area," said Nagel.

Similarly, the software is being used in contact centres to automatically route incoming emails to the most appropriate customer service agents.

Nagel suggests developers should look for opportunities to improve the ways businesses are run, by tapping into all the available data (structured and unstructured), allowing decisions to be made close to the point of action (to avoid delays), and to take advantage of "mass personalisation".

He thinks data-centricity and massive scalability are important for the next generation of enterprise applications, and will allow companies to run "in profoundly different ways".

"we're very excited about some of the successes customers have had with the [Caché] technology," he said.



With 4 keynotes + 33 talks + 10 in-depth workshops from world-class speakers, YOW! is your chance to learn more about the latest software trends, practices and technologies and interact with many of the people who created them.

Speakers this year include Anita Sengupta (Rocket Scientist and Sr. VP Engineering at Hyperloop One), Brendan Gregg (Sr. Performance Architect Netflix), Jessica Kerr (Developer, Speaker, Writer and Lead Engineer at Atomist) and Kent Beck (Author Extreme Programming, Test Driven Development).

YOW! 2018 is a great place to network with the best and brightest software developers in Australia. You’ll be amazed by the great ideas (and perhaps great talent) you’ll take back to the office!

Register now for YOW! Conference

· Sydney 29-30 November
· Brisbane 3-4 December
· Melbourne 6-7 December

Register now for YOW! Workshops

· Sydney 27-28 November
· Melbourne 4-5 December



Australia is a cyber espionage hot spot.

As we automate, script and move to the cloud, more and more businesses are reliant on infrastructure that has the high potential to be exposed to risk.

It only takes one awry email to expose an accounts’ payable process, and for cyber attackers to cost a business thousands of dollars.

In the free white paper ‘6 Steps to Improve your Business Cyber Security’ you’ll learn some simple steps you should be taking to prevent devastating and malicious cyber attacks from destroying your business.

Cyber security can no longer be ignored, in this white paper you’ll learn:

· How does business security get breached?
· What can it cost to get it wrong?
· 6 actionable tips


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.


Popular News




Sponsored News