Big Data Market Segment LS
Big Data Market Segment RS
Tuesday, 03 March 2020 17:40

IoT data calls for a strategy: storage firm CTO

Pure Storage CTO Matt Oostveen Pure Storage CTO Matt Oostveen

The rapid growth of machine-generated data from IoT sensors and devices presents a challenge for organisations, according to Pure Storage chief technology officer Matt Oostveen.

The situation with machine-generated data has changed markedly over the last two or three years, Oostveen told iTWire, so it "can be challenging" to predict capacity requirements.

Obviously, some organisations collect more data than others, but even some small operations generate massive amounts of data.

Consequently, there is a risk of "being wrong-footed in a short period of time" if your competitors can cope with this data growth but you have failed to make the right platform and infrastructure decisions to set you up for tomorrow, he said.

That might sound self-serving as Pure Storage is a storage company, but Oostveen added "I don't think that one company has the single answer, because there's more than one question."

Pure does have some of the answers, he said, but the questions are not all about technology.

"The business element is extremely important," he observed. What is the business goal? What questions are you trying to answer?

Oostveen advocates the creation of teams of experts with a clear senior leader, whether they are the chief data officer or bear some other title. However, it is important to realise that this role transcends finance and IT, and touches every part of the organisation.

Team expertise is about more than just data skills. Creativity is also important, as analytics isn't just about data sets and complex calculations – these teams need to understand what could be done for the organisation with the available tools, and how they can unlock existing intellectual property.

When looking at other companies for inspiration, it would be a mistake to focus only on your own industry. Rather, look at industries that you admire, he said. For example, a mining company might look at high-tech manufacturing firms.

Another possibility is to investigate partnerships with companies such as infrastructure providers, telcos and consultancies, with the aim of being able to consume data services instead of running everything in-house.

It is sometimes suggested that the cloud is the answer to storing and analysing vast amounts of IoT data, but Oostveen isn't convinced. Elasticity isn't always an issue, and it can be less expensive to keep those large data sets on premises.

Data is increasingly being collected by edge devices, and moving all of it to another location can be prohibitively expensive. Data gravity means it is more efficient to move infrastructure to the point of data creation rather than moving data to existing infrastructre

So what's needed is an architecture that allows data to be preprocessed and used at the edge before moving a subset to the cloud or a data centre, with a single control plane for all those locations.

Local flash storage – such as that offered by Pure – is generally better for real-time analytics such as using data from infrared cameras to detect overheating, allowing equipment to be repaired before it fails.

Different architecture and infrastructure choices lead to different outcomes, Oostveen observed, but so do the executive owner of the initiative, and whether or not good use is made of the talent and knowledge that exists within the organisation.

WEBINAR event: IT Alerting Best Practices 27 MAY 2PM AEST

LogicMonitor, the cloud-based IT infrastructure monitoring and intelligence platform, is hosting an online event at 2PM on May 27th aimed at educating IT administrators, managers and leaders about IT and network alerts.

This free webinar will share best practices for setting network alerts, negating alert fatigue, optimising an alerting strategy and proactive monitoring.

The event will start at 2pm AEST. Topics will include:

- Setting alert routing and thresholds

- Avoiding alert and email overload

- Learning from missed alerts

- Managing downtime effectively

The webinar will run for approximately one hour. Recordings will be made available to anyone who registers but cannot make the live event.



Security requirements such as confidentiality, integrity and authentication have become mandatory in most industries.

Data encryption methods previously used only by military and intelligence services have become common practice in all data transfer networks across all platforms, in all industries where information is sensitive and vital (financial and government institutions, critical infrastructure, data centres, and service providers).

Get the full details on Layer-1 encryption solutions straight from PacketLight’s optical networks experts.

This white paper titled, “When 1% of the Light Equals 100% of the Information” is a must read for anyone within the fiber optics, cybersecurity or related industry sectors.

To access click Download here.


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments