The situation with machine-generated data has changed markedly over the last two or three years, Oostveen told iTWire, so it "can be challenging" to predict capacity requirements.
Obviously, some organisations collect more data than others, but even some small operations generate massive amounts of data.
Consequently, there is a risk of "being wrong-footed in a short period of time" if your competitors can cope with this data growth but you have failed to make the right platform and infrastructure decisions to set you up for tomorrow, he said.
Pure does have some of the answers, he said, but the questions are not all about technology.
"The business element is extremely important," he observed. What is the business goal? What questions are you trying to answer?
Oostveen advocates the creation of teams of experts with a clear senior leader, whether they are the chief data officer or bear some other title. However, it is important to realise that this role transcends finance and IT, and touches every part of the organisation.
Team expertise is about more than just data skills. Creativity is also important, as analytics isn't just about data sets and complex calculations – these teams need to understand what could be done for the organisation with the available tools, and how they can unlock existing intellectual property.
When looking at other companies for inspiration, it would be a mistake to focus only on your own industry. Rather, look at industries that you admire, he said. For example, a mining company might look at high-tech manufacturing firms.
Another possibility is to investigate partnerships with companies such as infrastructure providers, telcos and consultancies, with the aim of being able to consume data services instead of running everything in-house.
It is sometimes suggested that the cloud is the answer to storing and analysing vast amounts of IoT data, but Oostveen isn't convinced. Elasticity isn't always an issue, and it can be less expensive to keep those large data sets on premises.
Data is increasingly being collected by edge devices, and moving all of it to another location can be prohibitively expensive. Data gravity means it is more efficient to move infrastructure to the point of data creation rather than moving data to existing infrastructre
So what's needed is an architecture that allows data to be preprocessed and used at the edge before moving a subset to the cloud or a data centre, with a single control plane for all those locations.
Local flash storage – such as that offered by Pure – is generally better for real-time analytics such as using data from infrared cameras to detect overheating, allowing equipment to be repaired before it fails.
Different architecture and infrastructure choices lead to different outcomes, Oostveen observed, but so do the executive owner of the initiative, and whether or not good use is made of the talent and knowledge that exists within the organisation.