Splunk chief executive Doug Merritt, among other executives and presenters, made 20 product announcements last week at Orlando during Splunk’s annual .conf conference. Highlights include data stream processing, business user reporting, augmented reality and smartphone and Apple TV applications.
These new capabilities in Splunk>next represent the largest change to how Splunk administrators — and non-technical users alike — can interact with Splunk since the product’s inception.
Splunk Data Stream Processor, for example, is a major shift. Splunk’s historic distinctive feature over traditional data aggregation tools was its approach to ingesting unstructured data rapidly without any mapping or interpretation or other processing at that time. The data was then indexed and searchable. Now data can be analysed and acted on, and modified, while it is in motion, not only when at rest.
An example of modifying data might be to redact credit card numbers from a point-of-sale system before hitting the indexer. Previously, Splunk administrators would edit configuration files to achieve this. Splunk>next provides a visual pipeline editor whereby a simple rule can be added to modify this field before it is ingested.
Of course, you don’t want to adversely impact this ingestion pipeline. Splunk says its performance permits 100TB of data to be ingested per day, with millisecond latency over trillions of events. Even so, to make sure your own pipelines aren’t a problem, you can drill into the latency and the volume of inputs and outputs in your own rules and diagnose problems, all within the GUI. The pipeline is versioned so you can revert to a prior edition, should you find you’ve inadvertently added a bad rule.
The pipeline need not be a single path, either. The visual editor allows you to effortlessly send your Splunk data several ways. This means you could still ingest the raw data to one repository, and transformed data into another.
On the topic of storage, a second significant architectural change in Splunk>next is dubbed SmartStore, essentially separating compute and storage. Now storage can be any S3 API-compliant storage. This provides Splunk admins flexibility in configuring their resources, and also allows considerable pricing benefits. Splunk states this change alone means admins can improve their total cost of ownership by up to 75%.
Additionally, Splunk>next makes more use of smartphone applications, recognising Splunk admins and users alike are increasingly transacting on-the-go. This includes responding to out-of-hours alerts.
Pulling out a laptop is a thing of the past; Splunk Mobile will let you carry the power of Splunk and remedy situations on your smartphone immediately. Data is encrypted, and the app supports both on-premise and cloud Splunk environments.
This app shows your Splunk dashboards and provides alert information on both your phone and smartwatch. With a single button click -— implemented via actions within Splunk’s alert management — you could realistically identify a process is consuming all your server’s CPU, then reboot it, or perform other relevant tasks, without once using any device beyond your mobile and, optionally, your watch.
Splunk Augmented Reality
Meanwhile, Splunk Augmented Reality allows people to see their data where it lives. The use case Splunk imagines is an Internet of Things environment such as a factory with many machines generating data. With a smartphone — or even a headset — Splunk Augmented Reality will allow an administrator to pin dashboards and status items in literally the air around equipment, which the factory supervisor or other users can see as they walk around. Again, there’s no need to pull a laptop out, with information easily available in an efficient and modern way.
Of course, this need not be applied to only an industrial setting, enabling any physical real-world experience to be augmented. This could include routers and switches and servers, or even simply the boardroom door.
Tim Tully, Splunk’s chief technology officer, told iTWire exclusively the augmented research idea came to the team out of an aquaponic research lab. The lab used Splunk to plot and measure performance around plant data such as room temperature, humidity and nitrogen content, but wanted to easily open this data to students. The augmented reality idea came in here – instead of requiring students to look up plant IDs and type into an app, the team imagined they could position QR codes around the plants, and allowing students to walk in and point their phone at the QR codes and see data straight away.
Splunk Natural Language
Splunk’s mobile apps are also voice-enabled with natural language understanding — dubbed Splunk Natural Language — so users can simply ask questions like “What’s the most popular unit sold today?” and receive an answer. Tim Tully confirmed to iTWire this required a small degree of ontology management to understand the data, but “is very lightweight,” he said, with the bulk of the functionality out-of-the-box.
Splunk IT Service Intelligence 4.0
Another major announcement is Splunk IT Service Intelligence (ITSI) 4.0, adding predictive analytics with cause analysis. This major release promises to identify any issue with any entity across your environment, predicting outages before they occur. You can drill-down in ITSI and learn probable causes behind predicted degradation and see where a problem is occurring to get the right people involved to solve the problem before it occurs.
This prediction has vast business ramifications, says Andi Mann, Splunk’s chief technology advocate. “It’s not just fixing problems early and doing what you’d have to do anyway with no money saving, The business aspect is it saves huge amounts of time, resources, regulatory impact, fines and reputation damage.
“If I can get ahead of the problem and resolve IT issues before they occur there’s a bunch of stuff you don’t have to put to that problem if you know it’s coming. If you find and fix a problem before it happened you don’t have to issue a statement to the press, issue an email to your sales team, experience customer churn due to unavailable systems, you don’t have to recover from a breach if it never happened, your stock price is affected, you don’t have to mandatorily report a breach if it never happened,” Mann told iTWire. “It’s a game changer.”
Splunk Business Flow
Also announced was Splunk Business Flow, providing a graphical environment for non-technical business users to visualise business processes in a flowchart style, using the same data Splunk is collecting anyway. Users can drill-down into performance metrics and problems.
Splunk’s executive team see this product as one that will unlock significant value for its customers.
Tully told iTWire this product is “really ground-breaking”, optimising scenarios like manufacturing, e-commerce, help centres, or anything with a process. “There’s a ton of value customers will unlock over time,” he said.
Rick Fitz, senior vice-president, IT Markets, said to iTWire “The CIO agenda is to move more dollars from running the business into value-creation to make the business more innovative. Business people want to improve the customer experience, the product experience and drive business issues.”
However, “back-office systems to support digital processes are massively complex. There isn’t one single app, but 20 or more like ERP, web, commerce, sales … they are really complex and all there to serve the customer but the systems are siloed with no way to stitch them together to get a holistic view. This is what Splunk Business Flow does – stitches together processes so business owners can view their processes and refine and test and create value. This is the CIO agenda.”
“Splunk Business Flow helps CIOs achieve their value-creation agenda.”
Other new product announcements include:
- Splunk Workload Management, permitting Splunk admins to dedicate CPU and memory to where they are needed most
- Splunk Data Fabric Search, providing federated search across multiple Splunk instances, and doing so with performance improvements well in excess of 1000% faster, and scaling to trillions of events
- Splunk App for Infrastructure, free from Splunkbase, with one view, seamlessly presenting correlated metrics with logs enabling easier investigation
- Splunk for Industrial IoT, with real-time monitoring of industrial equipment providing detection and mitigation of security risks, and condition- and predictive-based maintenance advice6. Splunk Machine Learning Toolkit 4.0, with connectors for Apache Spark and containers for TensorFlow
- Splunk TV, providing an Apple TV app to display your dashboards and information, designed for NOC and SOC customers who would otherwise run a dedicated PC 24x7. This app is also voice-enabled
- Splunk Developer Cloud
- Splunk Insights for Web and Mobile Apps
- Splunk Enterprise Security 5.2
- Splunk User Behaviour Analytics 4.2
- Splunk Phantom 4.1
The writer attended Splunk .conf 2018 as a guest of the company.