Wednesday, 27 September 2017 04:42

The need for data-driven DevOps and how to start


In IT, we well know what gets measured gets improved, but our own teams may still be operating on best intentions. Data-driven DevOps matters, says Andi Mann, chief technology advocate at Splunk, and you can have it in your business.

DevOps is, at heart, the application of programming techniques to infrastructure issues. Using tools like Puppet, virtual servers and other environments can be scripted, built, removed, and generally maintained with full version control and automation.

DevOps brings together IT departments, renewing focus on measurement, sharing and continuous feedback loops.

However, as systems become more complex, DevOps workflow becomes complex and opaque. The business wants solutions and features implemented. Programmers want to plan, code, test and release their applications, and along the way, ideally hear back from the business how the new code was received. Infrastructure teams want to manage capacity, ensure uptime, and deliver reliable, predictable and robust systems to users.

Perversely, the more advanced a business is on its DevOps journey, the greater the risk of failure, suffering potential pitfalls like impaired collaboration, increased waste, harder to scale, reduced agility, longer mean time to release and other related problems.

Here's where data-driven DevOps comes in, says Mann.

Mann is an Australian living in Colorado, and responsible for spreading Splunk's message globally, and learning from customers. He describes his role as a 50/50 split between incubating Splunk's DevOps strategy and building products, and time spent on the road working with customers and learning what people really want.

Mann speaks with great enthusiasm about the vast amount of customer insight he has gained and how he has distilled it down to three key metrics, across developers, engineers, QA and management.

1. Cycle time: what is the velocity of delivery? How fast can the business go from an idea to having it in front of its customers, be they external or internal.

2. Quality. It's possible to release quickly but have low quality and see your customers complain. To combat this, typically Scrum teams come together to make go/no-go decisions whether to push a release out. Yet, anecdotally, this is often based on guesswork. Mann says he knows a company with a six-page PowerPoint slide deck which includes 40 data points, filled in by e-mailed responses by 21 people. This was their release process.

However, much of this was guesswork. The data may exist, but it's difficult to find. Some of the respondents are guessing, and some don't have time to look into properly so say "yes" to avoid holding up the process.

"This isn't the right way to release software," Mann says. "It's important to make data-driven decisions on code quality, the number of tests, code coverage, pass/fail rate on tests, whether security has been checked, if the libraries are authorised or unauthorised, and so on."

3. Impact. What is the business impact on the customer? Once the software is released it's important to get information back to the developers so they know if they're doing the right thing. What is the business impact on the code you've written? You may be writing code and delivering fast, but if it is bad code with no business impact then what did you actually achieve? "Even if it's high quality but has no business impact then why do it?" Mann asks.

"These metrics are like a three-legged chair," Mann says. "App delivery falls over if you don't have all three legs."

Yet, knowing you should use metrics — cycle-time, quality and impact — is one thing. How do you put it into practice in your existing environment?

Fortunately, it's easy, Mann says, and it doesn't even have to cost you a cent to get started.

His own product, Splunk, plays a key role in DevOps, he says. "It's the ultimate feedback loop, with systems thinking connected to the customer, helping understand your role in the entire system whether dev or QA security. Splunk provides a common feedback loop for everyone, with lots of people using the same data in different ways. Ops are interested in response time and performance as they keep uptime. Devs are interested in the same numbers to make their app more efficient and use resources more efficiently. There are so many data points that so many people in that data chain will share."

Mann gives this advice for companies who would like to start with data-driven DevOps but don't know where to begin:

  1. Download the free trial of Splunk and sign up for the cloud service. You don't need any on-site infrastructure.
  2. Apply some straightforward techniques to get data out of systems. If you want to measure speed, for example, get data out of your repository of stories. If you use Jira there's a Splunk add-in in the Atlassian marketplace.
  3. Next, get data out of your release system like Jenkins. This gives you an endpoint.

Immediately, with little effort and no cost, you can begin using Splunk to see who in your team is working on new code and when it is being released. You gain insight into your velocity and cycle time right away. This is an important KPI.

You can take it from here, Mann explains, by ingesting from your QA tools like AppScan or Selenium. "Bring that in and see the quality of your releases. Track this over time and see who is doing good work and who might need training."

A more mature statistic is business impact metrics and customer engagement, but this is also achievable, Mann says.

"You can use the Splunk HTTPS Event Collector, or HEC, to see what your users are doing on a web front-end whether they are using mobile or otherwise. Bring in business impact metrics by using Social Sentiment to measure Twitter and Facebook feedback," he says.

"Put this in the dev room and show the current social sentiment on your releases. When people have a problem with a system they don't typically phone you to log a ticket; they turn to others, then complain on Twitter. Measuring your social sentiment gives instant feedback to developers."

"Trying these things out with the free Splunk trial will give your team value very quickly," Mann says. "If you want to get into something deeper and more complex, anyone can look me up and make contact," Mann says.

"With Splunk, you can make decisions on real-time data. Organisations iterating quickly can really benefit. Your developers and engineers can see the customer experience and business impact and customer engagement. This type of data-driven DevOps is really unique in this space," Mann says, "but you get lots of data points. As a QA professional you want to know class quality and open incidents. This is information Ops wants to hear as well so they know what to prepare for, and the quality of the next release."

Mann predicts machine learning will be the next big thing in data-driven DevOps. "The number one question from Operations is 'when will this release come?' followed by 'and how many bugs does it have?'," he says.

"With machine learning, you will know this in advance. You will know how long a certain team takes, the code complexity, the file complexity, and more. It's not a use case I've seen yet, but I'm excited to see who wants to try it out."

The writer is attending Splunk .conf 2017 as a guest of the company. 

Read 5829 times

Please join our community here and become a VIP.

Subscribe to ITWIRE UPDATE Newsletter here
JOIN our iTWireTV our YouTube Community here


If you're looking at enabling Microsoft Teams for your contact centre, you should bookmark this webinar.

Marketing budgets are now focused on Webinars combined with Lead Generation.

Our panellists from Whangarei District Council (NZ) and Maurice Blackburn Lawyers (Aus) were closely involved in recent projects to enable Microsoft Teams for their own contact centres.

They have kindly agreed to join Enghouse and Microsoft to talk about some of the things they would recommend as most critical for IT and CX professionals planning a Teams Contact Centre migration.

Date: 11 May 2022
Time: 12pm AEST | 2pm NZST | 10am SGT

We look forward to having you join us. Please click the button below to register.



The past year has seen a meteoric rise in ransomware incidents worldwide.

Over the past 12 months, SonicWall Capture Labs threat researchers have diligently tracked the meteoric rise in cyberattacks, as well as trends and activity across all threat vectors, including:

Encrypted threats
IoT malware
Zero-day attacks and more

These exclusive findings are now available via the 2022 SonicWall Cyber Threat Report, which ensures SMBs, government agencies, enterprises and other organizations have the actionable threat intelligence needed to combat the rising tide of cybercrime.

Click the button below to get the report.



It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site and prominent Newsletter promotion and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinars and campaigns and assistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.


David M Williams

David has been computing since 1984 where he instantly gravitated to the family Commodore 64. He completed a Bachelor of Computer Science degree from 1990 to 1992, commencing full-time employment as a systems analyst at the end of that year. David subsequently worked as a UNIX Systems Manager, Asia-Pacific technical specialist for an international software company, Business Analyst, IT Manager, and other roles. David has been the Chief Information Officer for national public companies since 2007, delivering IT knowledge and business acumen, seeking to transform the industries within which he works. David is also involved in the user group community, the Australian Computer Society technical advisory boards, and education.

Share News tips for the iTWire Journalists? Your tip will be anonymous




Guest Opinion

Guest Interviews

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News