Home Whiskey Tango Foxtrot! Marketing people and useful statistics – n'er the twain shall meet

Author's Opinion

The views in this column are those of the author and do not necessarily reflect the views of iTWire.

Have your say and comment below.

Marketing people and useful statistics – n'er the twain shall meet Image: Max Pixel (used with permission)

Word doesn't make you an author, Photoshop doesn't make you an artist, and SurveyMonkey most certainly doesn't make you a researcher.

I recently read an article noting that the expected feedback value for Uber riders and drivers was 5 (out of 5) unless there were major reasons to offer something lower. It seemed that a 4 was for unreasonable behaviour and a 3 meant disaster. Who knows how bad you'd have to be in order to rate a 1 or 2!!

There is something inherently wrong with this kind of assessment – the expectation that maximum marks will be given unless there was an exceptional reason not to (this smells a lot like modern primary schools' "everyone gets a prize" mentality that we hear of so often). Surely, there is almost always room for improvement, no matter how exceptional the service delivery.

Until recently, this seemed to be relatively "academic", I'd not experienced it personally (not being an Uber user – am I the only one left?). However, I came to have my car serviced at a local dealership that represented the vehicle's very well-known international brand.

At the time I paid the account and collected my keys, I was warned by the assistant that I may receive an email requesting a response to the brand's standard customer survey. As part of the conversation, the assistant warned me that any rating lower than a '9' (out of 10) would reflect poorly on the dealership. I elected to stay schtum and not reveal a surprised/aghast expression on my face. My surprise extended in two directions – that the survey might actually expect this, and secondly that the person felt the necessity to warn me of this likelihood.

Sure enough, a few days later I did receive the email and proceeded to open the survey. Equally sure enough, the opening salvo looked like this:

 Redacted Rating

I felt compelled to write to the team managing the survey to express my total dissatisfaction with their understanding of suitable data-gathering. The following is a slightly edited version of that email.

Whoever added the annotation that a 7 or 8 was "neutral" and a 9 or 10 was "extremely likely" has a remarkably limited understanding of maths and statistics.

On any reasonable 0-10 scale, "neutral" is a 4, 5 or 6, while 7 and 8 would be rated as "positive" and 9 or 10, as you suggest "extremely likely". Labelling a 6 as "not at all likely" is both condescending and highly inaccurate, not to mention mathematically misleading.

Any customer interaction will contain room for improvement, therefore insisting (as your annotations do) that the only acceptable rating is a 9 or 10 is the worst kind of virtue-signalling I've seen in a long time. Even worse that the on-side attendant felt the need to set his expectation in advance.

Worse, you are NOT collecting any useful data — all you will get is a collection of 9s and 10s everyone in your organisation is smiling and happy, safe in the knowledge that the entire customer body is satisfied. The marketing team achieves their KPIs and everyone's pay-packet is suitably enriched.

Unfortunately, if there is a level of dissatisfaction, you won't see it. Having been told that the dealership will be "penalised" for low ratings, I would reasonably assume that people will probably not respond (as I am doing) or they will grit their teeth and respond according to your expectations.

I can only assume there is no-one in your team with even rudimentary statistical training, otherwise this would surely never have been foisted upon an unsuspecting public.

By the way, I stepped through the remainder of the pages (with random responses and without submitting) and it became remarkably clear that EVERY QUESTION was directly linked to a specific measure of the performance of the workshop. Questions like, "did they greet me by name," or "was the work to be performed fully explained, along with a break-down of costs."

As they say, "what gets measured gets done," or to quote Goodhart's Law, "When a measure becomes a target, it ceases to be a good measure".

I wrote to the company in the hope that some statistical "moral fibre" might be instilled, but I'm not holding my breath. I write this on iTWire in the hope that the message might be seen more broadly; again, my breath is not bated.

As I noted in the opening, just because you know how to use a tool, doesn't mean you're an expert in the process for which that tool is used. Knowing how to use Microsoft Word does NOT make you a writer and knowing how to create a survey in SurveyMonkey (or any of the other tools available) doesn't mean you're going to get meaningful, actionable data.

Perhaps the company in question doesn't want to learn the truth – if so they've done an excellent job of ensuring that specific outcome, as absolutely nothing useful will be learned from the survey in its current form.

Addendum: Prior to publishing this comment, I caught up with a family member who worked for a multi-brand car dealership in a major Australian country town. His own observations were very similar, noting that they were very close to losing one of their brands due to "low" feedback ratings. He also mentioned that his own mother had told him (I paraphrase both statements), "I thought the service was really good – I gave you 7s and 8s for everything." He replied, "but Mum, but we need 9s and 10s to stay in the car company's good books."


Site24x7 Seminars

Deliver Better User Experience in Today's Era of Digital Transformation

Some IT problems are better solved from the cloud

Join us as we discuss how DevOps in combination with AIOps can assure a seamless user experience, and assist you in monitoring all your individual IT components—including your websites, services, network infrastructure, and private or public clouds—from a single, cloud-based dashboard.

Sydney 7th May 2019

Melbourne 09 May 2019

Don’t miss out! Register Today!



Australia is a cyber espionage hot spot.

As we automate, script and move to the cloud, more and more businesses are reliant on infrastructure that has the high potential to be exposed to risk.

It only takes one awry email to expose an accounts’ payable process, and for cyber attackers to cost a business thousands of dollars.

In the free white paper ‘6 Steps to Improve your Business Cyber Security’ you’ll learn some simple steps you should be taking to prevent devastating and malicious cyber attacks from destroying your business.

Cyber security can no longer be ignored, in this white paper you’ll learn:

· How does business security get breached?
· What can it cost to get it wrong?
· 6 actionable tips



Popular News




Guest Opinion


Sponsored News