Data data everywhere but not a KPI to drink
Category : Blogs
The world we live in is currently being flooded with a tsunami of data – from both online and offline worlds. With the continued development of wearable tech, along with other new and improved methods for gathering data, this tsunami is not just washing over the business world, it is leaking into other realms such as our personal lives and sports activities.As Andy pointed out in in his latest blog [link], if you collect too much data, you are at risk of collecting the wrong data. That will inhibit your decision making rather than improve it.In the business world (as well as in many others) we often make decisions based on the data that is readily available to us, simply because it is there. Any data is better than no data, right? But we often do this without considering what the data itself really means.
Any Data Is Better Than No Data, Right?
A case in point: I’m not an Arsenal fan (in fact, I still haven’t forgiven them for beating us in two cup finals in one season!) but I remember watching an interview with Jurgen Klopp, the Borussia Dortmund manager, before a champions league game a few seasons ago.Aware that a repeat of the victory two weeks before would put his team in a position to qualify, he triumphantly explained how they’d achieved that 2-1 win. What seemed to excite Klopp the most was a statistic that revealed that his players had run a collective 11.5km more than their opponents.“Coaches will say that it’s not important for their team to run more and they prefer to make games the right way,” Klopp says. “I want to make games only the right way and run 10km more. It’s a rule to give all and it can make the difference if you work more.”11.5 K more, collectively, than their opponents. That’s a huge number – and certainly a lot of work from his team.But is it? In the game there are 10 outfield players. So that’s only 1.15 k more per person. If you factor in substitutes, that’s potentially 13 outfield players. That equates to just under 900 meters each (I could factor in positional differences, but that would get quite boring quite quickly!).So, the figure that Jurgen Klopp looks at shows that his players, on average, run 900 meters more than their opponents. Again, what does that mean for his team?Running 900 meters more means:
- They will get more tired than their opponents
- This fatigue will ultimately result in fewer spurts of lung-bursting, Lampard-esque running to score a goal
- It also increases each player’s chance of injury
When you look at it this way, does that 11.5k actually reflect positively on performance? Probably not. And that’s before we even get into correlations and causations.Is this a statistic that can be (relatively) easily collected with current technology, and shared quickly – almost definitely!
Not all stats are created equal
Now, lets take a different sport: Rugby. The world cup is playing out, which means we are going to get bombarded with statistics in every game. Some of this data will be good – and some will be bad. But when looking at the numbers, its important to think about what they actually mean. What do the figures actually show when analysed together?As the games unfold and the teams are evaluated, you will inevitably hear about ‘pack weight’. It’s easily collected (mark it on a scale, or an abacus, and you’re there!), but what does it really signify? Does a sizeable ‘pack weight’ guarantee success?In the scrum, possibly (you’ve got force on your side if you’re heavier). But in modern rugby, where agility and motion are key to realising success, does having heftier players necessarily result in triumph?Bigger doesn’t always equal better. But it isn’t until you put all the numbers under proper scrutiny that you find that answer out. This is where analytics comes in – and this is where the real value of the data that you have collected, really lies.Another, and my favorite stat, is ‘yards after contact’. It’s a compound stat in that it’s formed of 2 parts. Those components are:
- The point of contact between two players
- The distance that a player covers afterwards
As a stat it’s quite distinct. I like it because it can’t have multiple meanings, but it is indicative of a number of things. It tells you a lot about distance made, punishing contact, and momentum – as this article points out.Again, it is the effective analysis of all these numbers that makes them mean something. In business, it’s the same.In business, you have to find the right metrics to measure, too – and look at those numbers with the same critical eye. There are many measurements that are taken for the sake of quantification (like marking up ‘pack weight’). But they don’t necessarily prove anything until you put them into context.
Data must be contextualised
This is true of all statistics: data must be contextualised.At Station10 we use a process called ‘value mapping’ to do this. This approach links all behaviour, strategy and tactics back to the wider business goals. It very quickly reveals which, of all the metrics that are being used for decision making or evidencing performance, don’t in actuality show much.It highlights those stats that don’t relate back to the overarching business goals – and in turn exposes where gaps and opportunities in your data strategy might lie.In essence, you need to be able to interrogate data – not just collect it. You need know the right questions to ask to get answers that signify something.
- What are we trying to understand or monitor?
- Do these numbers help answer specific questions?
- How does this data help us to meet specific objectives?
- Do these figures highlight issues and opportunities?
- There’s data, data everywhere. But what does it actually mean?