On the future of analytics
If you’d asked me what the future would hold for the analytics industry back when I started as a baby-faced analyst, I’d have said, “like, whatever yeah? I don’t care, because like, I’m mainly focused on my music yeah? So…this is like a temporary thing or whatever”.
Nowadays, having spent the intervening fifteen years very much not being a rockstar, I can probably have a better stab at where analytics might be headed. Which is lucky, because I do keep getting asked to predict the future by wafting, wide-eyed management types, and as such I’m going to rummage the back of the office stationary cupboard, dig out my robe and wizard’s hat, and give you a glimpse into my crystal ball of speculation.
So here, in no particular order, are my top 5 “things that will definitely probably maybe be the future of analytics”, based on anecdotal evidence, wild flights of fancy and maybe a smidgen of experience (all things that I absolutely wouldn’t rely on to actually deliver any analysis. Except maybe the last one. Obviously).
- It’s going to get more complicated.
The profusion of tracking software platforms, the endless expansion of data processing systems and their capacity to store and access more and more things about businesses and their customers is only going to continue to spiral. Business has woken up to the idea that they can use customer data and so should make sure they are collecting as much of it as they can, but at the moment it’s more of a rush to capture the data correctly than a real understanding of what value is going to be extracted from it. Give it five years and almost every company will be sitting on vast repositories of web analytics, CRM, social media, operational, transactional and who-knows-what else kinds of data available. The risk here is that all that data needs a whole lot of people with differing skills to analyse in different ways – the data science teams will move the macro around, doing the big data stuff that needs statistical analysis, but data science tends to be too involved and too slow (if it’s done properly) to deliver smaller scale business insights, so you’ll be needing specialists in each of the sub areas too. Then there’s the fact that all data is not created equal, and just because you’ve gathered it doesn’t mean it’s useful, so there’s a pretty good chance some pretty big inefficiencies might creep in. Are you confused yet? Because you will be.
- It’s going to get more fragmented.
As companies get more and more au fait with their own data, it stands to reason that they’re going to want more and more bespoke solutions to their questions. This is already leading to a diversification in the tools businesses are using to store, enrich and analyse their customer information – it used to be that one primary analytics platform and an off-the-shelf data warehouse were likely all that were used, and that caused enough “where’s my data” type problems, but now even a small business is likely to have multiple systems from multiple suppliers that may be cloud-based or in-house, integrated or separate, siloed or open. This means that analysts looking for holistic views of company data (like us) are going to need a broader and broader range of knowledge in order to get to the crunchy insights hidden within this vast proliferation of data lakes and cloud services, potentially accessing them using skills that were not considered by traditional analytics – SQL, Python, API scripting, DBA management, R, D3 and so on ad infinitum. I predict that in five years there will be no single template that can be applied to solve the question of “how do I get insight from my data” – every attempt will require a bespoke approach.
- There will be a backlash against points 1 and 2.
If everything I’ve outlined around the complication and fragmentation comes to pass (and it will, trust me, probably) then I suspect there will be a backlash from the business end users. There’s a tendency to forget that unless end users can understand and make changes based on analysis, that analysis is essentially pointless, no matter how rigorous and complex it was. Let’s face it, Data Science isn’t necessarily a relatable subject, especially when you remember you’re trying to make quick decisions in a fast-paced marketplace, not publish a hyper-accurate research paper. My suspicion is that as analysis becomes bigger in scope, but also less human in its language, teams on the ground are going to feel left behind, because they still need answers to the same questions they always have: How is my area performing? Are the changes I’m making improving sales/customer experience/whatever? I predict new technologies coming forward (likely led by a resurgence for Google Analytics) with claims of simplicity and efficiency, distilling and translating massive bespoke and complex datasets into simple business user views.
- Data privacy is going to become a much bigger issue.
This one is the curve-ball. Data privacy has been a growing issue over the past couple of years, with governments playing hot-potato between its benefits to business vs the privacy of the individual. Legislation like the EU’s GDPR regulations have tightened controls on what data can be collected and how, but the cynic in me suspects that legislation will remain fairly lax, allowing companies to gather progressively more and more information about their customers. The risk is that this ends up becoming a morality issue for business, where the negative impact on brand image and public relations from data collection, or worse, leaks of personal data far outweigh the value that can be added through its analysis. I expect more than one big name company to get into severe hot water in the next few years, and for this to lead to a kind of “self-policing” set of guidelines to develop, heavily restricting what data is collected, because the risks of getting caught out are just too great.
- I will be hailed as the all-knowing Nostradamus of analytics, and all companies will hire Station10, while speaking of our near-omniscient grasp of all things analytical in hushed and reverent tones. I think this one is fairly self-explanatory.
So there you go, that’s what I think the future looks like. No need to thank me, I already know you will later.