Yahoo reports, with credit to Precedence Research, the worldwide data analytics market at $41B USD in 2022. This investment in data is not too shabby. But, if you think that number is impressive, consider their prediction that the industry will grow 30% per year (CAGR) to $346B by 2030.
To put it in perspective, consider the international markets for everyday consumer goods such as bicycles, underwear, and booze. Believe it or not, we will spend more on analytics over the next 7 years than on bicycles, men’s boxers and briefs, and vodka markets combined. And that’s just the software. All the labor applied to analytics, buried in payroll, blows that number away.
The idioms are bountiful. We have drunk the Kool-Aid and taken the bait. Decisions by facts, and a pursuit to become a data-driven organization. Who could possibly argue with that? Every company, university, healthcare provider, and non-profit is seemingly invested in data analytics to one extent or another.
But what does your personal experience tell you about the payoff from an investment in analytics? Is it always obvious what all this data is doing for you? Do you ever wonder, “should we spend more on vodka and less on analytics?”
Gartner was one of the first to give us pause, if not downright rain on the parade. About six years ago, they estimated that 60% of data projects fail to go into production and generate any value. A few years later, Gartner analyst Nick Heudecker infamously said that 60% was far too conservative an estimate… and the real number is more like 85%. In present day, the general pattern persists.
Big Data Mining In The Wrong Cave?
Forbes council member and SVP at Course5Intelligence, Ajith Sankaran, reminds us that the prevailing theories of why-we-fail include lack of a cohesive strategy, application of AI and analytics to the wrong projects, poor organizational alignments, and lack of continued C-suite commitments. Or is it about the actual data? Is it an absence of adequate governance? How about data quality? Surely, that can trip us up. Heck, is the data we need even available?
Harvard Business Review authors Ross, Beath, and Quaadgras postulate that investments in big data projects fail to pay off because companies don’t do a good job with the information they already have. They don’t know how to manage it, analyze it in ways that enhance their understanding, and then make changes in response to new insights. “Companies don’t magically develop those competencies just because they’ve invested in high-end analytics tools.”
With no shortage of authors standing ready to opine on the subject of failure, it’s easy to conflate insights with excuses, and give executives a migraine in an effort to get to root cause. Yet that’s exactly what executives seek. Why can’t we materially and consistently affect business outcomes from an investment in data? It’s a simple question, if you are a CFO or CEO.
But no sooner is the question asked, when those closest to the data (in the IT sense) make it all the more complicated. You don’t understand, Ms. CEO, all of the moving parts that have to be pulled together to mine our data. We need ETL and new cloud-based environments. Migrations are in order to turn off legacy systems. New tables must be built and new software must be purchased.
It can be easy…really easy, for an organization to get lost in the detail. Before you know it, your company has resources all over the org chart building data objects for analysis. They argue with each other about systems and tools, process and access. They may even argue about the data itself, “Mine is correct. Yours is wrong.”
Odds are that some in your firm get great benefit from the data they analyze. But as a wholistic matter, your company may look and feel like a lot of others in coming up short. You spend way too much money on data related projects, personnel, software, and services. But the needles that define your business don’t seem to move in kind.
It’s an expression of competency, really. As a company, how competent are you at analytics? It is a tricky question, in that consumers of data often blame the techies for being too slow or out of touch. While the techies blame the business people for being vague, incomplete, or flip-floppy with their requirements. Cynical managers believe they’re doing a good job and the other department is slack. To gain competency in data analytics in the aggregate requires more than the sum of the parts. It takes communication, commitment, and targeted business process to break through.
Role Models Of Quality Analytic Pursuits
Interestingly, there’s a think tank called the International Institute of Analytics that keeps track of analytics competencies across industries and regions. Sort of like the Moody’s for analytics, they assign scores for individual firms, roll up the data for whole markets, track relative performance, and closely watch market trends. It can be incredibly helpful to comprehend where your firm is relative to competitors, for example. This is the IIA schtick.
There are, thankfully, published examples of those doing analytics projects right. Clues for success can be found in this hot-off-the-press interview with the VP of Analytics & Data Science at DoorDash, Jessica Lachs, conducted by the Wall Street Journal. She explains, “It’s all about our desire to measure as much as possible. When we roll out a new product feature or program to customers, we can run an experiment and actually quantify the true impact that it had on the business. We are keeping a watchful eye on everything that’s happening in the market, particularly as it relates to inflation and consumer softening. We empower our CFO to make good decisions by building out what we call the DoorDash item price index.”
Go back. Read this paragraph again. There are some powerful concepts in her testimony. First, it’s business and all business. Second, it’s all very direct and unambiguous. Third, there is a prescribed resolution in the form of action…in this case, the CFO-ready item price index, applied to forecasting and pricing optimization in a squishy economy.
Business focus. Unambiguous objectives. Actionable outcomes.
Sounds simple enough. Senior business managers/executives routinely “get it” when the themes of success are put in this expression. But the day-to-day execution by our diverse, intertwined organization demonstrates that, in fact, we don’t get it. We chase challenges in source data ingestion, data transformation, data modeling, architecture, documentation, security, wrangling, and seemingly everything other than business focused, unambiguous objectives with actionable outcomes. It’s no wonder we get in the way of ourselves. The detail obfuscates the mission.
How Data Leads to Action
There’s a tried-and-true representation of the maturity stack for analytics that has been widely published over the years. I refer to this one at eCapital Advisors as a convenient, accurate rendition. It starts with “what happened,” a reporting agenda, then moves to, “why did it happen,” for diagnosis. This leads to predictive analytics, “what will happen?,” followed by…drumroll, please…the too often missed step of “what do we do about it?” The stairstep evolution is only complete when we embrace the full progression towards action, rather than being content with observations.
For me, it’s a matter of bending the future. It’s not good enough to predict the future. We want to bend it to our liking. That’s the action. If patterns in data reveal that certain customer behavior leads to sales, or attrition, or some other business knob we seek to turn, then let’s trigger off those patterns. The onus is on us to create better outcomes, based on what the data tells us, rather than hoping things somehow fall in line on their own.
We can apply this thinking to pricing (i.e., the DoorDash example), cross product upselling, targeting new customers, customer satisfaction, instrumenting sales process for better close rates, and a host of other measurable business objectives.
And at some level, the implementation details are noise. Do we choose ETL or ELT? It’s a means to an end. Pick one and get on with it. But be smart. Wherever possible, minimize manual coding efforts via automation. Adopt a nimble, agile approach to development and deployment. Leverage technology, but avoid traps wherein the tech choices cast off a second order set of excuses that compromise behavior and tolerate delay. In short, remain vigilant to relentlessly serve business needs.
What is your firm’s itch? If we can define it, confront it, and set a target for a better outcome, then surely an investment in data analytics can uncover the truth and set a course for action that will pay off. All the rest of it that is not business focused, ambiguous, and void of action is nice, I suppose. But be careful. Don’t fall into the Gartner 85% without a parachute. Contact us if you want some help with your data goals this year.