Getting from data to insight – best practices in data analytics

Survey after survey tells us that organisations are still struggling to fully leverage big data to get to insight and data driven decision making, despite investing billions in data analytics. One major commonly overlooked reason for this is the difficulty of bridging the gap between pure data analysis and actionable insight.

The gap between data analysis and insight

Getting to insights is challenging because it requires both the science of data analytics and the art of data storytelling. Data storytelling for insight goes beyond pure data analysis since it involves communication tailored to an audience, such as a decision maker. It also goes beyond pure analysis because it must incorporate factors which the data alone cannot reveal no matter how much analysis is applied, such as idiosyncratic knowledge, uncertainty and even intuition.

The challenge of leveraging big data

A recent report by CapGemini The Rise of Insight Driven Business highlighted the importance of using data analytics and insights for action to drive business growth. Yet in an extensive survey they also found that only 27% of senior executives believed their big data initiatives were successful. The survey highlighted data silos and found that 52% of executives believed the IT development process constrained their ability to quickly create insights.

Of course, it is not data or technology alone that constrains insight, but the inability to find the right people who are able to leverage it. As KMPG found in their global 2015 CIO Survey, 36% of technology leaders suffer from a shortage of big data analytics skills, higher than any other function asked about, and also high on the list, 29% say they are short of business analysis skills.

Clearly getting from data to insights is not easy and needs investment in technology and people. But can the lack of investment alone fully explain why organisations are struggling to create data driven businesses? What happens when you have the data and the systems in place, but the actionable insight still feels thin on ground?

Telling stories with data

As Google have reminded us, data analysis is not about graphics and visualisations, it is about telling a story. This means identifying the audience and speaking to their aspirations and concerns. Too many data analysts and analytic platforms create more information from data without clear direction as to precisely why the recipient, the decision maker, should be interested, and what they should do with it.

There is no shortage of data: organisations are drowning in it. Unfocused and poorly communicated analysis just adds more fuel to fire, more information for the time poor executive to navigate as they try to pinpoint what actions to take. Bridging the gap has to be about tailoring the analysis the needs of business decision makers. It is also about making sure the complexities of analysis are made easy to grasp and compelling, and essential points breakthrough for those who need to know. This is data storytelling.

Insight often requires a leap in the dark

For data storytelling to work you need a receptive audience and many rightly stress the importance the creating a data driven culture (for example this post by McKinsey and this one by PWC). Typically the move to data driven culture is understood as move away from decision based on gut instinct and opinion. However as anyone who has tried to use predictive modelling for business forecasting will know, often it is impossible to remove all uncertainty and to resolve the decision problem you have without making some very big assumptions and relying on intuition.

In fact, data analysis and intuition are not mutually exclusive means to making smart choices, rather organisations must be comfortable with exploring problem solving from all angles and understand the limitations as well as the opportunities inherent in their data. Blind adherence to the dictates of data analysis which is patchy, has a margin of error or which cannot be generalised, may be even more dangerous than ignoring data altogether. A post by MIT Technology Review makes this point very well, notably in respect of risky decision making for innovation.

Summing up: Seven steps to get from data to insight

No online article is complete today without a list to summarise the thinking, so here goes – the first four are widely advocated as essential, while the fifth, sixth and seventh are about bridging the data to insight gap –

1. Integrate your data
2. Accelerate big data technology
3. Invest in smart people
4. Communicate insight by creating data stories
5. Tailor your analysis to your business goals
6. Drive for a data driven culture
7. Tolerate some intuition and uncertainty

So, there it is, in just seven easy to understand steps. In practice it will take years and is ongoing process. Getting from data to insight is a major overhaul of how your organisation behaves, requiring commitment, imagination and effort at all levels.

Gabriel Hughes

Can we help you become more data driven?
Please email us at hello@metageni.com