Posts

Just how bad is your analytics data?

If you do not know how bad your analytics data is, then the chances are, it is much worse than you think. With data analytics, it is not the known data quality issues that will cause you the most trouble, not the known unknowns, but the ‘unknown unknowns’ – those issues you uncover and discover as you explore and analyse your data.

Usually, it is only the practitioners who are very close to the data who understand the full extent of the data quality problem. Too often the poor quality of data is kept as something of a dirty secret not to be fully shared with senior management and decision makers.

Common issues in web and marketing analytics

Let’s look at just some of the most common issues affecting web and marketing analytics data. To begin with, do not assume that the data sources provided by the most common analytics solutions are robust by default. Even the best ones are prone to big data quality issues and gaps. Take Google Analytics referrer traffic, which often reports an unrealistic level of ‘Direct’ traffic, supposedly visits made directly through users typing in URLs, or bookmarking, both low frequency methods of site access. The reason is that ‘Direct’ is, in fact, a default bucket used where no referrer data is available to the analytics server.

Experienced web analysts know that high levels of direct traffic usually mean high levels of broken or missing tags, or other technical issues, that means the true referrer data has been lost.

The major providers are also contributors to that other major source of poor data quality, which is fragmentation and disjoint data sources. Google search marketing tags will track conversions, but only from the Google search and display network. Facebook similarly provides tags which only link Facebook marketing to sales, ignoring all other channels. Affiliate networks do the same thing leading to widespread duplicated over attribution of sales to multiple sources. This challenge is exacerbated by different marketing attribution platform look back windows and rules which are different between platforms.

Having worked with multiple brands of all sizes, I have yet to come across a brand that does not have some level of tagging issue. A typical issue is a big mismatch between emails delivered and emails opened and clicked. Another is social campaigns which are delivered by 3rd party solutions and then appear as referral sources, due to the use of redirect technology.

Tagging and tracking

Tag management systems help manage this, but unfortunately not by linking the data, just by de-duplicating tag activity at source, which is hardly satisfactory if your goal is to understand Multi Touch Attribution (MTA) and marketing channel synergy.

Assuming you solve all your tagging issues and have well-structured soundly applied tags, you should not forget that the tag is only as good as the tracking itself. A great challenge here is the gap that exists tracking users across devices. You cannot link visits by the same user on different devices without sophisticated tracking that users have signed up to beforehand. This means your tags cannot tell the difference between the same user visiting twice on two different devices and two different users.

The idea every one of us can be closely tracked and monitored online is an illusion for all the biggest technology companies – and perhaps we should be glad of that. Indeed, unique ID tracking and linking is now more closely under scrutiny the age of data security breaches, increased concerns over user privacy and the GDPR. This is yet another source of difficulty for companies looking for a 360-degree view of the customer. Companies have to work with fully consented and well-defined samples of data to make progress in understanding their customers.

For the analyst, this is yet another reason why having huge volumes of data is not enough for user insight and data-driven decision making.

So what can you do about all these data quality challenges?

Data quality is perhaps like muscle memory in sport. You use it or you lose it. It’s only by trying to analyse and find patterns in your data that you uncover the issues that need to be addressed. Where there is a need, strategies can be devised to manage these gaps in data quality and take steps for improvement. It is a process.

The best advice is to get stuck in. Pick one data source and run with it, making sure to compare it to others and ask if the data makes sense given what you know about your customers. There are always discrepancies between data sources which in theory should report the same numbers: in my experience, this is a kind of law of all data analytics, so you need to get used to it. Use these differences to help you validate your sources, understand why differences might arise, and just accept that there is an acceptable level of difference – say 2-3%.

In data analytics, as in life, you must not let perfect be the enemy of the good. Be wary of the massive data technology project which promises to link all data together in one big data lake and thereby solve your challenges. Bad data plus more bad data does not equal good data. Face up to your terrible data quality, and tackle the ugliest issues head-on. If you ignore the problem, it can only get worse and you will continue to struggle forwards in the dark.

Gabriel Hughes PhD


Can we help unlock the value of your analytics and marketing data? Metageni is a London UK based marketing analytics and optimisation company offering support for developing in-house capabilities.

Please email us at hello@metageni.com

Getting from data to insight

Survey after survey tells us that organisations are still struggling to fully leverage big data to get to insight and data driven decision making, despite investing billions. One major commonly overlooked reason for this is the difficulty of bridging the gap between pure data analysis and actionable insight.

The gap between data analysis and insight

Getting to insights is challenging because it requires both the science of data analytics and the art of data storytelling. Data storytelling for insight goes beyond pure data analysis since it involves communication tailored to an audience, such as a decision maker. It also goes beyond pure analysis because it must incorporate factors which the data alone cannot reveal no matter how much analysis is applied, such as idiosyncratic knowledge, uncertainty and even intuition.

Data-to-insight-2

The challenge of leveraging big data

A recent report by CapGemini The Rise of Insight Driven Business highlighted the importance of using data and insights for action to drive business growth. Yet in an extensive survey they also found that only 27% of senior executives believed their big data initiatives were successful. The survey highlighted data silos and found that 52% of executives believed the IT development process constrained their ability to quickly create insights.

Of course, it is not data or technology alone that constrains insight, but the inability to find the right people who are able to leverage it. As KMPG found in their global 2015 CIO Survey, 36% of technology leaders suffer from a shortage of big data analytics skills, higher than any other function asked about, and also high on the list, 29% say they are short of business analysis skills.

Clearly getting from data to insights is not easy and needs investment in technology and people. But can the lack of investment alone fully explain why organisations are struggling to create data driven businesses? What happens when you have the data and the systems in place, but the actionable insight still feels thin on ground?

Telling stories with data

As Google have reminded us, data analysis is not about graphics and visualisations, it is about telling a story. This means identifying the audience and speaking to their aspirations and concerns. Too many data analysts and analytic platforms create more information from data without clear direction as to precisely why the recipient, the decision maker, should be interested, and what they should do with it.

There is no shortage of data: organisations are drowning in it. Unfocused and poorly communicated analysis just adds more fuel to fire, more information for the time poor executive to navigate as they try to pinpoint what actions to take. Bridging the gap has to be about tailoring the analysis the needs of business decision makers. It is also about making sure the complexities of analysis are made easy to grasp and compelling, and essential points breakthrough for those who need to know. This is data storytelling.

Data-to-insight-3

Insight often requires a leap in the dark

For data storytelling to work you need a receptive audience and many rightly stress the importance the creating a data driven culture (for example this post by McKinsey and this one by PWC). Typically the move to data driven culture is understood as move away from decision based on gut instinct and opinion. However as anyone who has tried to use predictive modelling for business forecasting will know, often it is impossible to remove all uncertainty and to resolve the decision problem you have without making some very big assumptions and relying on intuition.

In fact, data analysis and intuition are not mutually exclusive means to making smart choices, rather organisations must be comfortable with exploring problem solving from all angles and understand the limitations as well as the opportunities inherent in their data. Blind adherence to the dictates of data analysis which is patchy, has a margin of error or which cannot be generalised, may be even more dangerous than ignoring data altogether. A post by MIT Technology Review makes this point very well, notably in respect of risky decision making for innovation.

Summing up: Seven steps to get from data to insight

No online article is complete today without a list to summarise the thinking, so here goes – the first four are widely advocated as essential, while the fifth, sixth and seventh are about bridging the data to insight gap –

  1. Integrate your data
  2. Accelerate big data technology
  3. Invest in smart people
  4. Communicate insight by creating data stories
  5. Tailor your analysis to your business goals
  6. Drive for a data driven culture
  7. Tolerate some intuition and uncertainty

So, there it is, in just seven easy to understand steps. In practice it will take years and is ongoing process. Getting from data to insight is a major overhaul of how your organisation behaves, requiring commitment, imagination and effort at all levels.

Gabriel Hughes


Can we help you become more data driven?
Please email us at hello@metageni.com