Modern Marketing Effectiveness Measurement: A conversation between Meta and Metageni

Modern Marketing Effectiveness Measurement: A conversation between Igor Skokan: Marketing Science Director at Meta and Gabriel Hughes: Founder and CEO of data analytics agency Metageni.

Watch the full video interview here or see below for a detailed summary of what we learned –

How is Meta Driving Better Marketing Effectiveness Measurement?

Metageni focuses on providing independent measurement approaches, including Market Mix Modeling (MMM), multi-touch attribution and incrementality testing, to help brands measure campaign effectiveness on Meta platforms. The Metageni team are therefore keen to understand what Meta is doing to support improved measurement. 

The Marketing Science team at Meta are working on this challenge worldwide. They assist brands in measuring campaign effectiveness, comparing performance across channels, and transforming marketing practices based on data and science. The focus areas for Meta’s efforts include: 

1. Providing better data access and granularity for independent measurement. 
2. Improving modeling accuracy and cadence, including open source code for transparency. 
3. Collaborating with partners to advance the industry and methodologies.

The conversation that followed considered a Harvard Business Review article co-authored by Meta supporting this resurgence of MMM and arguing that that it can be complemented with incrementality testing within a robust measurement framework. So is MMM is the future of social measurement and what do brands need to know about this and related methods?

Growing Interest in MMM and the Launch of Meta’s MMM API

Meta have recently release a Marketing Mix Modelling (MMM) data access feature through an API. This feature has expanded to anyone with a Facebook Business Manager account and knowledge of how to use the API. This release coincides with a growing interest in MMM for social measurement.

The goal of the API release is to support the trend towards more MMM, by making high-quality data more broadly available in the required frornats, promoting transparency and enabling better econometric models. Meta emphasize the importance of high-quality data as the foundation for constructing accurate models. The API release also represents a step toward facilitating more frequent high-quality modelling. Feedback from the market will be used to improve and refine this MMM feature in future iterations.

Modern Market Mix Modelling: The Renaissance 

Some key reasons for the resurgence of Marketing Mix Modeling (MMM) are because it addresses some newly arising the challenges and opportunities in marketing measurement: 

1. The availability of high-quality digital data that is not being fully utilised. 
2. Challenges posed by changes in tracking methods, such as the deprecation of third-party cookies and Apple’s privacy measures. 
3. The incorporation of modern data science techniques into MMM, making it more dynamic and granular.

These factors have led to increased interest in from data scientists, open-source contributions and tools, and growth in commercial offering to support MMM analysis.

The Challenges of MMM 

Challenges in utilizing MMM include the need for a clear mental model of the business, complete data sets, multicollinearity, and potential bias in model specification.

The accuracy of the Market Mix Models are improved through a rigorous process of incrementality testing in the context of marketing effectiveness .Market Mix Modelling is like a toolkit that requires expertise and careful consideration to use correctly. While automation can handle certain aspects, human input is still necessary today for defining variables, exploring potential spaces, and validating and interpreting models. It is important to emphasize methodological rigor in model development, suggesting that while different analysts may produce different results, a systematic relationship should exist between input variables and model outcomes. 

Without understanding these nuances, it is very easy to produce a ‘bad’ model i.e. one that produces results giving a very inaccurate picture of marketing effectiveness. It is therefore incredibly important to have the right level of expertise as well as high-quality data access, in order to apply best practice and develop sound Marketing Mix Modeling which can be trusted to guide strategic decisions.

Incrementality Tests are the Gold Standard of Marketing Effectiveness Evidence

Meta continue to offer “conversion lift,” a product by which allows advertisers to execute incrementality tests on Facebook and other platforms. Incrementality testing with these kinds of tools is considered a gold standard for establishing cause and effect in marketing effectiveness, and it can be used alongside Market Mix Modelling to validate model estimates.

The conversation underscored the value of both systematic model development and empirical validation through real-world tests, such as incrementality testing, to ensure accurate and meaningful insights into marketing strategies.

Understanding the value of Incrementality Testing on Market Mix Modelling and Practical Steps to Implement Learnings

Market Mix Modeling involves analyzing the effects of marketing activities on sales. The “baseline” represents the sales a company would achieve without any marketing efforts, accounting for factors like repeat customers, economic conditions, seasonality, and external drivers. It is essential to incorporate this baseline to accurately estimate the contribution of marketing activities.

Incrementality testing is a method for measuring the true impact of marketing campaigns by comparing a control group (not exposed to the campaign) with a treatment group (exposed to the campaign). As such it is a more accurate way to estimate the baseline for the specific marketing activity being measured, at that point in time. This approach allows marketers to determine the incremental effect of the campaign beyond the baseline, providing more reliable insights into the campaign’s effectiveness.

Methods of Incorporating Incrementality Testing Results:

• Simplistic Approach: Compare the incrementality test results with the output of a Market Mix Model to ensure alignment. If the results differ significantly, further investigation into the model’s accuracy is required.
• Guiding Principle: Use the incrementality test as a guiding principle when selecting the most appropriate model among candidate models. The model that best aligns with the incrementality test’s results may be chosen, even if it has slightly lower statistical quality.
• Complex Methods: More advanced approaches involve incorporating incrementality test results as priors or constraints in regression models, enhancing the accuracy of the modeling process.

The Trifecta of Marketing Measurement: the Role of Attribution

Market Mix Modelling (MMM), Incrementality Testing and Multi-Touch Attribution (MTA) roles are evolving in marketing analytics and seen as the ‘trifecta’ of Modern Marketing Measurement. Data granularity is still incredibly important and the need to go deeper into variables for better understanding. 

• While incrementality testing provides robust insights, it is not scalable due to the need for control groups and adjustments to media plans.
• Multi-touch attribution (MTA) is another approach to measure the impact of various marketing touchpoints on customer journeys, but it has faced challenges with the decline of third-party cookies. However, using first-party data, it is possible to analyze sequences of clicks and understand the paths that lead to a final purchase.
• MTA remains relevant for multi-channel click based marketing measurement and for much more granular analysis than can be provided with MMM, for example, combining social marketing with PPC and Affiliates. Some approaches, including Metageni attribution, incorporate social and display impressions into MTA models using statistical methods similar to MMM.

Overall, across these 3 types of measurement, no one method conclusively addresses all the marketing measurement problems required to make smart data driven marketing decisions that maximise ROI and sales potential. A hybrid of these methods is therefore inevitable, especially for brands with larger marketing budgets working across multiple marketing channels.

But Will an AI do All Your Marketing Measurement One Day?

This was a fun topic to end on, but an idea that does not now seem so fanciful given recent advances in AI.

Meta (formerly Facebook) is working on improving data availability and third-party measurement programs and the potential for machine learning-supported, semi-automated, or fully automated MMM models in the future. Challenges remain around  data integration and validation and it is important that a human acts as the  “piano tuner” to maintain and calibrate measurement systems. 

Metageni is also very interested in the use of AI for analysis and is investing in this area. For now, brands still need experienced analysts to make sense of their marketing, but AI tools are already accelerating the pace and quality of marketing measurement today.