How to test incrementality with MMM

View profile for Dr. Ramla Jarrar

President @MASS Analytics | Marketing Mix Modeling Expert

Great overview of the different incrementality testing tools available to marketers across platforms.    All of these can be integrated with MMM, helping validate the model results, as well as providing insight into channels without much data.   At the same time, MMM can tell you what experiment you should run to measure true incrementality. I talk about how to do this in a dedicated MMM Masterclass episode on triangulation: https://lnkd.in/eNkBcRPG

View profile for Nico Neumann

Last-click attribution is still an option in many platforms, but it rarely provides a true picture of ad effectiveness. While it may be comforting, relying on it is unlikely to drive actual growth. Incrementality tests, in particular AB tests, randomised experiments and geo-holdouts, may not show you large "claimed conversion" numbers. The true incremental uplift is often only between 1-10%. However, this is the truth. Think about this from a sensitivity point of view: If the last-click (or last-touch) attribution numbers were accurate (often suggesting 50-80% sales are driven by one campaign), every company running multiple ad campaigns would need to grow as fast as NVIDIA over the past year. The good news: The same platforms that offer last-click or last-touch attribution also tend to provide various incremental lift tests in many regions. For example: Meta - Conversion Lift and AB Testing https://lnkd.in/gr6nyg7m and https://lnkd.in/gTip8uJ9 TikTok - Split Test https://lnkd.in/gaCiZ-G7 Amazon - AB Test and Brand Lift https://lnkd.in/gF2YBmsN and https://lnkd.in/g5KDzgek Google - Conversion Lift https://lnkd.in/gWCYnwsu Trade Desk - Conversion Lift https://lnkd.in/gWBaaGdS Otherwise, there are also great third-party tools that can be used for incrementality tests. Additional comment 1: The comparison of last-click/ touch attribution versus incrementality tests mainly refers to within-channel/within-platform optimisation. Additional comment 2: Not all split or AB test tools will assign users randomly to different ads or test conditions. Sometimes a targeting algorithm decides on the selection of users too. This means we cannot speak of pure causal incremental effects. Unfortunately, often tutorials and explainers are not clear on the settings and manipulations. That's why it's super important to always check this. One great research study outlining the differences in types of experiments is the following study (link below): Braun, Michael, et al. "Leveraging digital advertising platforms for consumer research." Journal of Consumer Research 51.1 (2024): 119-128.

  • No alternative text description for this image

To view or add a comment, sign in

Explore topics