Media Mix Modeling is gaining momentum, no doubt about it. But what kind of data do you actually need to be confident in the model’s results?
If you’re not new to Media Mix Modeling, you already know that the primary input is, of course, your own marketing and sales data. But dig a little deeper, and you’ll quickly find a growing conversation around additional types of data that should be included in your model, namely, control variables.
What are control variables in MMM?
To begin with, control variables are defined by Chan (2017) as weather, seasonality, and market competition data points used in media models. Earlier versions of MMM were relying heavily on control variables. As the MMM adoption increases control variables become easier to find and incorporate.
The literature does acknowledge the necessity of control variables in media models. However, research is lacking when it comes to best practices and the actual impact of including (or excluding) control variables. In this article, we’ll try to shed some light on the topic.
Why MMM require control variables?
Econometric models require control variables to avoid false conclusions. Without them, the model may wrongly attribute changes in outcomes (like sales) to advertising, when they’re actually caused by other factors like seasonality, trends, or competitor actions.
The practitioners challenge now is to understand what are the right control variables for an MMM. Sadly the fit of the model is not really a great indicator that you can chosen the right control variables. Although, it is usual practice most of the times.
An example of an MMM with and without control variables.
Are control variables mandatory for a Media Mix Model?
The short answer is no. All open-source software can run without control variables. The real question is what kind of results you are getting with and without control variables on an MMM.
But the real question is: Are control variables good to have? The answer is yes. Control variables are great to have. Control variables help your MMM create the right trend. In some cases it is very important for an accurate representation of your baseline. On top, they support support justifying the scale of your baseline which is a very essential component of MMM.
The challenge with control variables for MMM is where can you find them and how much you can trust them.
What kind of control variables can you use for your Media Mix Model?
It depends on the industry and your stage as a business.
In an ideal world, you’d have access to competitor data, industry growth trends, seasonality, behavioural insights, and major market events that could influence shopping behaviour. But the reality is, securing those datasets is often extremely difficult especially for smaller or newer businesses. It was the case for many years (prior to Robyn / Meridian) that control variables were the biggest blocker for an MMM. This has changed.
For example, if you’re a FinTech operating in Europe, you might be able to find useful datasets through official EU portals. The European Union offers access to over 1.9 million national datasets, and most countries also have their own national statistics services. In this case, sourcing high-quality market or industry data should definitely be a priority.
Another option is to subscribe to survey or data insight platforms like Statista. However, it’s important to note that these platforms can be quite expensive and may not always fit early-stage budgets.
What is the difference in MMM with and without control variables?
Perhaps the most important question. What is actually the difference between a Media Mix Model with and without control variables. To answer this question, we build two models. Both with the same parameters and of course dataset.
The difference is remarkable – Below we use an example dataset and with the support of Meridian we run two models:
MMM with control variables
MMM without control variables
In our example above, the overall accuracy hasn’t been significantly been affected. However, it’s worth noting that the absence of one or more control variables can sometimes impact the model’s accuracy affecting both the error metrics and the R-squared. In other cases, the absence of control variables may distort the trend rather than the accuracy. The trend is also important for budget optimization scenarios.
In our case, the most noticeable impact is on the baseline of conversions.
Taking a step back: in Media Mix Modeling, whether you’re presenting results to clients or discussing findings with colleagues trust in the output is essential. Even if the model performs well statistically, if the baseline is consistently negative or shows no trend at all, your credibility as a modeler may be questioned.
Remember, your job isn’t just to build an accurate model. It’s also to make sure that the results are interpretable, reliable, and aligned with real-world expectations.cult.
What kind of control variables can I use for my MMM?
Step 1: Create a seasonal control variable
One simple way to create a control variable for your Media Mix Model is by capturing seasonality. Seasonality is a common non-media factor that can affect conversions, like increased demand during holidays or weekends. To control for this, you can decompose your time series data. There are plenty of libraries that can help you do that. This allows you to extract the seasonal component of your target variable (e.g., conversions or app downloads) and add it back into your dataset as a new feature (e.g., seasonal_conversions). This seasonal signal can then be included in your regression model, helping you isolate the true impact of your media channels.
Step 2: Find available data sources (free of charge)
Although creating control variables can be time-consuming, there are plenty of publicly available data sources you can leverage. Let’s take an example: suppose you’re building a Media Mix Model for a retail business based in Los Angeles. A useful control variable could be the number of new stores opened in the same industry and region. This external factor could influence consumer behavior and overall market demand, independent of your advertising efforts.
Datasets like this are often available through open government portals, for instance, data.gov offers access to business registration and economic activity datasets. However, to make this information suitable for MMM, it needs to be pre-processed. That includes aggregating it by time period (e.g., weekly or monthly), aligning it with your campaign dates, and ensuring it’s in a format compatible with your modeling pipeline. Incorporating such external control variables helps your model account for market-level changes, offering a more realistic estimate of your media’s impact.
Step 3: Gain access to paid datasets
Traditional Media Mix Modeling (MMM) required access to extensive datasets particularly around market growth, competition, and macroeconomic indicators. However, with the rise of open-source tools like Robyn and Meridian, more businesses are now adopting MMM as part of their marketing strategy. As a result, modeling approaches have become more flexible and accessible.
That said, incorporating high-quality external datasets can still significantly improve model accuracy and ultimately business outcomes. For example, if you’re an eCommerce brand operating in the UK, adding a Statista dataset on national eCommerce sales can enhance your model by grounding it in broader market trends. While platforms like Statista can be expensive, even a single dataset can be a worthwhile investment for improving your model’s explanatory power and credibility.
Lastly, If you need help with your dataset, reach out to MarSci, and we would be happy to help.
Email us at: info@mar-sci.com or access the MMM platform here.
Literature:
- Jin, Y., Wang, Y., Sun, Y., Chan, D. and Koehler, J., 2017. Bayesian methods for media mix modeling with carryover and shape effects. Google Research, pp.1-34.
- Chan, D. and Perry, M., 2017. Challenges and opportunities in media mix modeling. Google Inc, 16.
- Chen, A., Chan, D., Perry, M., Jin, Y., Sun, Y., Wang, Y. and Koehler, J., 2018. Bias correction for paid search in media mix modeling. arXiv preprint arXiv:1807.03292.
- Wang, Y., Jin, Y., Sun, Y., Chan, D. and Koehler, J., 2017. A hierarchical bayesian approach to improve media mix models using category data. Google Inc., research. google. com.