Plenty of vendors will happily sell you Marketing Mix Modelling (MMM) as something you refresh weekly – a live dashboard, updated in near real time, telling you how each channel is performing right now. It has the shape of progress. In my view it is, at best, a niche solution, right in a very small number of cases. At worst, it is a route to bad decisions.
Start with the question underneath it: what can a business genuinely change, strategically, on a week’s notice? Almost nothing. A retailer doesn’t rebuild its category priorities on a Wednesday because Tuesday’s numbers moved. A brand doesn’t shift its channel mix because last week’s model run looked slightly different from the week before. Any business that actually ran its strategy this way would be reckless, not responsive – and everyone in the room would know it. Strategy operates on a planning cycle for a reason: the decisions it produces are meant to hold for the longer term, not to be re-litigated every Monday morning.
So if the model itself is being re-estimated weekly, what is it actually doing? Not answering a new strategic question, because a new one hasn’t arrived. It is producing new numbers from noisy, thin, week-on-week data, and those numbers will move – sometimes a great deal – for reasons that have nothing to do with anything real shifting in the market. The result looks precise. It updates, it responds, it has decimal places. None of that makes it accurate. It is false accuracy wearing the clothes of rigour, and the confidence with which it’s usually presented is exactly what makes it dangerous.
Think about how little one week adds. A model built on three years of weekly data has around 150 weeks to learn from; one more week is less than 1% of the evidence. If the answer changes materially because of it, the model isn’t learning something new – it’s unstable. Two things usually drive the swings. Channels that tend to run at the same time, TV and online video being the classic pair, give the model room to shift credit between them from one run to the next without the overall fit changing much. And advertising works with a delay, so this week’s sales reflect months of activity. A single extra week simply doesn’t contain the information needed to tell you which part did what.
The swings also have a cost. A team that acts on them moves budget towards a channel one month and away from it the next. Every move disrupts campaigns and agency plans, and the numbers keep moving anyway, because nothing real has changed. Before long, people stop trusting the model altogether – including the parts it got right. That loss of trust is much harder to rebuild than any budget line.
This is not a criticism of the statistics. I’d guess most weekly-refreshed models are built on Bayesian methods, and there is nothing wrong with Bayesian MMM – it is a genuinely strong approach, used properly. The fault isn’t in the technique. It’s in the cadence. A statistically sound method, run far more often than the underlying strategic question actually changes, produces numbers that are technically defensible and practically meaningless, and the two get confused constantly.
None of this means weekly data is useless. Adding each new week to check that the model still predicts sales well is good practice, and if its forecasts start to drift, that’s worth investigating. The distinction is between checking the model and rewriting it. And for genuinely operational questions – bids, creative rotation, in-flight pacing – there are better tools: platform data, and properly designed tests. MMM sets the frame those decisions work within; it isn’t the tool for making them.
The right cadence for Marketing Mix Modelling follows the planning cycle it’s meant to serve: the annual reset, the quarterly review – or sooner, if something fundamental changes, such as a new channel, a pricing reset or a shock to the market. That’s not a limitation of the technique. It’s the technique doing its actual job, on the timescale strategic decisions genuinely operate on.
A real example. I worked with an entertainment company where a new chief marketing officer was being sold the daily MMM dashboard vision by a third-party vendor. Technically, the solution was excellent. We spoke to the people who would deliver it: they could build the technology, their modelling approach was appropriate, and the team was first class. But the organisation had no defined business need for it, and little idea of the data it would require or the complexity involved.
It sounds easy to say, but we asked some simple questions. What decisions will you make off the back of this? How often do you need to make those decisions? How are you making them now? And what improvement can you expect? Unfortunately, there weren’t any good answers. The company was under a lot of competitive pressure, and the cost wasn’t just a seven-figure investment – it was the management time and attention the project would take. Could the organisation make use of it? Did it have the people to act on it? Most importantly, would it tell them anything week to week that their existing reporting couldn’t? It wouldn’t. So we recommended they didn’t pursue it – for practical reasons, not technical ones – and they stopped.
This is a different question from whether MMM is the right tool at all – see When Marketing Mix Modelling isn’t for you.
If a proposal in front of you promises weekly MMM updates, it’s worth asking what business problem that update cadence is really solving, because it is very rarely yours. Then, whenever the numbers move, ask what changed in the market to move them. If nobody can point to something real, you’re looking at noise, not news. If you’d like a second opinion, drop a line to [email protected].