“Guess what? Advertising doesn’t work!” makes a great narrative hook for a podcast, but it’s not helpful if what you’re trying to do is measure advertising effectiveness. My concern here is that a lot of decision-makers believe Freakonomics’ general premise that advertising is “guilty until proven innocent.” However, anecdotes aren’t a substitute for a rigorous measurement strategy bespoke to a brand’s characteristics and conditions. Instead of trying to find “smoking guns” that highlight the inadequacy of advertising, the more productive path for a CEO or CMO is to understand the inadequacy of measuring advertising correctly. How do I know? Over the last 20 years, I’ve measured advertising for hundreds of brands and witnessed the refinement of measurement to a point where advertisers can reliably predict business outcomes. So let’s do a point-by-point walkthrough of how to think about advertising measurement in a more productive, less knee-jerk way:
Freakonomics conclusion #1 — Teasing out the causal part of TV uplifts is a near-impossible task
Freakonomics presumes here that academic studies have failed a number of times to understand the causal impact of ads on sales because there was no randomized experiment to examine. While randomized experiments are preferable when conducting research, they aren’t always achievable, especially in the case of advertising where countless variables must be accounted for. Evaluation techniques, such as Market Mix Modelling (or MMM) can pull apart all sales drivers without the need for prior tests to be set up.
Freakonomics conclusion #2 — You can’t disentangle seasonality and media uplifts
Brands will indeed — and quite rightly — advertise during seasonal peaks. The podcast puts forward an endogeneity problem: brands that advertise over seasonal peaks are getting advertising uplift confused with seasonality (seasonal demand spikes that would have happened anyway) and “it’s impossible to disentangle” the two. Approaching this problem the wrong way can cost brands a great deal in mis-allocated spending. Seasonality and media uplift can indeed be disentangled by analyzing a longer time series of data. For example, analyzing three years of data builds up as much real-world behavior as possible to inform the measurement model, giving you three chances of isolating seasonal effects (e.g., 3x Christmas, 3x Easter, etc). Also, advertising when customers organically look for your product doesn’t have to be a bad thing. For some brands, seasonal peaks are the highest-ROI opportunity for advertising, so it makes the most financial sense to advertise over the highest possible baseline of demand.
Freakonomics conclusion #3 — Using randomized experiments to measure advertising uplifts in its entirety
Freakonomics presents randomized experiments — or A/B testing — as the best method to measure media uplifts. While I think A/B testing is great — it is after all one of the most common means of measuring uplifts — it has its shortfalls. It can miss the mark on advertising uplift as it: Market Mix Modelling considers the initial sales lift of the advertising, as well as measuring the short to medium-term impacts, sometimes lasting up to three years. These kinds of longer-term branding impacts can add around 250% on the initial uplift.
Freakonomics conclusion #4 — TV ads are unprofitable
It’s put forward that TV is unprofitable, and the uplifts are underwhelming. Now to break it down, I’d say there are two things going here: one is the uplift, and the second is the way ROI is measured. Firstly, the initial uplift suggested by Freakonomics of between 1% and 10% rings about true, but this is true of a consumer packaged goods (CPG) brand, which is the use case the paper is derived from. Media uplifts for CPG are notoriously small, not because the media impacts are nominal, but because CPG brands tend to spend 10x their total media budgets on trade promotions (temporary price reductions, multibuys, etc.) and this dwarfs media uplifts. Outside of CPG, you can expect to see media uplifts anywhere between 5% and 40% of sales. Secondly, the ROI calculation is a bit squiffy — they are calculating the marginal ROI rather than the absolute ROI. Freakonomics puts it thusly: This is a very different point to whether the advertising is paying back or not. It is simply saying the ROI will increase if the budget is reduced. It is not an absolute ROI. Increases in spending will tend to lead to lower ROI/efficiency, but not the absolute profit uplift. In my experience, 8 times out of 10 clients can increase their spend on media and enjoy higher absolute profits.
Advertising doesn’t always work — the key is being able to tell when
While entertaining, sensational anecdotes from Freakonomics aren’t helpful for brands and decision-makers when it comes to understanding ad effectiveness. The key is to measure and find out what works specifically for your brand. In your personal life, I hope I haven’t diminished your enjoyment of Freakonomics. In your professional life though, I hope this has been useful input for the next time you are drawn into an “advertising doesn’t work” debate.
