In advertising there are three words that often get used interchangeably; measurement, attribution, and incrementality. However, they each represent something a bit different.
“Which channel should get credit for this conversion?” This is a question of attribution.
Incrementality asks a much more important question:
“Would this conversion have happened without the advertising?”
Measurement is the broad umbrella. It includes essentially everything you use to evaluate advertising performance: impressions, reach, frequency, clicks, conversions, CPA, ROAS, brand lift, as well as attribution and incrementality.
Traditional attribution methods, whether first-click, last-click, multi-touch, promo codes, post-purchase surveys, or platform-reported attribution, are ultimately systems for assigning credit.
Incrementality does not try to win the attribution argument. It measures whether the advertising actually changed the outcome.
Attribution Is Not the Same as Causation
Imagine a customer hears an Audiohook ad on a podcast Tuesday morning.
On Wednesday, she searches for the brand on Google and visits the website.
On Thursday, she sees a Meta retargeting ad.
On Friday, she searches again, clicks a paid search result, and makes a purchase.
Who gets credit?
Paid search may claim the conversion because it generated the final click. Meta may claim it because the customer was exposed to an ad within its attribution window. A post-purchase survey may give credit to any one of the touchpoints.
Each system is attempting to answer who gets credit.
None of them necessarily answers whether the purchase would have occurred if the Audiohook impression had never happened.
That is the problem incrementality is designed to solve.
Rather than reconstructing a complicated customer journey and assigning percentages of credit, we model a world in which Audiohook was absent and compare it with the world in which Audiohook was present.
The difference between those two outcomes is the incremental impact of the campaign.
That makes incrementality fundamentally different from attribution.
Why Incrementality Is Particularly Important for Audio
Audio is largely a clickless medium.
A consumer generally does not hear an advertisement in a podcast, stop what they are doing, go to the audio player and click the ad (in the rare case the ad player supports display ads), and immediately complete a purchase.
Someone hears an ad while driving, walking their dog, exercising, cooking, or listening to a podcast. Hours or days later, they may search for the brand, navigate directly to the website, convert from another device, purchase through Amazon, visit a retail store, or eventually click an advertisement from another channel.
Traditional attribution systems often easily give the final interaction credit for demand created much earlier.
This is one reason clickless media is particularly vulnerable to under-attribution.
Incrementality changes the conversation.
We do not have to prove that every conversion followed a perfectly observable path from an Audiohook impression.
Instead, we can ask:
Did the population exposed to Audiohook produce more conversions than we reasonably would have expected without Audiohook?
That is a far more meaningful measurement of business impact.
The Incrementality Tests We Use
At Audiohook, we primarily use three approaches to incrementality measurement:
Geo Holdout Testing
Inverse Holdout Testing
Causal Impact Analysis
The methodology we choose depends on the client’s conversion volume, geographic distribution, historical data, campaign maturity, expected lift, and ability to create a meaningful control.
The goal is always the same:
Build the most credible possible counterfactual for what would have happened without Audiohook.
1. Geo Holdout Testing
Geo Holdout testing is one of the cleanest ways to measure incremental advertising impact.
We identify comparable geographic markets and separate them into two populations.
Treatment markets receive Audiohook advertising.
Control markets do not.
We then evaluate whether conversion behavior changes differently between the two groups.
Instead of asking which individual conversions Audiohook should receive credit for, we are looking at the aggregate business outcome.
If comparable markets historically behaved similarly, but the markets receiving Audiohook begin outperforming the markets without Audiohook after the campaign launches, we can measure the incremental lift associated with that exposure.
Why We Use It
Geo Holdouts create a real-world control.
This dramatically reduces our dependence on cookies, clicks, promo codes, survey responses, device matching, or arbitrary attribution windows.
We do not need to perfectly observe every consumer journey.
We need to determine whether introducing Audiohook changes the aggregate outcome.
What We Need
We ideally request 150 days of historical conversion data before beginning the test.
The dataset generally needs to identify:
Location | Date | Conversion
When revenue is part of the analysis, we may also request revenue associated with each conversion.
Historical data allows us to identify geographic markets that have behaved similarly before the campaign and understand seasonality, volatility, baseline performance, and expected variation.
We also need sufficient geographic flexibility to create meaningful treatment and control populations.
Timeline
A typical Geo Holdout measurement period is approximately 30 to 60 days.
Depending on the client’s consideration and conversion cycle, we may also incorporate approximately 7 to 30 days of cooldown so consumers exposed near the end of the test have sufficient time to convert.
Ideal Sample Size
There is no responsible universal conversion threshold for incrementality testing.
As a practical starting point, we generally prefer brands with at least 250 weekly conversions, although significantly greater volume creates much stronger experimental flexibility.
The necessary sample is ultimately driven by statistical power.
A campaign expected to generate 15% incremental lift requires less data to detect that effect than a campaign expected to generate 3% lift.
Similarly, a highly stable business requires less data than one where conversion volume naturally swings dramatically from week to week.
The goal is not simply to have a large sample. It is to have enough signal to reliably separate incremental lift from normal business noise.
2. Inverse Holdout Testing
An Inverse Holdout applies the same experimental philosophy in the opposite direction.
Instead of introducing Audiohook into selected markets, we begin with a campaign that is already broadly active and remove Audiohook from selected geographies.
Most markets continue receiving Audiohook.
Selected markets stop receiving it.
We then evaluate what happens.
Why We Use It
This methodology is particularly useful for mature campaigns.
Once Audiohook is successfully operating at scale, turning off the majority of a campaign simply to conduct another test may be commercially irresponsible.
An Inverse Holdout allows us to preserve most of the campaign while creating a smaller population where Audiohook’s influence is intentionally removed.
The question becomes:
What happens to conversions when Audiohook disappears?
If conversion performance deteriorates disproportionately in the holdout markets compared with markets where Audiohook remains active, that gives us evidence of incremental contribution.
Again, we are not trying to decide which channel gets credit for individual orders.
We are measuring whether the business performs differently when Audiohook is present.
What We Need
We ideally want 150 days of historical geographically segmented conversion data, along with continued reporting during the experiment.
We also need enough geographic scale to create a statistically meaningful holdout without unnecessarily damaging overall campaign performance.
Timeline
The measurement period is generally approximately 30 to 60 days, with an additional 7 to 30 day cooldown period when appropriate for the client’s conversion cycle.
Ideal Sample Size
Inverse Holdouts generally work best on larger, more mature campaigns.
Because only a portion of the campaign is being withheld, the selected holdout needs enough conversion volume for a meaningful difference to become detectable.
The ideal holdout is therefore:
Large enough to measure, but small enough to avoid unnecessarily sacrificing business performance.
3. Causal Impact Analysis
Not every business can support a clean experimental holdout.
Some brands need national coverage. Some have highly concentrated geographic demand. Some markets simply do not behave similarly enough to create reliable treatment and control groups.
In those situations, we can use Causal Impact analysis.
Rather than creating a literal untreated population, we use historical performance and comparison data to estimate what likely would have happened if Audiohook had not been running.
This creates a statistical counterfactual.
Conceptually:
Expected performance without Audiohook = counterfactual
Actual performance with Audiohook = observed outcome
Difference = estimated incremental impact
Why We Use It
Causal Impact allows us to ask the same fundamental incrementality question when a clean geographic experiment is impractical.
It is particularly useful as an ongoing measurement methodology because the analysis can evolve as additional campaign data becomes available.
During the first several months of a campaign, we can repeatedly examine estimated incremental lift and observe how that lift develops as impressions accumulate and customers move through the funnel.
This helps us understand something we refer to as bake time.
Bake time is the amount of time required for campaign exposure to work through the client’s normal consideration and conversion journey before its full impact becomes visible.
What We Need
Historical data is particularly important for Causal Impact analysis.
We generally request 150 days of geographically segmented historical conversion data so that we can understand the normal relationship between markets before evaluating post-launch changes.
We then need that same conversion dataset supplied consistently as the campaign progresses.
For brands with meaningful leakage, this is one reason Audiohook may request weekly conversion reporting during the first several months of the campaign.
Timeline
The first 20 to 30 days can begin providing useful signals, but Causal Impact analysis becomes increasingly informative as more post-launch data accumulates.
Rather than treating the analysis as a single pass/fail test, we can repeat it as the campaign matures.
This gives us a view not only into whether incremental lift exists, but how quickly it develops.
Ideal Sample Size
Causal Impact analysis benefits from both conversion volume and historical consistency.
Thousands of relatively stable weekly conversions generally create a stronger model than a few dozen conversions that fluctuate dramatically.
Our typical qualification benchmark of approximately 250 weekly conversions provides a useful starting point, but it is not a universal statistical requirement.
The quality of the counterfactual is just as important as raw volume.
How the Tests Compare
Geo Holdout
Inverse Holdout
Causal Impact
Fundamental question
What happens when we add Audiohook?
What happens when we remove Audiohook?
What likely would have happened without Audiohook?
Best use
New or controlled launches
Mature, scaled campaigns
When clean holdouts are impractical
Control
Actual untreated markets
Markets where Audiohook is removed
Statistical counterfactual
Historical data
Ideally 150 days
Ideally 150 days
Ideally 150 days
Initial measurement
~30-60 days
~30-60 days
~30-60 days
Potential cooldown
~7–30 days
~7–30 days
Depends on conversion cycle
Geographic flexibility
High
Moderate
Lower
Particularly useful for
Experimental validation
Revalidation at scale
Ongoing measurement
The methodologies differ, but the philosophy does not.
We are trying to measure what changed because the advertising existed.
Why Incrementality Is Superior to Last-Click Attribution
Last-click attribution asks which interaction happened immediately before the conversion.
That can be useful operationally, but it says nothing about causation.
If Audiohook creates interest on Monday and the consumer searches for the brand and clicks a paid search ad Friday, last-click attribution can assign the conversion to search.
But did paid search create the customer?
Or did it simply capture demand that already existed because another channel created it?
Incrementality is designed to answer that distinction.
Why Incrementality Is Superior to Promo Codes and Vanity URLs
Promo codes and vanity URLs do not measure advertising impact.
They depend on consumers remembering a code, remembering a URL, using it correctly, and not discovering another offer somewhere else.
They also tend to capture the consumers most willing to take an immediate, trackable action while missing the larger population influenced by advertising but converting through another path.
Incrementality does not require the customer to behave like a tracking mechanism.
The consumer can simply behave like a consumer.
Why Incrementality Is Superior to Post-Purchase Surveys
“How did you hear about us?” sounds like a straightforward attribution question.
Unfortunately, human memory is not an attribution system.
Consumers may have encountered a brand through five different channels. They may remember only one. They may choose the easiest answer. They may credit the channel they encountered most recently rather than the one that actually created their interest.
Surveys can provide useful directional information.
They should not be confused with causal measurement.
Why Incrementality Is Superior to Platform-Reported Attribution
Every advertising platform has an understandable incentive to measure the conversions associated with its own advertising.
But multiple platforms can observe the same customer journey and independently claim influence over the same conversion.
The result can be mathematically impossible.
A brand might generate 10,000 conversions while its collective marketing platforms report 15,000 attributed conversions.
Incrementality approaches the problem from outside the fight for credit.
Instead of asking every platform how many conversions it believes it influenced, we look at what happened to the actual business when advertising was introduced, removed, or changed.
That is a fundamentally different standard of evidence.
Attribution Still Matters
None of this means attribution is useless.
Quite the opposite.
At Audiohook, attribution is essential for understanding campaign behavior and making optimization decisions.
We want to know which audiences are converting, which inventory is performing, which creative is producing results, how long conversion journeys take, and where we should allocate additional investment.
Attribution helps us operate the campaign.
Incrementality helps us prove the campaign is actually growing the business.
Those are different jobs.
The mistake is asking attribution to answer a causal question it was never designed to answer.
The Standard Should Be Causation, Not Credit
Digital advertising spent years becoming increasingly sophisticated at tracking people.
Cookies became more advanced. Attribution windows became more complicated. Device graphs emerged. Multi-touch models assigned fractional credit. Platforms built enormous reporting systems capable of connecting thousands of signals.
But sophistication does not automatically equal truth.
You can build an extraordinarily sophisticated system for assigning credit and still never answer the most important question in advertising:
Did the advertising cause additional business?
That is the standard incrementality attempts to meet.
For Audiohook, this is especially important because we operate in a medium where the influence of advertising frequently occurs long before the trackable action.
We therefore use attribution for what it does well: optimization, diagnostics, and campaign management.
But when we want to validate whether advertising is creating real growth, we look to incrementality.
Because ultimately, a marketer should not care which platform wins the argument over who gets credit for a conversion.
They should care whether the conversion would have happened without the investment.
Attribution measures credit. Incrementality measures impact.
And impact is what we are ultimately here to create.