Incrementality

The counterfactual is the whole idea

Every incrementality question reduces to one comparison: what happened, against what would have happened had you not run the activity. The second half of that sentence is not observable. Nobody can watch the same week twice, so the counterfactual always has to be constructed from a group of people, markets or time periods that stand in for the version of the business where the advertising did not run.

Attribution never attempts this. It observes conversions that occurred and allocates credit among the touchpoints it can see, according to a rule — last click, data-driven, position-based. Every one of those rules distributes 100% of the credit, because a rule that concluded "none of these touchpoints caused it" would look broken. So attribution answers "who touched this order" and is routinely read as an answer to "what caused this order", which is a different question with a frequently different answer.

The clearest illustration is branded search and retargeting. Both are shown almost exclusively to people who have already decided, both post excellent attributed returns, and both can be close to entirely non-incremental — the conversions would have arrived through an organic result or a direct visit. Switching them off is the only way to find out, and the reason so few brands know is that switching them off means watching an attributed ROAS number collapse while the business does not.

The tests that produce a causal answer

They differ in how the control group is built and in what running the test costs. The ranking below is roughly by evidential strength, and evidential strength is bought with foregone revenue in every case.

Why a PSA or ghost-ad control beats "people we did not target"

The naive control group is everyone the campaign did not reach. It is almost always wrong, because the people an ad platform chooses to show your ad to are selected precisely for their likelihood of converting. Comparing them against the people it declined to show is comparing a high-intent group with a low-intent group and reading the pre-existing difference as advertising lift.

A PSA test fixes this by running the same targeting and the same auction for both groups and substituting the creative — the control sees an unrelated public-service ad. Ghost bids go further and record the auctions the control group would have won without serving anything. In both cases the control is the population the platform actually selected, which is the only population your treatment group is comparable to.

Geo tests dodge the problem differently: markets are assigned before the platform selects anyone, so selection happens identically inside both arms. The cost is that markets are noisier than individuals and there are far fewer of them, which is why geo designs need matched pairs and a pre-period where the two groups are already tracking together.

What a lift number does and does not license you to conclude

Work an example. A geo test runs across 40 matched markets, 20 treatment and 20 control, and the pre-period shows both groups running at $900,000 of revenue each, which is what makes them usable as a matched pair. During the test the treatment markets do $1,200,000 and the control markets $1,000,000. Incremental revenue is $200,000. Treatment spend was $80,000, so incremental ROAS is 2.5, against the 5.0 the platform reported for the same campaigns.

That result licenses one conclusion: at that spend level, in those markets, in that period, with that creative, the campaign returned $2.50 of revenue per dollar. At a 45% contribution margin the break-even is 2.22, so it cleared it — narrowly. It does not license the conclusion that the channel returns 2.5 at twice the spend, because returns diminish; the marginal dollar at $160,000 of spend is a different dollar and has to be measured separately.

A null result is the most misread outcome of all. "No significant lift" usually means the test could not detect an effect of the size you were looking for, not that the effect is zero. Before running anything, decide what lift would change your decision, and be honest about whether the test as designed could see it — a channel that is 10% of spend running for two weeks generally cannot.

And a lift number expires. It is a measurement of a market, a creative set and a competitive auction at one moment. Treat it as evidence with a shelf life of a quarter or two rather than a constant you can put in a model and leave there.

The practical minimum for a DTC brand

Most brands do not need a measurement vendor to start, and the ones that buy one first usually cannot act on the output. The ladder below is in order of cost, and each rung is worth climbing only when the rung below has stopped answering the question.

• Track marginal MER continuously. Every deliberate spend change is a free, weak experiment: divide the change in revenue by the change in spend and compare that against break-even rather than against the blended ratio.
• Run the branded-search test first. It is the cheapest meaningful holdout most brands have available, the effect size is usually large, and the result frequently frees budget immediately.
• Graduate to a geo holdout for the largest channel. Withhold advertising in matched markets covering enough volume to detect the effect you care about, run it for at least one full purchase cycle plus the attribution window, and check that the two arms tracked together in the pre-period before trusting anything in the test period.
• Treat platform-run lift studies as a sanity check, never as the deciding evidence. The platform designs the test, selects the control and scores the result.
• Accept the constraint: if you cannot tolerate switching off a meaningful share of spend for several weeks, you cannot measure incrementality. There is no version of this that costs nothing, and every method that claims to is modelling rather than measuring.