Attribution tells you which ads received credit. Incrementality testing asks the harder question: how many sales would disappear if those ads stopped running?
That distinction matters when an ecommerce account looks healthy in-platform but the business is not growing at the same pace. Branded search, returning customers, email, retail distribution, seasonality, and several ad platforms can all claim some of the same demand. A strong reported ROAS can still include orders that were likely to happen anyway.
Incrementality testing compares a group that can receive advertising with a credible control group that cannot. The difference between their outcomes estimates the conversions or revenue caused by the advertising. It is a better tool for budget allocation than asking a platform to grade its own homework, but it is not a magic report. The test needs enough signal, a clean holdout, and a decision worth making.
Start with the budget decision, not the testing method
“Is Meta incremental?” is too broad to test well. “Should we move part of our prospecting budget into branded search?” is closer, but it still mixes channels and objectives. A useful question names the spend, audience, outcome, and action that could change.
Try this format: If this campaign produces at least [decision threshold] of incremental [orders, revenue, or contribution], we will [scale, hold, or cut] the budget.
The threshold belongs to your business. It could be an incremental return that clears your contribution target, a cost per incremental customer that fits your payback window, or simply enough evidence to avoid a large budget cut. Do not borrow a universal benchmark from a vendor deck.
Use this ecommerce incrementality test-readiness worksheet
Before opening a platform experiment, answer these questions. A “not yet” is useful. It tells you what to fix before sacrificing revenue to a holdout that cannot answer the question.
| Readiness check | What a usable answer looks like | If the answer is no |
|---|---|---|
| What decision will the result change? | A specific budget, channel, campaign, or market decision with an owner. | Do not test yet. Write the decision first. |
| Which outcome matters? | Net orders, net revenue, new customers, or contribution measured consistently outside the ad-platform view. | Repair tracking and definitions before choosing a holdout. |
| Is there enough outcome volume? | The platform or test design tool shows adequate power for the outcome, holdout size, budget, and duration. | Use a more frequent outcome, combine eligible campaigns, lengthen the test, or wait. |
| Can treatment and control stay separate? | Low risk that control users or regions will see the tested media through another campaign or market. | Map overlapping campaigns, audiences, promotions, and geographies first. |
| Can the business stay stable? | No planned price change, site migration, stockout, retail launch, or promotion that lands differently across groups. | Move the test window or design around the known disruption. |
| How long will the test run? | A precommitted window that covers conversion lag and normal weekday or promotional variation. | Set the window before looking at results. Do not stop because an early chart looks good. |
| What result will trigger action? | A written scale, hold, repeat, or cut rule that accounts for uncertainty. | Agree on the rule before stakeholders see the answer. |
Choose the holdout your account can actually support
User-level conversion lift
A platform creates exposed and unexposed groups from eligible users, then compares conversion outcomes. This can be a practical option when the platform supports the campaigns and conversions you want to measure. It also means living with the platform’s eligibility rules and measurement boundaries.
Google’s current Conversion Lift setup includes a feasibility estimate based on factors such as conversion actions, daily budget, study duration, traffic split, historical data, and estimated lift. Google also notes that holding back an audience creates an opportunity cost. That is exactly why the business decision needs to be clear before launch.
Geo holdout
Geo tests split comparable regions into treatment and control markets. They can be useful when user-level suppression is unavailable or when the question spans channels. They also become fragile when regions are too different, shoppers cross boundaries, national promotions land unevenly, or one large market dominates the result.
Google’s published geo-experiment research describes random assignment of non-overlapping regions and estimation of the counterfactual response. In plain English: the control markets help estimate what would have happened in the advertised markets without the tested spend. Matching a favorite city to a random city and comparing two screenshots is not the same thing.
Campaign experiments are useful, but answer a different question
An A/B campaign experiment can tell you whether a bid strategy, creative, landing page, or setup performs better than the original. That is valuable optimization work. It does not automatically tell you whether advertising created demand compared with showing no advertising at all.
Google’s experiments guidance makes the same design point: set one clear hypothesis and avoid changing several variables at once. Use a campaign experiment to choose between two versions. Use a lift design when the question is whether the media added outcomes beyond the counterfactual.
A hypothetical ecommerce example
Say a brand is considering a meaningful cut to nonbrand paid social because its blended efficiency has weakened. The platform still reports attractive revenue, but a large share of buyers are returning customers.
The decision is not “prove paid social works.” It is: Should the brand keep the current prospecting budget for the next planning cycle? The team chooses net new-customer orders as the primary outcome, confirms that order status and customer type can be measured consistently, and runs a supported holdout after checking power. Before launch, it writes three outcomes:
- If incremental acquisition clears the brand’s preselected economics threshold with useful certainty, keep or scale the budget.
- If the estimate is positive but too uncertain to support the decision, repeat with a stronger design rather than calling the channel a winner.
- If the estimate is small or negative within a credible test, reduce or redirect spend and document what changed.
No made-up lift percentage is needed. The useful part is the decision rule, the clean customer definition, and the willingness to accept an inconvenient result.
Read the result without turning it into a permanent truth
A lift result belongs to the campaigns, audience, market, offer, creative, and time period tested. It does not prove the same channel will always produce the same incremental return. Confidence intervals matter. So do stock levels, promotions, competitors, and creative quality.
Separate “no detected lift” from “proof of no effect.” A noisy or underpowered test may be inconclusive. Google’s guidance explicitly says lower-certainty results should be treated directionally and that results reported as “no lift” may need more measurement. Do not turn an inconclusive result into a dramatic budget decision just because the dashboard produced a label.
Then reconcile the test with the rest of the business. Marketing efficiency ratio is still useful for monitoring the whole system, while branded versus nonbranded search reporting can reveal where existing demand is being captured. Neither one proves causality on its own. Your profitable ROAS threshold can define the economics the incremental result needs to clear.
When an incrementality test is not worth running yet
Skip the test for now if the result will not change a decision, the account cannot produce enough outcome volume, the control group will be heavily contaminated, or basic purchase tracking is still disputed. First fix the measurement layer and the budget question.
That is not avoiding rigor. It is avoiding false precision. A clean weekly channel report can be more useful than an elaborate experiment whose groups were never comparable.
Plan the test around the decision you need to make
BPM manages paid social and paid search on a retainer model, with measurement tied to the business decisions behind the media plan. If you need help deciding whether an incrementality test is feasible, start with the budget question, the available conversion volume, and the action the result would trigger.
Explore paid social management or explore paid search management.