
Incrementality
Part of CTV attribution and incrementality
Comparing exposed and unexposed markets carefully
Plan a CTV market comparison by selecting credible controls, checking the delivery contrast and reporting an effect estimate with uncertainty.
To estimate whether a CTV change affected sales or enquiries across markets, choose treatment and comparison areas before launch and measure the same business outcome in both. Under the test design, “unexposed” means the planned CTV treatment was withheld or reduced. It does not mean no resident saw any advertising.
Select markets the test can use
Choose boundaries that the media buyer can target and the business can use to assign outcomes. A postcode cluster is unsuitable if delivery cannot be controlled or sales cannot be assigned to it consistently. States may be easy to name but can leave too few independent areas for a useful comparison.
In Australia, the ABS Australian Statistical Geography Standard offers stable units between those extremes: Statistical Areas Level 4, 3, 2 and 1. Its Postal Areas approximate postcodes and enable comparison of ABS data with other data collected using postcodes, such as customer addresses in a CRM or delivery records.
Define one outcome before choosing areas: online orders by delivery postcode, store sales by trading area or qualified enquiries by customer address. Google's Meridian GeoX guidance says conversion data must be accessible at the geographic level used for the experiment, should be an absolute count or revenue, and should be raw rather than attributed conversions.
Inspect a pretest series of the same outcome for each area. Compare scale, trends and volatility, then check stock, service coverage, pricing and planned local promotions. Similar sales on one date do not establish comparable trends. GeoX asks for at least 3 times the test length in daily pretest data, and a year or more where the business is strongly seasonal.
Where feasible, group broadly comparable markets and randomise treatment assignment within those groups. If markets are selected without randomisation, record the selection reasons and the assumptions needed for the comparison to be credible.
Estimate whether the available markets and outcome volume can detect a change large enough to affect the decision. A design that cannot separate a useful effect from ordinary variation may still produce a number, but that number will not settle the spending question. GeoX recommends a single-cell design, one treatment group against one control group, because it needs fewer geos and less budget to reach a low minimum detectable effect.
Planning a CTV Market Test in Australia
- Define a single outcomeE.g., online orders by delivery postcode, store sales by trading area, or qualified enquiries by customer address
- Ensure data accessibilityConversion data must be available at the geographic level used and be raw counts or revenue
- Randomise within groups if possibleGroup similar markets and randomise treatment to reduce bias; document selection reasons without randomisation
Key Requirements for a Valid CTV Geo-Test
- Minimum pretest data
- 3× test duration in daily data
- Recommended pretest length
- 1 year or more for seasonal businesses
- Data type requirement
- Raw conversion counts or revenue, not attributed conversions
- Design recommendation
- Single-cell design (1 treatment vs 1 control) for lower budget and MDE
Protect the treatment contrast
Write down what differs between groups: the CTV inventory, dates, spend or delivery rule. For a new campaign, treatment markets might receive CTV while controls continue their existing media. GeoX calls this a holdback test, used to measure brand new ad strategies such as new channels.
For a spend increase, both groups might continue existing activity while treatment markets receive the additional CTV spend. These designs answer different questions.
Check delivered impressions by market during the flight. Material delivery in controls or little delivery in treatment areas weakens the planned contrast. Keep other controllable activity as steady as possible and record changes such as a local promotion or supply interruption. Travel, purchases outside a home area and imperfect location signals can also blur the comparison.
Use area-level totals rather than individual location records where you can. If the business does handle personal information to assign outcomes, the Australian Privacy Principles in the Privacy Act 1988 govern its collection, use and disclosure.
Do not replace a control market midway because its result looks inconvenient. If a serious operational change occurs, document it and assess whether the original analysis remains credible.
Exposed vs Unexposed Markets: Key Differences in CTV Testing
- Treatment MarketReceives planned CTV treatment (e.g., new campaign or increased spend)
- Control Market (Unexposed)No additional CTV treatment; continues existing media activity
- Delivery ContrastMust be maintained—no material delivery in controls, minimal in treatment areas can weaken results
- Key AssumptionBoth groups are otherwise comparable in trends, pricing, promotions and stock availability
Pros and Cons of Holdback vs Spend Increase Tests
- Holdback Test (New Campaign)Pros: Measures impact of entirely new ad strategy. Cons: Not suitable for evaluating incremental spend.
- Spend Increase TestPros: Measures effect of added investment. Cons: Requires baseline stability; harder to isolate effect from existing media.
Interpret the tested change
Analyse the outcome using the plan agreed before results were inspected. Compare treatment outcomes with the counterfactual estimated from the controls and pretest data; GeoX does this mainly with time-based regression. Report the effect estimate, uncertainty range, markets, dates and exclusions. A positive estimate with a wide range may be inconclusive.
A complete report gives the point estimate of incremental conversions, the percentage lift, confidence intervals and p-values, and incremental return on ad spend where revenue is the outcome.
A raw post-campaign sales gap is not CTV lift: the areas may have started at different levels or faced different conditions. Even a credible market test estimates the effect of the tested change across its included areas. It does not identify which household responded, and its result may not carry unchanged to another inventory mix, offer or period.



