Creative Specs
CTV creative experimentation
Plan CTV creative tests around one decision, comparable delivery and an outcome that can answer the question.
A useful connected TV (CTV) creative test starts with one decision. Decide which part of the ad to change, how the versions will get comparable delivery, and what outcome would justify picking one. Two ads running at once are not, on their own, a controlled test.
Planning a Controlled CTV Creative Experiment
- Define the hypothesisStart with one clear decision about creative change
- Choose a measurable outcomeAlign metric with the creative change (e.g., recall, understanding)
- Ensure comparable deliveryConfirm assignment method and inventory control
- Verify delivery before analysisCheck impressions, devices, locations, and frequency
- Make a bounded choiceReport results with uncertainty and context
Start with the decision
Write the hypothesis and the primary measure before building variants. An opening that demonstrates a product, for example, might help viewers recall its benefit better than one that introduces a problem — and that question needs a suitable recall measure. Impressions and completed plays describe delivery and playback; they cannot show what viewers understood.
Choose a measure that fits the proposed change and decide in advance what difference would matter for the next creative choice.
| Decision | Main change | Evidence to plan |
|---|---|---|
| Which opening communicates the benefit? | Opening scene or first line | The same neutral message-recall measure for comparable viewers |
| Can a shorter edit carry the explanation? | Duration and information included | Product understanding, with playback reported separately |
| Does a QR treatment improve response? | Response treatment | A common outcome measured comparably across both versions; scans alone are insufficient |
Keep other material features, including the offer and destination, as consistent as the question allows. Record the differences that cannot be held steady.
CTV Creative Test: Decision, Delivery, and Outcome Alignment
- Decision
- Which opening communicates the benefit?
- Main Change
- Opening scene or first line
- Evidence to Plan
- Same neutral message-recall measure for comparable viewers
Check how versions will be assigned
Ask the buyer how versions will be assigned and rotated in the actual setup. Check whether inventory supports separate test arms and a controlled comparison, whether one person or household could encounter both versions, and how delivery will be checked.
Keep dates, Australian locations, intended audience and measurement method aligned. If assignment is uncontrolled, describe the result as a comparison of delivered versions and investigate who received each one.
Some creative changes also require a different placement. If a QR version uses an interactive format and the plain version uses ordinary in-stream video, the result compares those format packages as well as the response prompt. Record eligible screens, inventory and reporting coverage for each arm.
Review delivery before the outcome
Check finished versions on a television and in the booked placement preview where available. Confirm the intended difference is visible and that both ads make the same qualified offer. Supplier specifications determine which durations and treatments can serve together.
After launch, compare eligible impressions, dates, publisher and device mix, and exposure frequency. Investigate material departures from the plan before interpreting the primary outcome. Completion metrics do not establish attention or understanding, and different ad lengths change what a completion comparison means.
Pre-Launch CTV Creative Test Checklist
- Test on a TV screenConfirm intended difference is visible
- Verify offer consistencyBoth ads make the same qualified offer
- Check inventory compatibilityDuration and treatment support co-serving
- Review placement previewEnsure expected delivery in booked environment
Make a bounded choice
Report the chosen outcome for each version with its denominator, covered inventory and uncertainty. Ask whether the difference is large enough for the original decision, and whether delivery or measurement gaps could explain it.
A small or unclear result can support keeping the current edit or designing a better test. It does not require a winner.
Use the unresolved question to choose the next step: more comparable delivery, a better outcome measure or a narrower creative change.
Key Metrics for CTV Creative Testing
- Eligible Impressions
- Aligned across test arms
- Completion Rate
- Reported separately by version
- Exposure Frequency
- Checked for balance across groups
- Device Mix
- Matched between test versions
In this guide
- Testing a shorter edit against a longer product explanationCompare short and long CTV edits on product understanding and comparable delivery, with completion rates and placement differences in context.
- Comparing a QR response with a non-interactive call to actionCompare QR and plain CTV response treatments with the same offer, eligible delivery and a genuinely common outcome measure.
- Measuring message recall for different creative openingsCompare CTV openings with a neutral message-recall question, pre-agreed scoring and a separate check of brand linkage.
- Deciding when a CTV creative test needs more evidenceDiagnose whether a CTV creative test needs more responses, more comparable delivery or a better measure before choosing a version.
