Measuring CTV message recall: Compare message takeaway, not just ad memory; Use same question and scoring rules for both openings; Check brand linkage separately from message recall
Image: Streaming Advertising Guide

Incrementality

Part of CTV creative experimentation

Measuring message recall for different creative openings

Compare CTV openings with a neutral message-recall question, pre-agreed scoring and a separate check of brand linkage.

Measurement Desk · · 2 min read

Compare CTV openings against the message viewers should take away, not whether they remember seeing an ad. Give comparable viewers one opening each. Ask the same neutral question after exposure and score answers using a rule set before responses arrive. Check brand linkage separately: a viewer may remember a benefit but attribute it to the wrong advertiser.

Isolate the opening

Write the intended takeaway in one sentence. Make two openings that lead into an otherwise consistent ad. One might show the problem first; another might show the product solving it.

Keep the offer, later voice-over, duration and end card as consistent as the idea permits. Record unavoidable differences, such as showing the brand earlier in one version.

Message recall concerns retaining the main takeaway; brand recall and message association concern remembering or linking the brand. Choose the primary question before launch, because a video completion cannot answer any of these recall questions.

Process for Measuring Message Recall in CTV Ads

  1. Define the core message in one sentencee.g., 'A modular sofa can be rearranged for different room layouts.'
  2. Create two openings with consistent later elementsMaintain offer, voice-over, duration, end card; record differences.
  3. Use neutral unaided question post-exposureAsk: 'What was the main message of the ad you saw?'
  4. Apply pre-agreed scoring rulesCorrect, partly correct, unrelated – coders blind to opening variant.
  5. Report brand linkage separatelyCheck if message is correctly attributed to brand.

Ask and score the same way

An unaided question could ask, “What was the main message of the ad you saw?” Ask it after a comparable delay in both groups. Agree in advance which answer elements count as correct, partly correct or unrelated.

For an illustrative homewares ad, “a modular sofa can be rearranged for different room layouts” may meet the rule while “furniture” does not. Have coders assess responses without knowing the assigned opening where feasible.

If recognition is useful too, present the same answer options to both groups after the unaided question. Do not show the desired wording first and call its later selection unaided recall.

Record question order, survey timing, recruitment and usable responses. An immediate survey after forced viewing answers a different question from one following ordinary CTV exposure or a later follow-up; label the setting.

Check the comparison and its limits

Ask the research provider how viewers are assigned, how exposure is established, whether someone can encounter both versions, and which audience and devices the sample covers. Before attributing a recall gap to the opening, investigate differences in inventory, geography or survey timing. Consider non-response and sampling bias.

A standard platform Brand Lift study should not be assumed to supply this specific open-ended creative comparison. Platform-based studies may use a different questionnaire or design. Confirm a suitable questionnaire and variant-level design with the research provider.

Report the share giving the intended message for each opening, usable response counts, scoring rule and uncertainty. Show brand linkage separately.

If a difference is small relative to sampling or delivery uncertainty, call it inconclusive. Any finding applies to the tested audience, versions, exposure setting and survey timing.

More from Incrementality