Before/After Prompt Experiments: How We Test AEO

Everything published in Answer Engine Weekly is backed by a before/after prompt experiment. This article is the method itself — the protocol we use, and the one we recommend any agency adopt before it claims an AEO result.

Step 1: Fix the prompt set

Choose a fixed set of prompts — typically 8 to 15 — that represent how a real customer would ask an engine about the category. Write them down verbatim. They must not change for the life of the experiment, or the comparison is meaningless.

Step 2: Capture the baseline

Run every prompt on every target engine — ChatGPT with search, Perplexity, Google AI Overviews, Claude. For each, record: the cited sources, whether the client appears, and a screenshot. The screenshot is non-negotiable; it is the evidence.

Step 3: Make one change at a time

Change one variable — an answer-first rewrite, a schema addition, a new corroborating mention. If you change five things and citations move, you have learned nothing about which thing worked.

Step 4: Wait for re-crawl, then re-measure

Engines need time to re-crawl and re-index. Wait two to four weeks, then re-run the identical prompt set and capture the same evidence.

Step 5: Report honestly

Publish the before screenshot, the after screenshot, the single change, and the citation-share delta. If the change did nothing, publish that too. The anti-hype position only holds if you are willing to report your own failed experiments.