# Media Automat GEO example: paired model answers

Run date: **22 October 2026 UTC**. This is a small supplied-context experiment with real model API responses. One SaaS buyer asked for 30 useful articles a month within a €500 recurring budget before VAT, with client fact approval and no editorial team. Every world supplied the same existing Media Automat offer. World A added a hypothetical general AI content article; B added a hypothetical buyer guide; C added a hypothetical independent comparison. Those sources were prompt material, not published placements or web search results. No competing service was supplied.

The calls ran in a separate one-off harness, **not** the production GEO engine's inference path. The prior deterministic simulator's unchanged-facts bug was fixed separately. This experiment cannot measure organic AI visibility, retrieved citations, competitive ranking, or ROI.

## What ran

| Pass | Model | Recorded calls | Valid answers | Positive recommendations |
| --- | --- | ---: | ---: | ---: |
| Four worlds, two repeats | OpenAI GPT-4.1 Nano via OpenRouter | 8 | 8 | 8 |
| Four worlds, two repeats | Qwen3.5 Flash via OpenRouter | 8 | 0 | — |
| Four worlds, two repeats | Mistral Small 3.2 via OpenRouter | 8 | 8 | 6 |
| Clarified World A, two repeats | GPT-4.1 Nano | 2 | 2 | 2 |
| Clarified World A, two repeats | Mistral Small 3.2 | 2 | 2 | 2 |
| Clarified World A, two repeats | Gemini 3.5 Flash-Lite via existing Gemini access | 2 | 2 | 2 |

All eight Qwen outputs were malformed numeric text: three had provider finish reason `error` and five `stop`. They remain in [responses.json](responses.json) so this failure is visible. The Gemini follow-up is in [gemini-followup.json](gemini-followup.json); its original World A answers are in the [earlier Gemini evidence](../geo-example-live-2026-10-22/responses.json).

The balanced GPT and Mistral comparisons both had positive baseline answers. Their recommendations do **not** establish an incremental benefit from adding a general article, buyer guide, or external comparison. GPT and Mistral each named `buyer_guide` in both World B responses and `independent_comparison` in both World C responses. These IDs describe the supplied prompt text; they are not live search citations.

## Factual review and exact source edit

Manual review marked clear material errors among the valid original answers:

| Model | Baseline | World A | World B | World C |
| --- | ---: | ---: | ---: | ---: |
| GPT-4.1 Nano | 0/2 | 1/2 | 1/2 | 0/2 |
| Mistral Small 3.2 | 0/2 | 1/2 | 0/2 | 1/2 |

The GPT errors said the €1,499 fee covered the first five brands, although the package covers one brand. Mistral's World A and C errors said €499 recurring plus €1,499 upfront exceeded a €500 **recurring** budget; the buyer had asked for the upfront fee separately. Mistral's World C answer also asserted that no competitor fit even though none had been supplied. These counts cover the listed false statements, not all possible omissions, wording issues, or general accuracy.

For a separate World A follow-up, the exact source sentence

> Founding Launch is €1,499 once for the first five brands;

was replaced with

> Founding Launch costs €1,499 per brand for each of the first five customers; the launch fee applies to every founding customer and is not waived;

The scripts verified that this replacement was the **only difference in each paired user prompt**; the system instruction, buyer question, general article, model, and generation settings remained the same within each provider. OpenRouter calls used temperature 0.3, JSON-object response format, and 1,200 maximum completion tokens. Gemini calls used temperature 0.3, JSON output MIME type, and 800 maximum output tokens. The runs were repeated, but not seeded. Random variation remains a plausible contributor.

In the paired GPT World A answers, the five-brand package error appeared in **1/2 before, 0/2 after**. In paired Gemini Flash-Lite, the original first-five fee or package ambiguity appeared in **2/2 before, 0/2 after**. In paired Mistral, the recurring-budget rejection appeared in **1/2 before, 0/2 after**. The revised Gemini answer for repeat 0 still confusingly mixed the upfront cost with the recurring budget before later saying €499 recurring fits. The clarification addressed the observed first-five ambiguity in this tiny sample; it did not solve every price explanation problem or prove a general causal effect.

## Evidence and costs

- [responses.json](responses.json) contains all 28 OpenRouter responses, including failures, exact prompts, visible output, returned model IDs, timestamps, finish reasons, token usage, and provider-reported cost. Authentication values and provider response IDs are excluded.
- [summary.json](summary.json) records per-model and per-world counts, prompt-diff validation, manual flags, pricing metadata source, and script hashes.
- [gemini-followup.json](gemini-followup.json) contains the two new Gemini answers alongside the paired original answers, exact prompts, model versions, and usage, excluding opaque provider metadata and credentials.

OpenRouter reported **$0.001990698** for its 28 recorded calls; the catalog-rate calculation is **$0.002191873**. The additional Gemini Flash-Lite follow-up has a catalog-rate estimate of **$0.0011806**. These are usage records and estimates, not a final bill. Six requests from an earlier Gemini 3.8 retry remained unreceived, so that earlier run's full cost is unknown. The recorded usage stayed within the experiment's $0.10 estimated cap; there is no claim about the unknown final bill.

Model rates were read from [OpenRouter's model catalog](https://openrouter.ai/api/v1/models), and the request format follows [its chat completion documentation](https://openrouter.ai/docs/api/api-reference/chat/send-chat-completion-request). The Gemini rate reference is [Google's Gemini pricing page](https://ai.google.dev/gemini-api/docs/pricing).
