Which AI model your agent really uses can only be proven with your own logs and invoices, never by asking the agent.
Start with the model ID recorded on each request, the same evidence you need when an agent changes behavior after a model update.
In a 2026 audit of 17 third-party “shadow” APIs, 45.83% of identity tests failed. A model name on a label is a claim.
Quick answer: To see which AI model runs your agent, climb a 5-rung proof ladder: written promise, model ID in your logs, token and cost receipts, a 20-task test, and a fingerprint. Never trust the agent’s own answer.
Key takeaways
- An agent’s answer to which AI model it runs is not evidence, because models guess their names and vendor prompts rewrite them.
- The risk sits in the middle layer: wrappers, gateways, and resellers, not the major providers directly.
- Most cheaper-model answers come from effort settings, routers, harness defaults, or provider incidents, not fraud.
- The same model can cost 4 to 6.6 times more at its top effort setting.
- Five checks add evidence step by step. No single check proves everything.
- A cheaper model is fine when it is disclosed and tested. Hidden is the problem.
What is the model behind your AI agent?
An AI agent is 3 layers: a model, an app, and a harness. Only the model carries the GPT or Claude name, and the exact model ID is the only name that counts.
The 3 layers: model, app, and harness
Ethan Mollick’s model, app, and harness split shows why 1 model can behave differently in 2 products.
- Model: the engine, such as GPT or Claude.
- App: the interface, with its own defaults.
- Harness: the tools and instructions that let the model act, as in an agent versus a chatbot.
A weaker result from the same model is therefore not proof of a swap.
The model ID: the only name that counts
A model ID is the exact version string. A dated ID stays fixed, while a plain alias such as “latest” can move to a new version without notice.
“Powered by GPT” names no model ID, so it promises nothing.
Why asking the agent which AI model it uses fails
A model cannot reliably tell you which AI model it is. Names are assigned after training, and a vendor’s instructions can replace the identity, so a confident “I am GPT-4” proves nothing.
Models guess their own name
In UC Berkeley’s tests, identity questions revealed only a broad family or developer name in under 5% of cases. When a system prompt overrode the identity, accuracy fell near zero.
Models trained on text that mentions rivals also repeat those names. A confident wrong self-description is 1 more kind of AI hallucination.
Vendor instructions rewrite the answer
Most business agents run behind a hidden brief such as “You are Atlas, our assistant.” The agent then has no reason to name the model underneath.
A bot that says “I was built by [vendor]” describes the wrapper, not the model.
The trust chain: which AI model answers, and who can change it
Every hop between you and the model is a place where the name can change. Risk rises with each hop and concentrates in the middle layer.
| Route | Who can change the model | Risk |
|---|---|---|
| Your own provider account | You alone | Lowest |
| Platform with a router | The platform, by stated rules | Low to medium |
| Vendor wrapper or agency | The vendor, out of sight | Medium to high |
| Reseller or relay | An unknown party | Highest |
If your chatbot vendor resells API access, you sit on the third row. Whoever signs up for the provider account decides who can see the real usage.
The same audit found:
- Performance gaps against official endpoints reached 47.21%.
- 4 of the 17 providers had already shut down.
- Discounts of 70% to 90% below list price are a common rule-of-thumb warning sign of a cheaper model behind the label.
5 harmless reasons which AI model answers can change
Most cheaper-than-expected answers come from settings, defaults, and incidents. Rule these out before accusing a vendor of running a cheaper model.
Reasoning effort is a dial, not a different model
Artificial Analysis publishes cost per task for each effort setting, and the spread inside 1 model is wide.
| Model | Lowest setting | Highest setting | Gap |
|---|---|---|---|
| Claude Opus 5 | $0.64 | $4.21 | 6.6x |
| GPT-5.6 Sol | $0.23 | $1.25 | 5.4x |
| GPT-6 Astra | $0.63 | $2.57 | 4.1x |
Cost per task in US dollars, read from Artificial Analysis charts. Check the live page for current figures.
A vendor on a flat fee keeps that gap by running the lowest setting. Choosing between GPT, Claude, and Gemini matters less than which setting runs.
Routers, harness defaults, and usage limits choose for you
- Auto routers: OpenRouter’s Auto Router classifies each prompt and picks a model within your cost band. Each response names the model that answered.
- Auto modes: consumer apps often default to weaker versions than a manual pick.
- Harness defaults: 1 model can score differently in chat, in a search app, and in an agent tool.
Provider incidents and moving aliases
Anthropic’s postmortem describes 3 infrastructure bugs in 2025, including a routing error that hit up to 16% of Sonnet 4 requests at its peak.
Anthropic states it never reduces quality because of demand.
An alias that moves to a new version looks the same. Check the provider’s status page first, especially for a chatbot that suddenly gives wrong answers.
| Symptom | Most likely cause | Quickest check |
|---|---|---|
| Answers turn shorter and shallower | Lower effort setting | Compare thinking tokens per task |
| Model name changes between requests | Auto router | Read the model field in each response |
| Quality dips at peak hours | Rate limit or incident | Status page, then retest off-peak |
| Quality shifts, nothing changed on your side | Alias moved to a new version | Pin a dated model ID |
| Flagship billed far below list price | Possible substitution | Rungs 2 to 5 below |
The 5-rung proof ladder for which AI model runs your agent
The ladder ranks 5 checks from cheapest to strongest. Start at rung 1 and stop when the evidence shows which AI model runs.
| Rung | Evidence | Cost | Who runs it |
|---|---|---|---|
| 1. Written promise | Contract names a model ID | Free | Owner |
| 2. Model ID, independently checked | Logs match the provider’s dashboard | Free | Owner |
| 3. Token and cost receipts | Spend fits the price list | Low | Owner |
| 4. Side-by-side test | Pass rate matches a direct call | Low | Owner with help |
| 5. Fingerprint | Output pattern matches the model | Low | Engineer |
1. Get the model promise in writing
The contract should name the exact model ID, the effort setting, and the notice you get before either changes.
Without that sentence, no later check can show that a cheaper model was swapped in.
2. Read the model ID, then confirm which AI model ran
The response’s model field is a claim, because a reseller can return any string. Run the agent on your own provider account, and the provider’s usage dashboard becomes a second, independent receipt.
A GVM Technologies AI review checks whether your agent can run on your own provider account.
3. Match tokens and cost to the price list
Multiply a week of logged tokens by the model’s published rate and compare the result to the invoice. A flagship model billed far below list price may hide a cheaper model.
Cheaper models also tend to use more tokens per task, which shrinks the real saving.
4. Test behavior on your own tasks
Run the same 20 tasks through the agent and straight against the promised model, to confirm which AI model you get. Demo prompts hide what real traffic exposes.
Practitioners also warn that 1 model name can hide different quantization or chat templates across providers, per a provider-testing checklist thread.
The next section shows how to run it.
5. Fingerprint the endpoint to see which AI model answers
A fingerprint asks a one-word question, such as “name a random number from 1 to 100,” about 30 times. The answer pattern is then compared to a reference.
One 2026 study probed 165 models with about 326,000 requests for under $35.
Read the result as evidence, not a verdict. A tool maker’s guide lists legitimate causes of a mismatch:
- Quantization, meaning a compressed copy of the same model
- Model updates
- Vendor system prompts
- Load balancing across backends
A 20-task test that shows which AI model you get
A 20-task test runs real tasks through your agent and straight against the promised model, so a cheaper model shows up as a gap. Each run gets a 5-field receipt that records quality and cost together.
1. Build the set from real conversations
Pick 20 tasks from actual traffic, each with a known right answer:
- 10 routine tasks: order status, policy lookups, everyday replies.
- 4 hard tasks: multi-step requests that stress a weaker model.
- 3 long-context traps: a fact buried deep in a long document, tested near the advertised limit.
- 3 nonsense traps: unanswerable questions, to see whether the agent refuses or invents.
Save the exact inputs and run each task 3 times through both paths.
2. Fill a run receipt for every task
- Attempted: did the agent try the task?
- Passed: did the answer meet your check?
- Escalated: was a stronger model needed?
- Cleaned up: did a human have to fix it?
- Cost: tokens and dollars spent.
Total cost divided by passed tasks gives cost per finished task, which beats cost per call.
3. Read the gap and repeat at 3 times of day
- A repeated gap of 3 or more tasks is a rule-of-thumb trigger for questions.
- A dip only at peak hours points to rate-limit fallbacks or an incident, not a swap.
- Berkeley found that mixing at 20% or less evaded statistical tests on larger models, so retest monthly.
- Some teams report that the wrapper shifts cost and speed almost as much as the model, so log the harness too.
What to ask your vendor about which AI model runs
A vendor running a cheaper model without saying so will struggle with specific questions. Vague answers are information too.
8 questions that force a specific answer
- Which AI model, by exact ID, runs each step?
- Do you route between models, and by what rule?
- Which effort setting is the default?
- What happens when the main model is down or rate limited?
- How much notice do I get before a model changes?
- Can I see per-request logs with model ID and token counts?
- Can the agent run on my own provider account?
- Where do prompts go, and are they used for training?
See giving an agent access to customer data.
Contract clauses that make proof possible
- The exact model ID and effort setting, named in the agreement
- Written notice before any model change
- Per-request log access
- The right to run fingerprint tests
- A remedy if the delivered model differs
These clauses also reduce vendor lock-in, and refusal to sign them is a post-launch red flag. In a GVM Technologies AI review, the routing map is among the first documents requested.
When a cheaper model is the right call
A cheaper model is right for classification, extraction, and routine replies. Disclosed routing saves money, and the honest test is the pass rate on your tasks.
A common pattern pairs 1 strong model that plans and reviews with cheaper workers that execute, as in single-agent versus multi-agent systems.
GVM Technologies AI treats routing as a design choice the client approves.
FAQs
1. Can I ask ChatGPT or Claude which model it is?
Not reliably. Models guess their names, and vendor prompts can change the answer. Use the app’s model selector or the API’s model ID.
2. How do I check which AI model a ChatGPT or Claude app uses?
Open the model selector, which is the most dependable signal. Avoid “auto” modes when you need a fixed model.
3. Does worse quality prove my vendor changed which AI model runs?
No. Effort settings, routers, incidents, and moved aliases cause the same symptom. Compare against a saved baseline first.
4. How much does checking which AI model runs cost?
The 165-model fingerprint study cost under $35 in total. A 20-task test costs far less, though price depends on task length.
5. Is a fingerprint mismatch proof of fraud?
No. Quantization, updates, and system prompts shift the pattern. Treat it as a reason to ask for logs.
6. What if my vendor refuses to share logs?
Treat refusal as a finding. Fall back to rungs 3 to 5, then weigh whether to stay with a vendor you cannot audit.
Know which AI model runs your agent before it costs you
Every month without evidence, you may pay flagship rates for a cheaper model you cannot confirm. A baseline taken today turns a hunch into a dated, provable gap.
Follow this plan:
- Day 1: Name the model ID and effort setting in writing, and request logs.
- Week 1: Run the 20-task test and save the baseline.
- Week 2: Compare invoiced tokens against the price list.
- Every month: Rerun the test and read the status page before blaming the vendor.
As routers and fallbacks become the default, the question shifts from which model to which model on each step.
Get a straight read on which AI model your agent really runs
An unverified agent keeps costing you in quality problems you cannot trace. If a cheap fix is enough, the review says so.
The GVM Technologies AI core build and review service covers:
- Which AI model answers at each step of your agent
- Whether effort settings, routing, or fallbacks explain your quality gap
- A 20-task baseline built from your real conversations
- What your vendor’s contract and logs do and do not prove
