What a chatbot is actually doing
A multimodal language model converts an image into a semantic description and then reasons about that description in text. It is answering the question does this picture look like the AI images I read about during training.
That is a question about content and composition. Detection is a question about texture: how neighbouring pixel values relate to each other, at a scale far below anything a semantic description preserves.
The information a detector reads is discarded before a language model starts thinking. It is not that chatbots are bad at this. They are working from a representation that does not contain the evidence.
What that produces in practice
- Confident answers with no threshold. A chatbot will say likely AI-generated without any scale behind the word likely. There is no number, no band and nothing to compare against another image.
- Reasoning from stale tells. Models cite fingers, teeth, waxy skin and warped text, because that is what the training text says to look for. Those were fixed years ago.
- Agreement with your framing. Ask whether an image is fake and you are more likely to hear yes than if you had asked neutrally. That is a property of the interface, not of the picture.
- No reproducibility. Ask twice and you can get two different answers about the same file, with equally confident explanations for each.
- No region information. It cannot tell you that one corner of a photograph behaves differently from the rest, which is the finding that usually matters.
| Capability | Chatbot | Pixel detector |
|---|---|---|
| Reads pixel-level texture | No | Yes |
| Calibrated score with published bands | No | Yes |
| Same answer every time | No | Yes |
| Region-level breakdown | No | Yes |
| Measured against a benchmark | No | Yes |
| Explains its reasoning in prose | Yes fluently | Partly as signals |
| Describes what is in the picture | Yes | No |
The last two rows are the ones worth keeping. A language model is genuinely good at describing an image and at reasoning about whether a scene makes sense. Those are useful and they are not detection.
Where a chatbot is genuinely useful
Dismissing the tool entirely would be the wrong lesson. Used for what it can do, it complements a detector rather than replacing one.
-
Ask it to describe the scene in detail
A careful description often surfaces things you did not consciously notice, including objects that do not belong together or signage you had not read.
-
Ask whether the scene is physically consistent
Shadows, reflections, scale and perspective are reasoning problems, and reasoning is what the model is for. This catches composites that pixel analysis can miss.
-
Ask what would confirm or refute it
Turning a suspicion into a list of checks is a good use of a language model, and the list is often better than the one you would have written.
-
Then run an actual detector
For the pixel question, use something built for it, and read the score against a published scale.
Why the fluency is the danger
A detector that is unsure returns a middling number, and the number itself communicates the uncertainty. A language model that is unsure returns a paragraph, and paragraphs do not have error bars.
The same well-structured explanation appears whether the model has identified something real or produced a plausible-sounding account of nothing. Readers calibrate their trust on the quality of the writing, which is exactly the signal that carries no information here.
This is why asking a chatbot is a worse starting point than looking at the picture yourself. Your own uncertainty is at least visible to you.
The same problem in other tools
This is not really about one product. Any system that reasons about an image semantically has the same gap, and several tools now presented as detectors are doing exactly that underneath.
A useful test: ask what the tool would report on a photograph of a photograph, or on an image with no recognisable subject at all. A pixel detector returns a score either way, because texture exists regardless of content. A semantic system has nothing to reason about and will either refuse or invent something.
A second test: run the same file twice. A pixel model is deterministic and returns the same number. A language model samples, and two runs can disagree. If a tool gives you different answers for the same bytes, it is not measuring anything.
Neither test requires understanding how the tool works. Both take a minute, and between them they separate measurement from description more reliably than reading a marketing page.