AI image detector accuracy: what 91% actually means

91.3% balanced accuracy means roughly one image in eleven lands on the wrong side of the decision line, under laboratory conditions on clean files. Your image is not a laboratory condition.

What balanced accuracy measures

Plain accuracy is easy to inflate. Feed a detector a test set that is 90 percent generated images and a model that says yes to everything scores 90 percent while being useless.

Balanced accuracy fixes that by weighting real and generated images equally, so the figure is the average of two rates: how often genuine photographs are correctly cleared, and how often generated images are correctly flagged. A tool quoting plain accuracy without naming its test mix is not telling you much.

Balanced accuracy by JPEG quality

Clean originals

91.3 %

JPEG quality 60

87.3 %

JPEG quality 40

84.6 %

Axis starts at 80% so the slope is readable. A full 0-100 axis would hide the effect entirely.

The published figures for the model used here, measured on its research benchmark.

The two ways a detector is wrong

One number hides two very different failures, and they carry very different costs.

False positive

A genuine photograph scored above the line
Caused by upscaling, retouching, night mode, screenshots
Cost: an accusation against a real person
The failure that does actual harm

False negative

A generated image scored below the line
Caused by new generators, small edits, laundered files
Cost: a fake gets through unchallenged
The failure people notice less

A single accuracy figure averages these two. They are not interchangeable, and most people care far more about one of them.

Moving the decision line trades one for the other. Lowering it catches more fakes and flags more real photographs. Raising it protects genuine images and lets more fakes through. There is no setting that improves both, which is why the line is published at 65 rather than tuned quietly.

Why benchmark figures overstate real performance

The gap between a test set and an image somebody sends you
Benchmark image	Your image	Effect on the score
Original file, never re-saved	Compressed two or three times	Accuracy falls by several points
Full resolution	Cropped or downscaled for a chat app	Less detail to read, more uncertainty
Generators known at training time	A model released last month	The standing weakness of every detector
Clean photograph or clean render	Real photo with one AI edit	Whole-frame score understates it
No adversarial processing	Deliberately laundered to pass	Designed to defeat the measurement

None of these are unusual cases. They describe how images actually travel. A picture that reaches you through two platforms and a screenshot has already given up much of the signal the benchmark was measured on.

How to read an accuracy claim

Ask what test set. A figure without a named benchmark is marketing. A figure on a public benchmark can be checked by somebody else.
Ask balanced or plain. Plain accuracy on a lopsided test set is the easiest number to inflate.
Ask where the decision line sits. Accuracy is meaningless without knowing the threshold it was measured at.
Ask which generators were included. Any detector scores well on the models it trained on and worse on the one released after.
Ask for the false positive rate separately. That is the number that decides whether the tool is safe to use on real people.

What accuracy cannot tell you

A benchmark measures how often a model is right across a population of images. It says nothing about whether it is right on yours. There is no confidence interval attached to a single result, and a detector cannot know which of its errors it is making.

This is why the region map and the file evidence matter more than the headline figure. Agreement between independent signals is worth more than a strong number from one of them, and disagreement is a finding in itself.

How to test a detector yourself

You do not have to take anyone’s figure on trust. A useful test takes an afternoon and tells you more about a tool than any published benchmark, because it uses images that look like yours.

Build a set you already know the answer for
Twenty photographs you took yourself, and twenty images you generated. Keep the originals untouched. Twenty of each is enough to expose an obviously weak tool.
Include the awkward middle
Add screenshots of your own photos, night-mode shots, an upscaled crop and a heavily retouched portrait. This is where detectors fail, and it is the part every vendor benchmark leaves out.
Record the score and the band, not a verdict
Write down what each tool returned. Comparing bands is more informative than comparing numbers, because two tools can put their decision lines in different places.
Count the two error types separately
How many of your real photos were flagged, and how many of your generated images were missed. A tool that never misses a fake by flagging a third of your own photographs is not usable on people.

The number that matters is the first one: how often it calls your own work fake. That is the failure with a cost attached, and it is the one a headline accuracy figure averages away.

Questions people ask

Is 91% accuracy good for an AI image detector?

It is a reasonable figure for a pixel-based detector on a clean benchmark, and it is not high enough to act on alone. Nine wrong calls in a hundred is a lot when each one may be an accusation. Treat it as a screening tool that tells you where to look rather than a test that settles the question.

Which AI image detector is the most accurate?

The honest answer is that published figures are not comparable. Different tools use different test sets, different decision lines and different definitions of accuracy, so the numbers measure different things. Compare what a tool shows you instead: published bands, a region map, and a stated false positive rate.

Does accuracy improve as models get better?

Detection and generation improve together, so the gap stays roughly constant. Every new generator is unseen data for every existing detector, and retraining follows release rather than preceding it. Expect a permanent lag rather than a solved problem.

Why does the same tool give different scores for the same picture?

It should not, and if it does the two files differ. A copy that has been re-saved, resized or passed through a platform is a different file with different pixels. Compare the original against the original, not the original against a download of itself.