Diffusion vs GAN: why detectors handle them differently

GANs leave a repeating artefact from upsampling that is almost a signature. Diffusion models do not, which is why detection got harder in 2022 and why architecture-specific tricks stopped working.

How a GAN builds an image

A generative adversarial network starts from a small vector and repeatedly upsamples it, doubling the resolution at each layer until it reaches full size. A second network judges the result, and the two train against each other.

The repeated upsampling is the important part. Each doubling step introduces a regular pattern, and those patterns accumulate into periodic structure that is visible when the image is transformed into the frequency domain.

That structure was close to a signature. Early detectors could look for a checkerboard pattern in the frequency spectrum and reach high accuracy without understanding anything about the picture. It was a fingerprint of the architecture rather than of any particular model.

How a diffusion model builds one

Diffusion works the other way around. It starts with a field of pure noise at the full target resolution and removes a little of it at each step, guided by what the model learned, until an image emerges.

There is no upsampling ladder, so there is no periodic artefact. The output is statistically different from a camera capture, and the difference is diffuse rather than structural. Nothing about it shows up as a clean pattern you can look for.

GAN

Small vector upsampled repeatedly
Periodic artefacts from each doubling
Detectable in the frequency domain
Architecture leaves a near-signature
Dominant until roughly 2022

Diffusion

Noise removed step by step at full size
No upsampling ladder, no periodic pattern
Difference is statistical, not structural
Needs a learned model to detect
Dominant since 2022

The two processes and what each leaves behind. The right-hand column is why detection had to be rebuilt.

Why the older detectors stopped working

A tool built to find frequency-domain checkerboarding finds nothing in a diffusion image. It does not report uncertainty; it reports that the artefact is absent, which reads as a clean result.

That is why detection accuracy across the field appeared to collapse between 2021 and 2023. The tools had not degraded. The thing they were looking for had stopped being produced.

What replaced architecture-specific detection

Breadth. Rather than looking for one artefact, current detectors are trained on output from as many different generators as can be collected, so the model learns what synthetic texture has in common instead of what one family produces.

The model used here was trained on millions of images from thousands of generators for exactly this reason. Coverage across architectures is what buys generalisation to the next one, and it is the only approach that has survived a change of dominant method.

The trade is that accuracy on any single known generator is lower than a specialist detector would achieve. That is the right trade, because a specialist is useless the day something new arrives.

What this predicts

The lesson from the GAN-to-diffusion transition is not about diffusion. It is that any detector tied to how images are currently made has a limited life, and the transition will not be announced in advance.

How each generation of detection aged
Approach	Worked on	Failed when
Frequency artefact search	GANs	Diffusion arrived
Face-specific analysis	Early face swaps	Whole-scene generation arrived
Metadata inspection	Files straight from a tool	Platforms stripped metadata
Error level analysis	Single-save JPEGs	Images started travelling
Broad multi-generator training	Most current output	Unknown, and later than the others

Does any of this matter to a user

Mostly it explains two things you will notice. First, why a detector cannot usually name which tool produced an image: it is reading a shared statistical property rather than a per-model fingerprint.

Second, why older free checkers perform badly. Several tools still online were built around GAN artefacts and have not been retrained. They return confident clean results on current output, and nothing in the interface says why.

What a hybrid pipeline does to a result

Most images that reach you were not produced by a single method. A diffusion model generates a base, a GAN-based upscaler enlarges it, a face restoration pass fixes the eyes, and an editor crops and re-saves the result.

Each stage rewrites texture, so the file carries traces of several processes layered over each other. The last one to touch the image usually dominates what a detector reads, which is why an upscaled diffusion image can look statistically more like the upscaler than like the generator.

This is the practical reason architecture identification does not work outside a laboratory. In a controlled test with clean single-model output it is a solvable problem. On an image that has been through a real production pipeline the question is not well posed.

It also explains a result people find confusing: a genuinely generated image that scores lower than a heavily upscaled photograph. Both have had their texture rewritten by software, and the detector is reading the rewriting rather than the origin.

Questions people ask

Are GAN images easier to detect than diffusion images?

They were, and there are far fewer of them now. GAN upsampling left a periodic artefact that a simple frequency analysis could find. Diffusion leaves no equivalent structure, so detection requires a trained model reading statistical texture rather than a targeted search for one pattern.

Can a detector tell me whether an image came from a GAN or a diffusion model?

Not from the pixels, in general. A broad detector reports how likely it is that something generative was involved, not which family produced it. Architecture identification is a separate research problem and it is considerably less reliable than detection.

Do GANs still get used?

Yes, in specific places. They remain fast and efficient for narrow tasks such as face synthesis and some upscaling, where diffusion would be slower. Some tools use both at different stages, which means an image can carry traces of either.

Will the next architecture break detection again?

It will reduce accuracy rather than break it, which is the improvement broad training bought. A model that learned what synthetic texture has in common across thousands of generators degrades gradually against something new, where a single-artefact detector fails completely.

Why do some tools still claim very high accuracy?

Usually because they are measuring on a test set built from generators they trained on. That figure is real and it does not describe performance on the model released last month. Ask which generators were in the benchmark before comparing numbers between tools.

Does this affect how I should read a score?

Only in one way. Treat a clean result on an image you have other reasons to doubt as weaker evidence than a flagged result, because the failure mode of an unseen architecture is a confident low score rather than an uncertain one.