How do AI image detectors work, and why do they disagree?

Three different things called "detection"

People use one word for three different checks, and they fail in different ways.

Method	What it looks at	Typical failure
Trained classifier	Pixel statistics and frequency patterns in the image	Fails on generators it was not trained on and on images that were resized or recompressed
Watermark detector	An invisible mark embedded by the generating vendor	Only answers for images made by tools that embed that mark
Provenance check	A signed Content Credential in the file	Nothing to read once metadata is stripped

Only the first is a guess based on the picture. The second and third read something a tool put there on purpose. The C2PA explainer describes provenance as complementing media literacy, fact-checking and "digital forensics approaches such as deep-fake detection", not replacing them.

How a trained classifier works

The research literature explains the idea well. In a study of diffusion model detection, Corvi and colleagues note that most state-of-the-art detectors rely on traces invisible to the human eye. Any method that creates an image leaves peculiar traces tied to the steps of the generation process, and each architecture has its own. The authors average the noise residuals of 1,000 images per generator and look at the spectrum. For several generators, including Stable Diffusion and Latent Diffusion, they see clear peaks, a sort of artificial fingerprint. For others, such as DALL-E 2 and ADM, the peaks are much weaker, which predicted more difficult detection.

So a classifier is learning "what do images from these generators look like at the pixel level?" It never sees your intent, the scene's plausibility or the file's history.

Why they disagree with each other

1. Different training generators. The same paper trained detectors on one generator and tested on others. Performance varied a lot from model to model, and the authors conclude that generalization to unseen architectures remains the main hurdle. A detector tuned for one family can be near chance on another.
2. Different thresholds. The paper reports that even where ranking quality (AUC) looked good, accuracy at a fixed threshold was often poor, because a threshold chosen in training did not fit images of other origin. A tool that says "92 percent AI" and one that says "likely real" may both be working as built, with different cut-offs.
3. Different treatment of resizing and compression. In their test, an image was cropped, resized to 200 by 200 pixels and JPEG compressed at a random quality from 65 to 100, mimicking social platforms. Detector accuracy generally dropped. In the paper's Table 1, for example, the average accuracy of the best detector trained on ProGAN fell from 75.8 percent on uncompressed images to 61.5 percent after resizing and compression, and for several generators it was close to 50 percent, which is chance. The authors say the traces are weak and can be removed even by non-malicious processing.
4. Real photos are compressed, synthetic PNGs often are not. The authors note that real camera images carry JPEG compression artifacts while raw generator output does not, which makes detection much easier on uncompressed synthetic files. Once everything is recompressed, that easy cue is gone, which is one reason a re-saved image behaves differently from the original.
5. Old and new generators. That paper tested 2022-era models. Current generators and detectors have both moved on, so use it for how the problem works, not for the present accuracy of any product.

What helps in practice

The Gemini verification help page lists other signals to use alongside a watermark or credential check: garbled text, errors in hands, teeth or background details, inconsistent shadows and lighting, repetitive patterns, a reverse image search to find the original source, and metadata such as EXIF camera data if you have the original file rather than a screenshot. Each is weak alone, and the paper's authors note that visual tells like lighting inconsistencies may disappear as generators improve.

A sensible order:

1. Provenance first. Look for a Content Credential on the original, as covered in the Content Credentials thread and the start-here guide.
2. Watermark check where it applies. See what SynthID can and cannot show.
3. Source and context. Who posted it first, and can they produce the original or the layered file?
4. Then a classifier, if at all. Run more than one, record each exact output, and expect disagreement.

Be careful with false accusations

A classifier flagging a human-made image is a known risk. If someone's work is labelled AI, ask for process files, not a second score. One member's experience is in the print-site labelling thread, and the same logic applies when you are checking a student's or colleague's file: a detector result alone should never be the basis for an accusation.