Last updated
The limit on these numbers
The tables below are pixel-model component results for Community Forensics frontier-2026-08, measured on 2026-10-28 on a local CPU. They did not call the production API and did not upload the images. Full-system testing (Gemini plus the pixel model plus metadata) is still in progress, so this page does not give a full-system number.
What a check actually runs
- Image visuals are read by a Google Gemini vision model. Video, audio, and text are also reviewed by Google Gemini. The pixel score is not written into the Gemini prompt.
- The pixel model is Community Forensics. The Worker picks thresholds from the revision the sidecar returns. frontier-2026-08 (Thermostatic/community-forensics-frontier-detector-2026-08, revision 16db135220b3) is scored on the raw logit: 7.0 moderate, 9.5 strong, 10.0 a stricter keep-gate. The sidecar default in this repository is still the upstream OwensLab/commfor-model-384 model until the server sets COMMFOR_PRIMARY_MODEL to frontier-2026-08. The figures on this page were measured on frontier, not on that upstream model.
- The browser reads an embedded C2PA Content Credential locally when an image still has one. A missing credential is not evidence that the file is fake. The video upload does not use that image credential reader.
- Unsigned XMP is not a cryptographic proof. If it declares trainedAlgorithmicMedia, algorithmicMedia, or ContainsAiGeneratedContent as yes, true, or 1, the current product rule sets an AI verdict and a score of at least 90. compositeWithTrainedAlgorithmicMedia, compositeSynthetic, or a Generative Edit note raises the result to review at least, and does not pull an existing AI verdict back down. That parser shipped on 2026-10-29. The bake-off below was run before the fix and has not been repeated.
- A video check samples up to 10 frames across the clip, sends them to face forensics, and Gemini reviews those frames. If the face service fails, Gemini still runs. See the video page. Video check
How the image signals combine
The frontier rules below apply only when the sidecar returns frontier-2026-08. The pixel model alone does not turn a “real” verdict into “AI”.
- Gemini proposes the visual verdict first.
- If Gemini says AI but the visual evidence is not consistent enough: raw logit ≥ 7.0 keeps the AI verdict and does not lower the score; 0 ≤ logit < 7.0 keeps Gemini’s AI verdict and score; logit < 0 becomes review without capping the score at 74. With no pixel result, the older downgrade still applies and the score is capped at 74.
- A stricter gate still exists (logit ≥ 10.0, short edge at least 256, normal reliability, not treated as a screenshot, JPEG quality above 60). The logit rule above still runs when that gate blocks.
- If Gemini says real and the frontier logit is ≥ 9.5, the result rises only to review, not to AI.
- A strong XMP declaration can set an AI verdict on its own. That is a separate rule from the pixel model.
October 2026 pixel bake-off
Main slice: 477 images, 254 AI and 223 real. The AI images are 167 from the Rapidata text-to-image human-feedback benchmark (CC-BY-4.0, 2026-09-30, photoreal prompts, platform-reencoded WebP), 80 from the AIGenImages2026 validation set (CC-BY-4.0, native PNG or JPEG), and 7 AI images from an older 16-image set. The real photos are 215 Wikimedia Commons originals uploaded after 2026-08-15 (113 phone, 102 camera) plus 8 from that older set. The AI images cover 66 generator versions. A screenshot slice of 31 (16 AI, 15 real), mostly simulated social-feed screenshots, is not part of the 477.
Read this confound with the table: AI images had a median of about 1.05 megapixels, real photos about 13.4. The original-file condition is closer to a production upload, but the resolution mix of real user uploads was not measured. The web1024q80 condition resizes both to a short edge of at most 1024 and JPEG quality 80, which is the fairer comparison. Thresholds were fit on this same dataset, so the operating points are in-sample and optimistic.
AUC
| Model | Original files | Short edge ≤1024, JPEG q80 |
|---|---|---|
| frontier-2026-08 | 0.953 | 0.969 |
| Previous model (upstream) | 0.793 | 0.858 |
AUC is on the logit, main slice of 477. Source: the 2026-10-28 pixel bake-off report.
Hit rate at a matched false-positive rate
At about a 2% false-positive rate (4/223 real photos), frontier on originals caught 178/254 (70.1%, Wilson 95% interval 64.2–75.4%). The previous model caught 55/254 (21.7%, interval 17.0–27.1%). On web1024q80 at that same false-positive count, frontier caught 212/254 (83.5%, interval 78.4–87.5%).
At about 1% (2/223), frontier on originals caught 169/254 (66.5%, interval 60.5–72.1%). At about 5% (11/223), frontier on originals caught 211/254 (83.1%, interval 78.0–87.2%) and the previous model caught 83/254 (32.7%, interval 27.2–38.7%). Source: operating_points.csv, 2026-10-28.
Frontier bands the Worker uses
These thresholds were fit on the originals in the same 477, so they are optimistic too. Source: the 2026-10-28 compression test. Raw logit ≥ 7.0: AI hit 81.9%, real false positives 4.9% (11/223). ≥ 9.5: AI 70.1%, real 1.8% (4/223). ≥ 10.0: AI 66.9%, real 1.3% (3/223).
The previous model at its own score bands, on originals: ≥90 caught 19.7% of AI images with 0.9% false positives; ≥95 caught 16.9% with 0.4% false positives. If frontier’s calibrated score were still read at ≥90, AI detection would be 92.5% (89–95%) but false positives on real originals would be 22.4% (50/223, 17–28%). That is why production does not use a score of 90 for frontier.
What compression did
The model only sees a 384×384 center crop after the short edge is resized to 440. Downscaling barely moves the logit by itself. An extra JPEG generation does: about −0.5 on average at quality 95, and about −1.0 at quality 90. Re-encoded copies raised AUC from 0.953 to about 0.96–0.97, but the logits shifted down, so the current thresholds would not transfer without a new calibration.
One conditional rule (keep the original when it is at most 2 MB, the long edge is at most 2048 px, and the format is JPEG, PNG, or WebP; otherwise use a long edge of 2048 and JPEG quality 95) moved AI detection at logit 9.5 from 70.1% to 69.7%, and real false positives at 9.5 from 1.8% to 1.3%, with 2 of 477 images crossing 9.5. The comparison is biased: only 21/254 AI images were large, versus 217/223 real photos, so the rule mostly transforms real photos.
Screenshot slice, 31 images: AUC 0.925 on the originals. At logit ≥ 9.5, AI detection was 25% and the false-positive rate on real screenshots was 0%. Compression variants in that test stayed at the same 25% / 0%. At the looser calibrated score of ≥90, frontier on that slice was AI 14/16 and real 4/15. Do not mix those two thresholds.
Known weaknesses
- Screenshots and photos of screens. At the strong logit of 9.5, the screenshot slice caught 25% of the AI images.
- Some generators are weaker, and those subgroups are small. On originals at a calibrated score ≥90, frontier was under 80% on GPT Image 2.5 Sunburst (3/5), Recraft V4.1 (2/4), Reve 2.1 (3/5), Ideogram 4.0 (1/3), and Midjourney v7 (6/8). The intervals are wide.
- Indoor and low-light real photos had higher false positives at the looser ≥90 score. The live strong band is much stricter than 90.
- AI and real images differ in resolution and encoding. Part of the original-file gap may be “small web image versus large camera JPEG.”
- A test from before the XMP fix does not reflect the metadata rule that shipped on 2026-10-29. The set has not been re-run since.
Our result on this four-image set
deepfakecheck.io's own result on this set will be added after a re-test on the current production version.
When to get a second opinion
- The decision is legal, financial, employment, or a news publication.
- The file is a screenshot, a social-app download, or heavily recompressed.
- Pixel, visual, and metadata signals disagree.
- You need the generator name, or a SynthID read. This product does neither.