Myth check: "AI detectors can reliably tell if a song is AI-generated"

The claim: AI music detectors can reliably tell whether a song was made by AI, so a detector's score settles the question.

Verdict: misleading. Detectors can be close to perfect on the generators they were trained on. The one public side-by-side test shows they disagree wildly on the same files, and a published research detector collapsed on a generator version it had not seen. A score is a lead, not a fact.

Where it spread

From the vendors themselves. Deezer's FAQ says its detection system "has an accuracy of 99.8%", missing about 2 in 1,000 AI tracks and wrongly flagging fewer than 1 in 10,000 human ones. IRCAM Amplify's home page advertises its AI Music Detector with "99% accuracy" and "Less than 1% false positives". Repeated in coverage and in arguments about flagged releases, those numbers turn into "detectors can tell".

What the evidence says

Vendor numbers are measured on vendor data. Neither page above publishes the test set behind its figure, so nobody outside can check how it was measured.

On one shared test, the detectors disagreed. Intrect Research ran eight public detectors on the same 2,104 files (1,388 AI, 716 real) at each one's public 0.5 threshold, published 14 Sep 2026. The share of real music flagged as AI ran from 1.8% (FST, which caught only 58.7% of AI tracks) to 97.8% (DeepFense). The public Deezer research model caught 64.6% of AI tracks and flagged 13.0% of real ones. Intrect makes ArtifactNet, which ranked first, so this is not an independent test, and the Deezer model tested is the public research release, not necessarily what Deezer runs in production.

New generators break old detectors. Deezer's own ISMIR 2025 paper reports perfect scores on Suno v3.5 and Udio 130, which it trained on, and 39.83% on Udio 32, which it did not. The authors suspect Udio changed its architecture. They also flag that manipulations such as resampling or pitch shift could affect the model.

Even Intrect, which sells a detector, writes: "Detection is not provenance," and says to treat a result "as forensic evidence to review, not a verdict to enforce automatically."

What we think

"Reliable" hides the question that matters: reliable on what? A detector is a pattern-matcher for generators it has learned. The day a new model ships, or a track goes through a different codec, the advertised number stops describing it, and the vendor's test set will not show you that. Both headline figures can be true and the claim still misleading.

The cost lands on the wrong people. At a 1% false-positive rate, a distributor screening a million human releases wrongly flags about 10,000. That is our arithmetic on IRCAM Amplify's own ceiling, not a measured figure, and the benchmark rates are far higher. Platforms using detectors should publish the false-positive rate at the threshold they actually use, on what test set, and run a human appeal with a deadline. Until they do, treat any score as a reason to look, not a verdict.

For the benchmark in detail, read AI music detectors are not proof. If a score has already landed on your release, use our appeal kit, and for why labels and watermarks don't settle it either, see SynthID and Content Credentials.