News · ai music — By WATMM Desk —

Can a music generator tap along? A paper finds the beat by making MusicGen play drums

An 11 Nov 2026 arXiv paper from researchers at McGill, NYU and MBZUAI adapts MusicGen to generate a percussion track in time with a song, then reads the beats off that track. On the GTZAN test set it lands close to dedicated beat trackers and loses less accuracy when the drums are removed. We think the drumless result is the part producers should watch.

Key facts

  • What happened: "Can Music Generation Models Tap Along?", posted to arXiv on 11 Nov 2026, adds a small trained adapter to a frozen MusicGen so it generates percussion synchronised with an input song. A lightweight beat tracker then extracts beat positions from that percussion.
  • The result: on the 999 songs of GTZAN the three versions score an F-measure of about 0.85 at a 70 ms tolerance, against 0.888 for Beat This! and 0.870 for madmom. With drums removed from the input, the paper reports its model shows the smallest drop in F-measure.
  • Why it matters: the output is an audible percussion track, not only a list of timestamps. The tests cover the first 30 seconds of each song, drumless audio came from a stem separator, and audio quality had no listening test.

Beat trackers normally listen to a song and output a list of timestamps. A paper posted to arXiv on 11 November 2026, "Can Music Generation Models Tap Along?", tries a different route: get a music generation model to play a drum part along with the song, then read the beat off the drums. The authors are Kun Fang and Ichiro Fujinaga of McGill University and Ziyu Wang of New York University and MBZUAI (paper).

What did the researchers test?

The starting point is MusicGen, the autoregressive music model the paper builds on. The team connects two pretrained copies with what the abstract calls an "audio-conditioning adapter". One copy (MusicGen-large) listens to the input song. The other (MusicGen-small) generates a percussion track. Both stay frozen, and only the adapter is trained; the results table lists it at 10.4 million trainable parameters.

They trained three versions, each aiming at a different kind of percussion:

  • Click: a kick drum on downbeats and a hi-hat on other beats, like a metronome.
  • Groove (synth): rule-based patterns on synthesised kick, snare, hi-hat, cowbell and conga, with 14 genre-specific patterns.
  • Groove (sampled): human-performed drum MIDI from the Groove MIDI Dataset, aligned to the annotated beats bar by bar and rendered with drum samples.

At test time the model gets only the audio, with no genre label, and generates a 30-second track from the first 30 seconds of the song. A small beat tracker of about 143,000 parameters then pulls beat positions out of the generated percussion.

The comparison has two sides (paper, section 4.2). On the generative side: a diffusion-based alternative built on Stable Audio Open 1.0, and STAGE, an existing drum accompaniment model. On the conventional side: Beat This!, which the paper calls "one of the current state-of-the-art methods", and madmom, which it calls "a classic baseline". The held-out test set is the 999 songs of GTZAN, which the paper says none of the models has seen.

What did they find?

It gets close, but does not win. The headline metric is F-measure with a 70 ms tolerance, which checks whether each detected beat falls near a human-annotated one. From the paper's Table 1, on GTZAN:

MethodF-measure (70 ms)AMLt
Beat This!0.8880.895
madmom0.8700.923
MusicGen + adapter (Click)0.8510.907
MusicGen + adapter (Groove synth)0.8500.912
MusicGen + adapter (Groove sampled)0.8490.907
Stable Audio ControlNet0.6910.753
STAGE0.3350.482

The choice of percussion target, the authors write, "has almost no effect".

AMLt is a looser score that still counts a beat as right if the model taps at double tempo, half tempo or on the off-beat. The adapted MusicGen models are below Beat This! on F-measure but above it on AMLt. The authors read that as continuity: the models "mostly keep up with the song, and the main errors lie in the metrical level or phase rather than in losing the song entirely".

It holds up better without drums. The researchers removed the drums from the input audio with the HT Demucs separator, keeping all other settings unchanged, and compared the methods again. The abstract says the model "shows the smallest drop in F-measure when drums are removed from the input at inference time". Because the main model had seen drum-removed audio in training and, as far as the authors know, the conventional trackers had not, the team also trained a version without that advantage. Even that version dropped less than Beat This! and madmom in both F-measure and AMLt, the paper reports.

It is less easily fooled by loud hits. The authors say they often see beat trackers put beats on loud onsets even when those are not beats, and miss beats at soft positions. On drum-removed metal and disco, the paper's model had the lowest miss rate in every onset-strength group. Its false-positive rate stayed low as off-beat onsets got stronger, whereas those of Beat This! and madmom "rise steeply". In one disco excerpt the two conventional trackers shifted by half a beat onto the off-beat for almost the whole passage once the drums were gone; the generative model stayed on the annotated beats.

The trade-off is sound quality. The two other generative systems scored higher on an automatic production-quality measure. By ear, the authors say, those tracks "sound more like real performances, but they are often misaligned with the beat or add rather arbitrary fills". You can judge for yourself on the demo page, which hosts listening examples.

What are the limits?

The paper lists its own. The generative models' training data are not entirely the same, so it calls this "a comparison of existing methods rather than of model architectures". The drumless audio "comes from source separation and is not equivalent to music that naturally has no drums". And audio quality "was assessed only preliminarily, without a listening test". Every method was scored on the first 30 seconds of each song only.

What we think

Everything in this section is our analysis, not a claim from the paper.

The drumless result is the one that maps to real sessions. Tempo detection in a DAW is usually fine on a mastered track with a kick on every beat. It gets shaky on the material people most often need to conform: a solo piano take, a guitar-and-vocal demo, a field recording, a stem with the drums muted. The paper tests a stand-in for that condition, songs with the drums removed by a stem separator, and it is where the generative approach closed the gap with the best dedicated tracker. If that survives on music that never had drums, rather than music with drums removed by a separator, it is a meaningful step for anyone who builds a tempo map around a free-time recording.

An audible guide track is a better interface than a row of markers. When a beat grid is wrong, you find out by squinting at warp markers or by hearing the metronome flam against the track. A generated percussion part makes the error audible straight away, in a musical form. We can imagine this as a "scratch drummer" you audition against an imported take before you commit the grid, then throw away. Nothing like that ships today on the back of this paper, and the low production-quality score for the most realistic drum target says the drums are a guide, not a keeper.

A 70 ms tolerance is not grid-tight. By our arithmetic, a sixteenth note at 120 BPM lasts 125 ms, so a beat can sit more than half a sixteenth away from the annotation and still count as correct at this tolerance. That is the tightest of the three tolerances the paper reports (70, 100 and 150 ms) and is no criticism of the paper, but it means an F-measure of 0.85 does not tell you whether the result is tight enough to quantise or chop to. Expect to nudge markers by hand, as you do now.

The double-time and half-time problem is still here. The paper says the model's main errors are in metrical level or phase. Producers know that failure well: the DAW reads a 140 BPM track as 70, or puts the "one" on the snare. A generative tracker that keeps steady time but picks the wrong level is easier to fix than one that wanders, since doubling a tempo is one click, though it is still a fix.

It is a heavy way to find a beat. The pipeline runs a large MusicGen to listen and a small one to play, where the results table lists madmom at 0.73 million parameters. For a one-off analysis of an imported file that cost may be acceptable. For real-time tracking of a live player it looks impractical as described, and the paper does not test it.

The wider point is about what generators know. This is a research result on 30-second excerpts, not a product. But it adds to the evidence that music generation models carry a usable sense of time, and that a small trained add-on can pull it out. For producers, we think the practical payoff of generative models over the next couple of years may come less from finished songs and more from this kind of listening task: finding the beat, following a player, keeping a part in time with what you recorded.

Sources

  1. Can Music Generation Models Tap Along? Evaluating Generative Approaches to Beat Tracking, arXiv (Fang, Wang, Fujinaga) (primary source). Accessed 14 Nov 2026.
  2. Can Music Generation Models Tap Along? full paper (HTML, v1), arXiv (primary source). Accessed 14 Nov 2026.
  3. Generative Beat Tracking: Listening Examples, Kun Fang (project demo page) (primary source). Accessed 14 Nov 2026.

Researched with AI assistance, written and checked by WATMM Desk.

Subscribe to We are the music makers

One email a week: the music-tech and AI-in-music news that mattered, and what we think about it.

Sign-ups open soon.