Text-to-MIDI vs text-to-audio: why symbolic music generation matters to artists

Type "melancholy piano ballad in D minor, 70 bpm" into an AI music tool and you will get one of two very different things back. Either a recording, a waveform that already sounds like a piano in a room, or a set of notes, a MIDI file that sounds like nothing until you route it to an instrument. The first is text-to-audio. The second is text-to-MIDI, also called symbolic generation.

The difference looks technical. For an artist it decides who controls the song afterwards. If you want the practical side, which DAWs and plug-ins generate MIDI today, we covered that in MIDI generators inside your DAW. This guide is about the idea underneath: what each kind of model actually outputs, and why it matters.

What MIDI is, and what it is not

The MIDI Association's own overview is blunt: "MIDI is not audio." It is not a digital version of a sound recording. It calls MIDI "a descriptive language", closer to sheet music than to a WAV file. A MIDI file carries instructions: which notes to play and for how long, how hard to play them (velocity), which instrument to use (program change), plus information such as tempo and relative volume. The sound comes from whatever device or plug-in reads those instructions.

That one property is the whole argument. A note in a MIDI file is a separate object. You can move it, delete it, transpose it, change its length or hand it to a different synth. A note inside an audio file is baked into the waveform with everything else.

How a text-to-audio model works

Meta's MusicGen is a well-documented example, because its authors published the method. Their 2023 paper describes a single-stage transformer language model that works over "compressed discrete music representation, i.e., tokens". It is conditioned on a text description or on melodic features and produces those audio tokens, which are decoded back into sound, in mono or stereo.

The output is a recording. Melody, harmony, rhythm, timbre, mix and room sound all arrive fused together. That is the appeal: you get something that sounds finished in seconds. It is also the limit. If the bass line is almost right, you cannot open the file and fix two notes. You regenerate, or you reach for stem separation and audio editing, which brings its own artefacts, as we set out in our stem separation guide.

How a text-to-MIDI model works

The research system text2midi, presented at AAAI 2025, shows the symbolic route. Its authors use a pretrained language model to encode your caption. That encoding conditions an autoregressive transformer decoder, which writes a MIDI sequence that matches the description. They report automated and human evaluations, and say the output can be steered by captions that use music-theory terms such as chords, key and tempo. Code and samples are public.

The output is notes, not sound. You still choose the piano, the reverb, the performance and the mix. That is more work, and it is the point.

Side by side

Why symbolic generation matters

It keeps the decisions small. Songwriting is thousands of small decisions: this chord or that one, push this note a sixteenth later. MIDI lets you make each of them. Audio generation asks you to accept or reject a whole take.

Your sound stays yours. A generated recording carries the model's idea of what a piano or a voice sounds like. A MIDI part plays through your instruments and your patches, so the record sounds like you, not like the model's average.

It fits how producers already work. Every DAW reads MIDI. A generated part lands next to your own takes as just another clip, with nothing new to learn.

It leaves a clearer human trail. The US Copyright Office's report on copyrightability says prompts alone do not make you the author of what a model produces, but that creative selection, arrangement or modification of generated material can be protected, case by case. Editing notes, rewriting a line and voicing the chords yourself is exactly that kind of modification, and a MIDI workflow makes it natural and easy to document. This is not legal advice, and it applies to US registration only.

The honest limits of text-to-MIDI

• It does not sound like anything on its own. A great MIDI line on a poor patch still sounds poor. Production skill matters more, not less.
• Notes are only part of music. Groove, tone, a singer's phrasing and the room are where a lot of feeling lives, and MIDI describes only some of that.
• Text is a blunt instrument for notes. "Hopeful chorus" leaves a lot to guess. Captions that name a key, tempo and chord movement give the model more to work with, which the text2midi authors' own framing reflects.
• Research is not product. text2midi is a published research model. What a commercial tool does under the hood is often not documented, so check what it actually exports before you build a session around it.

What we think

Text-to-audio is impressive and useful for demos, references and mood boards. But for artists who want to make records that are theirs, the symbolic route is the more important one. It treats the model as a co-writer that drafts a part, not a jukebox that hands over a finished track, and it leaves the decisions that make a song personal (which note, which sound, which take) with the person whose name is on it.

We would like to see more tools export editable notes and stems by default, and fewer that only offer a stereo file. A practical rule for now: if a tool gives you only audio, use it for inspiration. If it gives you MIDI, use it as a starting point and then make it yours.