Session notes: how Holly Herndon made PROTO with an AI called Spawn, from what she said in 2019

Most AI-music stories in 2026 are about prompts: type a sentence, get a song. PROTO, released on 4AD in May 2019 (MusicTech, Dazed), is the opposite case and still the most useful one. Holly Herndon, Mat Dryhurst and developer Jules LaPlace built and trained their own AI, called Spawn, on voices they recorded themselves (Vice). Everything below comes from Herndon's 2019 interviews. Where she did not say something, we do not guess.

The rig

Small, by AI standards. Herndon told Dazed: "We have one GPU unit", contrasting it with corporate data centres. The software side was credited to LaPlace: "Jules should be given the credit for reading the technical papers and installing the software" (MusicTech). One technique she named was SampleRNN: "we trained it on my voice, specifically" (MusicTech).

How the record came together

1. Home-made data first. "We started training it on my voice and Mat's voice, and some little Foley sounds around the house," she told MusicTech. She said she focused "on the development of the training sets with Mat".
2. Then the public. Vice reports that about 300 people performed vocals for training. "We would have the audience snapping their fingers, or tapping their beer bottles with their keys, or clapping," Herndon said, explaining that Spawn responds well to transients, the sharp burst at the start of a sound. She added: "Sometimes we found ourselves doing things to please her." Dazed reports she also recorded her tour audiences to build voice models.
3. Scripted speech for voice models. "We used a thing called a TIMIT script," she told Vice, a script that checks a voice model covers every sound in a language.
4. Performer, not composer. "We wanted to approach Spawn as a performer rather than a composer", and to "use sound as material rather than midi data scores" (Vice). To MusicTech she called Spawn "an ensemble member".
5. Slow loops, not live jams. "The training itself can't happen in real-time, because that takes weeks to process" (Dazed). And: "It's not like I sat down and wrote sheet music and trained things and then we all performed together" (MusicTech).
6. Many failures before one sound. On "Godmother", made with Jlin, the team tried a number of techniques before the voice-model approach worked: "the Holly voice model is performing Jlin tracks" (Vice). "On Birth and Godmother, that's all Spawn" (MusicTech).

What we think a producer can take from it

• Own the data, own the sound. Spawn sounds like Herndon because it was trained on Herndon. Seven years on, that is the clearest answer to the consent arguments around today's generators: the voices were the band's own, and the public sessions were opt-in.
• Treat the model as a player. Asking an AI for raw audio to arrange, not a finished song, keeps the composing with you. That is a workflow anyone can copy with today's voice and timbre tools.
• Latency shapes the music. Weeks-long training forced a back-and-forth way of working. Slow tools are not only a cost; they push you to commit to ideas between passes.
• Expect most experiments to fail. The account of "Godmother" is several attempts and one keeper. Budget time for that.

The caveat: one GPU in 2019 was a research-lab-style project run by people with time, skills and a label behind them. The lesson that transfers is the attitude to data and authorship, not the exact pipeline, which today's tools have overtaken.