A music Transformer generates music by predicting the next event in a sequence of earlier events. Unlike MuseGAN, which treats a piece as a multidimensional object and generates it as a whole, this GPT-style model extends a musical sequence one event at a time. Self-attention helps it use context across distant events. section 14.0 section 14.1
The path from performance to prediction has three steps:
- Convert MIDI performances into note-on, note-off, time-shift, and velocity events. Assign each distinct event an index, much as text tokens receive indexes. section 14.1
- Train on sequences of up to 2,048 indexes. Shift each target sequence one index ahead of its input so the model learns to predict the next event from preceding events. section 14.1.3
- Give the trained model a prompt of event indexes, append its next sampled index, and repeat. Convert the resulting sequence back to MIDI to hear it. section 14.4 section 14.4.2
The model is decoder-only: generation needs earlier music events, not a separate source sequence to encode. It adds token and learned position embeddings, passes them through six decoder layers with causal self-attention, then produces logits for a 390-token vocabulary. Its 512-dimensional embeddings and eight attention heads make a 20.16-million-parameter model. section 14.1.2 section 14.3.2
Continue a piano performance
Take the first 250 events of a test performance as a prompt. Sample until the sequence contains 1,000 indexes, then decode it to a MIDI file. The temperature parameter adjusts the creativity of the continuation; a sample with temperature 1.5 provides a higher-temperature comparison. section 14.4.2
Listing 14.7 A sample() function in music generation
softmax=torch.nn.Softmax(dim=-1)
def sample(prompt,seq_length=1000,temperature=1):
gen_seq=torch.full((1,seq_length),389,dtype=torch.long).to(device)
idx=len(prompt)
gen_seq[..., :idx]=prompt.type(torch.long).to(device)
while(idx < seq_length): #1
y=softmax(model(gen_seq[..., :idx])/temperature)[...,:388] #2
probs=y[:, idx-1, :]
distrib=torch.distributions.categorical.Categorical(probs=probs)
next_token=distrib.sample() #3
gen_seq[:, idx]=next_token
idx+=1
return gen_seq[:, :idx] #4
The sampling loop shows the autoregressive step: turn logits into probabilities, sample an event index, append it, and repeat until the requested length. section 14.4.2
