x is the input sequence of music-token indexes used to predict the next musical event in a training pair. section 14.1.3 section 14.2.3
It is formed by converting music into token indexes, then slicing fixed-length sequences and pairing each input with a one-step-shifted target. section 14.1.3 section 14.2.3
In this training setup, x supports the model workflow by:
- holding the current music context for each example. section 14.1.3
- feeding the forward pass before loss is computed against the shifted y sequence. section 14.1.3
- using 2,048 indexes per sequence to capture long-range musical dependencies. section 14.1.3 section 14.2.3
Music training pair
If a music piece exceeds 2,048 indexes, x uses the first 2,048 indexes and y uses the next indexes shifted one position to the right. section 14.2.3
Listing 14.5 Creating training data
import torch,os,pickle
max_seq=2048
def create_xys(folder):
files=[os.path.join(folder,f) for f in os.listdir(folder)]
xys=[]
for f in files:
with open(f,"rb") as fb:
music=pickle.load(fb)
music=torch.LongTensor(music)
x=torch.full((max_seq,),389, dtype=torch.long)
y=torch.full((max_seq,),389, dtype=torch.long) #1
length=len(music)
if length<=max_seq:
print(length)
x[:length]=music #2
y[:length-1]=music[1:] #3
y[length-1]=388 #4
else:
x=music[:max_seq]
y=music[1:max_seq+1]
xys.append((x,y))
return xys
Shows the create_xys() routine that builds x and y pairs, including padding, shifting, and end-of-sequence handling. section 14.2.3
Figure 14.2 The training process for a music Transformer to generate music
Diagram of the music Transformer training flow where x is the input sequence and y is the shifted target. section 14.1.3
