In music-token training pairs (x, y), y is the target sequence: at each position, it holds the token immediately after the corresponding token in x. The music Transformer uses x to predict y, learning next-event prediction rather than reproducing its input. section 14.1.3
MIDI performances are converted into indexed musical events. Each x and y has 2,048 indexes, a length chosen to capture long-range musical relationships while keeping training manageable. For a longer piece, x takes the first 2,048 indexes and y takes indexes two through 2,049. section 14.1.3 section 14.2.3
When a piece is shorter
For a piece of 15 indexes, x contains the piece followed by padding. The first 14 positions of y contain indexes two through 15; its next position contains the end-of-sequence index 388, and the rest contain padding index 389. Both sequences remain length 2,048. section 14.2.3
Listing 14.5 Creating training data
import torch,os,pickle
max_seq=2048
def create_xys(folder):
files=[os.path.join(folder,f) for f in os.listdir(folder)]
xys=[]
for f in files:
with open(f,"rb") as fb:
music=pickle.load(fb)
music=torch.LongTensor(music)
x=torch.full((max_seq,),389, dtype=torch.long)
y=torch.full((max_seq,),389, dtype=torch.long) #1
length=len(music)
if length<=max_seq:
print(length)
x[:length]=music #2
y[:length-1]=music[1:] #3
y[length-1]=388 #4
else:
x=music[:max_seq]
y=music[1:max_seq+1]
xys.append((x,y))
return xys
The create_xys() listing shows both branches: shifted slices for long pieces and an end marker plus padding for short ones. section 14.2.3
