A quantized latent representation replaces continuous latent vectors with discrete codebook vectors. In VQGAN, each latent vector maps to its nearest codebook entry, producing a compact sequence of integer indices. section 11.5.2
The conversion proceeds in three stages:
- Encode an image into a continuous latent grid containing 576 feature vectors. section 11.5.2
- Match every latent vector to one of 1,024 codebook vectors and record its index. section 11.5.2
- Look up the indexed vectors to rebuild zq, then pass it through the decoder to reconstruct the image. section 11.5.2
Image tokens
If the first two indices are 873 and 862, they select codebook vectors 873 and 862. Repeating this lookup for all 576 indices reconstructs the quantized latent grid. section 11.5.2
z_q, _ , indices= model.quantize(z) #1 int_sequence = indices[2] #2 print(int_sequence)
Inspect the quantization call that returns the quantized vectors and the integer index sequence. section 11.5.2
