A VQGAN decoder reconstructs an image from a quantized visual representation. It is the image-producing end of a VQGAN pipeline: the encoder turns an image into discrete codebook indices, while reconstruction looks up the corresponding feature vectors and decodes them into pixels. In text-to-image generation, a model can predict those indices instead of predicting every pixel directly. section 11.5.2 section 12.1.2
Trace the reconstruction path from integers to an image:
- Look up each integer in the learned codebook. The retrieved vectors form a quantized latent representation, often written z_q. section 11.5.2
- Arrange those vectors in the latent grid, apply the post-quantization conversion, and pass the result to the decoder. The decoder’s direct input is this prepared latent representation—not a bare list of integers. section 11.5.2
- Use a pretrained detokenizer when the interface accepts token indices directly; it handles the route from generated tokens to a viewable image. section 12.4 section 12.4.2
z_q2 = model.quantize.embedding(int_sequence) z_q2 = z_q2.permute(1,0).view(z_q.shape) #1 z_post2 = model.post_quant_conv(z_q2) z_recon2 = model.decoder(z_post2) #2
This implementation makes the boundary explicit: embedding retrieves codebook vectors, reshape restores the latent layout, post_quant_conv prepares it, and model.decoder reconstructs the image. section 11.5.2
Check a reconstruction
A pretrained VQGAN encodes a fish image as 576 integers. Reconstruct it from those indices and compare it with the original. You can also compare decoding the encoder’s quantized vectors with decoding vectors retrieved from the integer sequence; both paths reach the VQGAN decoder through a quantized latent representation. section 11.1 section 11.5.2
In min-DALL-E, a BART encoder processes the prompt and a BART decoder autoregressively predicts 256 image tokens for a 16 × 16 grid. The pretrained VQGAN detokenizer converts those tokens into a high-resolution color image. This division of labor keeps token prediction in the transformer and image reconstruction in VQGAN. section 12.1.2 section 12.5
Decode generated tokens
Load VQGanDetokenizer with pretrained weights, move it to the chosen device, and set evaluation mode. For the prompt “panda with top hat reading a book,” skip the beginning-of-sequence token and pass the remaining 256 tokens to detokenizer.forward(True, image_tokens[:, 1:]). Convert the resulting tensor to an image to display or save it. To inspect generation, decode saved token states at 32-token intervals; an exercise repeats the process for “an underwater fantasy city.” section 12.4.1 section 12.4.2
from PIL import Image
torch.cuda.empty_cache()
image = detokenizer.forward(True,
image_tokens[:, 1:]) #1
image = image.to(torch.uint8).to('cpu').numpy() #2
image = Image.fromarray(image) #3
image.save("files/minDALLE.png")
image
The inference call shows the practical detokenizer interface, including why the beginning-of-sequence token is excluded before image conversion. section 12.4.2
