Top-k sampling restricts the next-token choice to the k most probable candidates instead of the full vocabulary. Smaller k values favor highly likely tokens, producing more coherent but less creative output. section 12.3.2
During autoregressive image generation, the decoder predicts one image token at a time. The selected token updates the sequence and internal attention state before the next prediction. section 12.3.2
Configure top-k sampling alongside related generation controls:
- Set temperature to adjust prediction randomness independently of the candidate restriction. section 12.3.2
- Choose a small k for confidence and coherence, or a larger k to preserve more creative alternatives. section 12.3.2
- Use the resulting token in the decoder loop, repeating the process for all 256 image tokens. section 12.3.2
A configured image-token loop
A min-DALL-E configuration uses temperature 0.5, top_k 128, and supercondition factor 4, then samples image tokens iteratively while caching attention state. section 12.3.2
token_indices=torch.arange(256,device=device)
temperature=0.5 #1
top_k=128 #2
supercondition_factor=4 #3
settings = torch.tensor(
[temperature, top_k, supercondition_factor],
dtype=torch.float32,device=device)
Inspect the compact tensor of generation settings, including temperature, top-k, and supercondition values. section 12.3.2
Listing 12.7 Predicting image tokens iteratively
from copy import deepcopy
imgtokens=[] #1
for i in range(256): #2
torch.cuda.empty_cache()
with torch.amp.autocast("cuda",dtype=getattr(torch, dtype)):
image_tokens[:, i + 1], \ #3
attention_state = decoder.sample_tokens(
settings=settings,
attention_mask=attention_mask,
encoder_state=encoder_state,
attention_state=attention_state,
prev_tokens=image_tokens[:, [i]],
token_index=token_indices[[i]])
imgtokens.append(deepcopy(image_tokens))
Trace the iterative decoder call that samples each image token and updates the attention state. section 12.3.2
