Hugging Face Transformers supplies pretrained encoder–decoder models and tokenizers for building retrieval-augmented question answering. A FiD implementation can load Flan-T5 by model name, but it must assemble the multi-passage data flow itself. section 5.4
Pair each retrieved passage with the question, tokenize the pairs, and encode them in parallel. Reshape the resulting hidden states so one decoder can attend across all passages; reshape the attention mask to match before calling generate. Padding and truncation occur during tokenization, so the encoded inputs—not just the retrieved passages—determine what reaches the decoder. section 5.4
Listing 5.3 Decoding for FiD.
def decode_answer(encoder_hidden_states, attention_mask, tokenizer, model):
num_passages, seq_len, hidden_dim = encoder_hidden_states.shape
fused_hidden_states = encoder_hidden_states.reshape(
1, num_passages * seq_len, hidden_dim #A
)
fused_attention_mask = attention_mask.reshape(1, num_passages * seq_len)
encoder_outputs = BaseModelOutput(last_hidden_state=fused_hidden_states)
generated_ids = model.generate( #B
encoder_outputs=encoder_outputs,
attention_mask=fused_attention_mask,
max_length=64 #C
)
return tokenizer.decode(generated_ids[0], skip_special_tokens=True)The decoding listing shows the fused hidden states and attention mask passed to generate. section 5.4






