liveBook · Manning

Transformers

As explained in Retrieval Augmented Generation, The Foundational Ideas

Hugging Face Transformers is a library providing pretrained transformer models and tools for natural-language and other machine-learning tasks.

Hugging Face Transformers supplies pretrained encoder–decoder models and tokenizers for building retrieval-augmented question answering. A FiD implementation can load Flan-T5 by model name, but it must assemble the multi-passage data flow itself. section 5.4

Pair each retrieved passage with the question, tokenize the pairs, and encode them in parallel. Reshape the resulting hidden states so one decoder can attend across all passages; reshape the attention mask to match before calling generate. Padding and truncation occur during tokenization, so the encoded inputs—not just the retrieved passages—determine what reaches the decoder. section 5.4

Listing 5.3 Decoding for FiD.
def decode_answer(encoder_hidden_states, attention_mask, tokenizer, model):
 
    num_passages, seq_len, hidden_dim = encoder_hidden_states.shape
 
    fused_hidden_states = encoder_hidden_states.reshape(
        1, num_passages * seq_len, hidden_dim  #A
    )
    fused_attention_mask = attention_mask.reshape(1, num_passages * seq_len)
 
    encoder_outputs = BaseModelOutput(last_hidden_state=fused_hidden_states)
 
    generated_ids = model.generate(  #B
        encoder_outputs=encoder_outputs,
        attention_mask=fused_attention_mask,
        max_length=64  #C
    )
 
    return tokenizer.decode(generated_ids[0], skip_special_tokens=True)

The decoding listing shows the fused hidden states and attention mask passed to generate. section 5.4

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Works through model loading, passage encoding, fused generation, and a fine-tuning step; FiD data flow remains manual.
Examples
Uses Flan-T5 code to answer a multi-passage question about a multi-GPU training decision.
What it assumes
Assumes familiarity with Python tensors while explaining the encoder–decoder data flow needed for FiD.
What you can do after
Apply it via Hugging Face Transformers to load Flan-T5 and assemble a basic FiD answer-generation path.
How it connects
Connects pretrained models and tokenizers to T5, FiD, encoder–decoder architecture, attention masks, and retrieval-augmented generation.

Also explained in

Compare how 7 books explain transformers →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.