A pipeline turns a task into connected operations. Each step receives input, does a specific job, and passes its output onward. In LangChain, a sequence of these operations is a chain. This is useful when one LLM call cannot reasonably do the whole job—for example, when a research report requires searches, page extraction, summaries, and a final synthesis. section 3.0 section 4.8
Build a chain by connecting components that implement the Runnable interface. LCEL’s pipe operator (`|`) passes each component’s output to the next. Prompt templates prepare model input; output parsers turn model responses into usable text; `RunnableLambda` brings a Python function into the chain. Check the shape of each intermediate output: the next step must receive the fields it expects. section 3.1.2 section 3.1.3 section 4.7.1
Choose the arrangement that fits the task:
- Sequential: split a large document, summarize its chunks, then combine the summaries. A refinement variant updates a running summary one document at a time. section 3.1.3 section 3.2.5
- Parallel: use `RunnableParallel` for operations on the same input, or `.map()` to run a sub-chain for each item in a list. Gather the results before writing the final report. section 4.8 section 4.7.4
- Two-stage: ingest documents by splitting, embedding, and storing chunks; later retrieve relevant chunks to supply context for an LLM answer. section 6.1.3 section 6.4
A research pipeline
A travel question first selects suitable instructions, then generates web searches. Each result is scraped and summarized; the partial summaries become a research report. Test the sub-chains before combining them, and preserve the original question where later steps need it. The final chain can be run with `.invoke()`; its sub-chains can be changed without rebuilding every step. section 5.5.1 section 4.7.4 section 4.8
Listing 4.13 Web Research chain
web_research_chain = (
assistant_instructions_chain
| web_searches_chain
| search_and_summarization_chain.map() # parallelize for each web search
| RunnableLambda(lambda x:
{
'research_summary': '\n\n'.join([i['summary'] for i in x]),
'user_question': x[0]['user_question'] if len(x) > 0 else ''
})
| RESEARCH_REPORT_PROMPT_TEMPLATE | get_llm() | StrOutputParser()
)
Follow the LCEL expression from instruction selection through searches, parallel summaries, result gathering, and report generation. section 4.7.4
A straight chain is not always enough. Its linear flow makes it awkward to retry a search because too few summaries are relevant; implicit state also makes failures harder to track. For limited choices between next steps, a workflow can route work to a processor or have a controller assign tasks to workers. Whatever the arrangement, inspect ingestion and answering separately: poor chunking or retrieval can weaken the final answer even when later steps work. section 5.5.1 section 5.1.1 section 6.4
