liveBook · Manning

AI framework

As explained in Context Engineering

A software framework that provides reusable components and abstractions for building AI-powered applications.

An AI framework provides reusable software components for building applications on top of existing models. In context engineering, it helps assemble, transform, and route context into model calls and connect model outputs to application logic. This role differs from machine learning frameworks used to train and evaluate model weights. section 10.1 section 10.3

The landscape can be divided by the scope of context governed and the operating model:

  • AI application frameworks are developer libraries for composing prompts, retrieval, tools, memory, structured outputs, and model integrations. They primarily manage context for individual model calls. section 10.1 section 10.3
  • Agent orchestration frameworks coordinate multi-step execution, including state, tool use, planning, human review, and agent collaboration. Some provide machinery for constructing an agent loop; others provide an assembled loop. section 10.1
  • AI application platforms integrate design, deployment, monitoring, and operational workflows, often with visual builders and managed services. section 10.1
Table 10.1 A taxonomy of AI frameworks and platforms for context engineering
Category Primary role Examples

AI application frameworks

Developer libraries for building LLM applications with different context sources and model integrations.

LangChain, LlamaIndex, Haystack, Spring AI, LangChain4j, Pydantic AI, DSPy, AI SDK

Agent orchestration frameworks

Build agents and coordinate multi-step execution, planning, tool use, agent state, human-in-the-loop control, and multi-agent workflows. Some supply the machinery for building the agent loop, and others deliver it already assembled.

LangGraph, CrewAI, Agent Development Kit, Microsoft Agent Framework, Agno, Embabel, Parlant, DeepAgents, Claude Agent SDK

AI application platforms

Integrated environments for visual design, deployment, monitoring, team workflows, and operational control.

Dify, LangFlow, Flowise, Zapier, n8n, OpenClaw, Temporal, Amazon Bedrock AgentCore, Gemini Enterprise Agent Platform

Compare the three categories and their representative tools. section 10.1

Frameworks approach application construction differently. LangChain and LlamaIndex help combine prompts with external data, tools, and workflows. DSPy instead lets developers specify tasks declaratively and optimize prompts using examples, feedback, and evaluation signals. section 6.5

Choose for the required workflow

A single retrieval-augmented generation endpoint does not need multi-agent orchestration. A long-running coding agent, by contrast, needs explicit state and approvals; an orchestration framework may be a better fit. section 10.2

Select and operate a framework around the architecture, not its popularity:

  • Define context sources, retrieval needs, tool boundaries, memory, state, evaluation, and operational constraints before shortlisting tools. section 10.2
  • Keep model input, returned tool outputs, and persisted state inspectable. Layered abstractions can otherwise make incorrect answers difficult to trace to their source. section 10.2 section 10.3
  • Use orchestration only when needed: dynamic planners and multi-agent systems add failure modes. Apply version control, review, tests, access control, and deployment discipline to visual workflows as well as code. section 10.2
  • Evaluate traces, logs, evaluations, cost monitoring, policy checks, rollback, and secret handling. Keep prompts, retrieval configuration, tool definitions, and evaluation data outside framework-specific formats to preserve an exit path. section 10.2
Table 10.2 Trade-offs among AI application frameworks, agent orchestration frameworks, and AI application platforms
Dimension AI application frameworks Agent orchestration frameworks AI application platforms

Primary user

Developers building AI features inside applications.

Developers building stateful agents or multi-step workflows.

Teams building, deploying, and operating AI applications through integrated environments.

Main strength

Flexible composition of prompts, retrieval, tools, memory, structured outputs, and model integrations.

Explicit coordination of state, tools, agents, human review, and execution flow.

Fast prototyping, deployment, monitoring, collaboration, and operational packaging.

Main risk

Hidden complexity in context assembly and provider abstractions.

Context drift across steps, agents, and tool outputs.

Reduced transparency or portability when workflows depend on platform-specific features.

Best fit

Custom applications, RAG systems, typed integrations, and product features.

Research agents, workflow automation, coding agents, support agents, and multi-agent systems.

Internal tools, business workflows, prototypes, managed enterprise agents, and team workflows.

Compare the categories by strengths, risks, and intended users. section 10.2

How Context Engineering covers it

How deep it goes
Defines three framework categories and provides selection criteria, operational checks, and portability guidance.
Examples
Contrasts a single RAG endpoint with a long-running coding agent and compares LangChain, LlamaIndex, and DSPy.
What you can do after
Evaluate framework categories against context scope, workflow requirements, inspectability, and operational constraints.
How it connects
Connects frameworks to retrieval, tool use, memory, agent orchestration, structured outputs, and observability.

Also explained in

Compare how 2 books explain AI framework →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.