liveBook · Manning

Workflow service

As explained in Designing AI Systems

A workflow service is a platform service that coordinates sequential and parallel operations, state management, retries, and failure recovery in complex processes.

A workflow service coordinates multi-step AI processes and runs them as deployable applications. It handles sequential and parallel operations, state, retries, and failure recovery so developers can focus on business logic rather than rebuilding orchestration for each workflow. section 1.4 section 2.6

A workflow is defined as a function with an @workflow decorator that captures its name, API path, scaling parameters, and response mode. Each workflow is independently deployable, allowing it to scale and version separately from other workflows. section 9.0 section 2.6

Figure 9.1 Control plane vs. data plane. On the left, the Workflow Service (control plane) connects to Kubernetes to manage container Deployments and to the API Gateway to register routes. On the right, the data plane: multiple workflow containers, each running the SDK's runtime server, receiving HTTP requests from the API Gateway and making gRPC calls to platform services through the gateway. For synchronous and streaming workflows, the control plane and data plane do not communicate during request handling. For asynchronous workflows, the data plane calls the control plane to manage job state.

The control plane manages deployments and routes; runtime servers in workflow containers handle requests. Async workflows also use the control plane to manage job state. section 9.0

The service contract separates operations by consumer:

  • Registry operations register, update, list, and delete workflow specifications. A deployment CLI uses these operations to manage definitions. section 9.5
  • Deployment operations start deployments, check their status, and roll back versions. The service packages code into containers and registers routes with the API gateway. section 9.5 section 9.0
  • Job operations create and track asynchronous jobs, report progress, record completion or failure, and support polling and cancellation. section 9.5

For runtime management, the service generates a Kubernetes Deployment and HorizontalPodAutoscaler from workflow configuration. Health probes and replica counts keep containers available; CPU-based scaling adds or removes replicas. During a rolling update, the service monitors error rates and can revert to the previous image if the new version performs significantly worse. section 9.7

Recovering a long-running job

A research workflow saves a checkpoint after an expensive operation and reports progress through the workflow SDK. The SDK uses a context variable to identify the job without requiring the function to pass its ID. After a crash, the service can redispatch the job with the same input; the function loads the checkpoint and skips stages whose results were saved. Clients see progress on their next poll. section 9.2.2

Listing 10.20 Workflow composition for complex tasks
@workflow(
    name="claw-research",
    api_path="/claw/research",
    response_mode="async"    #A
)
def research_task(
    topic: str,
    user_id: str,
    session_id: str
) -> dict:
    # Search multiple sources in parallel
    results = platform.workflows.call_parallel([    #B
        ("/claw/knowledge-search",    #B
         {"query": topic, "index": "company-knowledge"}),    #B
        ("/claw/knowledge-search",    #B
         {"query": topic, "index": "engineering-docs"}),    #B
        ("/claw/web-search",    #B
         {"query": topic})    #B
    ])    #B
 
    # Synthesize findings
    combined_context = merge_search_results(results)    #C
    synthesis = platform.models.chat(
        model="gpt-4o",
        system_prompt_name="claw-researcher",
        messages=[{
            "role": "user",
            "content": (
                f"Synthesize a research summary on: "
                f"{topic}\n\n{combined_context}"
            )
        }]
    )
 
    return {
        "summary": synthesis.content,
        "sources": extract_sources(results)
    }

This async research workflow fans out searches to independently deployed child workflows, then combines their results for synthesis. The client receives a job ID and polls rather than holding an HTTP connection open. section 10.12

How Designing AI Systems covers it

How deep it goes
Implements runtime, job-state, contract, deployment, and Kubernetes management patterns, including checkpoint recovery and rollback.
Examples
Uses a checkpointed research job and parallel child workflows to illustrate recovery and composition.
What you can do after
Build a workflow service and apply its SDK to deploy, compose, and recover AI workflows.
How it connects
Connects orchestration to API gateways, Kubernetes, asynchronous jobs, checkpointing, and observability.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.