liveBook · Manning

Job record

As explained in Designing AI Systems

A job record is the Workflow Service's stored representation of one submitted workflow execution, including its status, payloads, progress, result, error, and assigned container.

A job record is the Workflow Service’s stored representation of one asynchronous workflow execution. It lets a client poll for progress without holding an HTTP connection open, while giving the runtime a place to write results and recovery logic the state needed to re-dispatch orphaned work. section 9.1.5 section 9.2.1

The async handler creates the record with the original request body, receives a unique job ID, and schedules the workflow in the background. It returns HTTP 202 with the ID and a status URL. The initial status is processing; completion or failure changes it to completed or failed. A client can also request cancellation, setting the status to cancelled. section 9.1.5 section 9.2.1

A relational jobs table holds the execution state:

  • job_id identifies the record; workflow_id identifies the workflow. created_at and updated_at record timestamps. section 9.2.1
  • status and progress_message expose execution state; result_json and error hold the eventual outcome. section 9.2.1
  • input_json retains the original request, assigned_endpoint identifies the assigned container, and checkpoint_json stores serialized intermediate state. section 9.2.1 section 9.2.2
Listing 9.9 Jobs table schema
CREATE TABLE jobs (
    job_id VARCHAR(64) PRIMARY KEY,    #A
    workflow_id VARCHAR(64) NOT NULL,
    status VARCHAR(20) NOT NULL    #B
        DEFAULT 'processing',
    progress_message TEXT,    #C
    input_json TEXT NOT NULL,    #D
    result_json TEXT,    #E
    error TEXT,    #F
    checkpoint_json TEXT,    #G
    assigned_endpoint VARCHAR(256),    #H
    created_at TIMESTAMP NOT NULL
        DEFAULT NOW(),
    updated_at TIMESTAMP NOT NULL
        DEFAULT NOW()
);

The schema groups polling, runtime-update, and recovery fields in one row keyed by job_id. section 9.2.1

The runtime writes progress and outcomes by job ID, while clients read the same record through GET /jobs/{job_id}, served by the Workflow Service. Recovery queries by assigned endpoint and status to find jobs whose container disappeared; the original input allows re-dispatch to another container. section 9.2.1

Workflow code can call update_job_progress() without passing a job ID: the SDK reads it from a context variable set by the handler. Similarly, save_checkpoint() writes intermediate results to the record. After re-dispatch, load_checkpoint() can retrieve that state so the workflow can skip completed stages. On termination, complete_job() stores the result; fail_job() stores the error. section 9.2.2

GET /jobs/job_8x7f9k2m
 
{"job_id": "job_8x7f9k2m", "status": "processing",
 "progress": "Retrieved 47 documents",
 "created_at": "...", "updated_at": "..."}
 
GET /jobs/job_8x7f9k2m
 
{"job_id": "job_8x7f9k2m", "status": "processing",
 "progress": "Analysis complete, formatting report",
 "created_at": "...", "updated_at": "..."}
 
GET /jobs/job_8x7f9k2m
 
{"job_id": "job_8x7f9k2m", "status": "completed",
 "result": {"research_report": "...",
            "sources": [...]},
 "created_at": "...", "updated_at": "..."}

A polling sequence shows progress messages followed by a completed result. section 9.2.1

How Designing AI Systems covers it

How deep it goes
Specifies a jobs table and traces creation, polling, progress updates, completion, and checkpoint-based recovery.
Examples
Uses a long-running research workflow and polling responses that progress from document retrieval to a completed report.
What you can do after
Evaluate how a stored job record supports progress visibility and recovery for long-running workflows.
How it connects
Connects asynchronous execution with polling, relational storage, checkpointing, and crash recovery.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.