Prompt flow

As explained in AI Agents in Action

A visual editor is a graphical interface for inspecting and modifying a flow without directly editing its underlying configuration text.

Prompt flow is an open-source Microsoft tool for building, running, deploying, and evaluating prompts and agent profiles. It began within Azure Machine Learning Studio and supports scalable, parallel evaluation of many prompt variations. section 9.3 section 9.8

A flow connects prompt, LLM, embedding, Python, and output components in a directed acyclic graph. The VS Code visual editor displays these components so you can inspect and modify a flow while its configuration remains YAML-based. section 9.3.1 section 10.1.1

A practical prompt flow workflow includes:

• Install the prompt flow packages and VS Code extension inside a Python virtual environment. section 9.3.1
• Create an LLM connection through the extension, entering the key through the terminal rather than storing it in YAML. section 9.3.1
• Open flow.dag.yaml in the visual editor, configure inputs and connections, and run the flow with Play or Test. section 9.3.1 section 10.1.1
• Define prompts with Jinja2 templates, then change roles, inputs, or other profile elements without rebuilding the entire flow. section 9.3.2 section 9.4
• Compare prompt, model, temperature, and token-limit variations through single or batch runs, then score results with an evaluation flow. section 9.4 section 9.8

Time-travel movie recommender

Open a recommender flow, provide inputs such as Subject, Genre, Format, and Custom, and run its Jinja2 profile. To test improvements, create prompt variants, execute them together, and evaluate which recommendations best satisfy a rubric. Build the result as a local web application, enter user_input in the browser, or deploy it as a Docker container. section 9.3.2 section 9.3.3 section 9.4

Figure 9.9 Opening the prompt flow visual editor

Use the visual-editor figure to see how a flow is opened and arranged before configuring its LLM blocks and inputs. section 9.3.1

Table 9.1 LLM variation options in prompt flow

Option	Evaluation option examples	Notes
Jinja2 prompt template	Compare system prompt variations, user prompt variations, or mixed prompt variations.	Some endless combinations and techniques can be applied here. Prompt engineering is evolving all the time.
LLM	Compare GPT-9.5 to GPT-4. Compare GPT-4 to GPT-4 Turbo. Compare open source models to commercial models.	This is a useful way to evaluate and ground model performance against a prompt. It can also help you tune your profile to work with open source and/or cheaper models.
Temperature	Compare a 0 temperature (no randomness) to a 1 (maximum randomness).	Changes to the temperature can significantly change the responses of some prompts, which may improve or degrade performance.
Max tokens	Compare limited tokens to larger token sizes.	This can allow you to reduce and maximize token usage.
Advanced parameters	Compare differences to options such as top_p, presence_penalty, frequency_penalty, and logit_bias.	We’ll cover the use of these advanced parameters in later chapters.
Function calls	Compare alternative function calls.	Function calls will be addressed later in this chapter.

Use the variation table to compare prompt templates, LLMs, temperature, and maximum-token settings as evaluation dimensions. section 9.4

How AI Agents in Action covers it

How deep it goes

Explains setup, YAML and Jinja2 flows, deployment, batch evaluation, rubrics, and DAG execution, but defers broader implementation details.

Examples

Uses a time-travel movie recommender, question-answering flow, prompt variants, local web app, and self-consistency batch run.

What it assumes

Assumes VS Code, its prompt flow extension, a Python virtual environment, installed packages, and access to GPT-4 or above.

What you can do after

Build prompt flows, run and compare variations, evaluate outputs with rubrics, and deploy a flow as a local application or Docker service.

How it connects

Connects prompt engineering with agent profiles, Jinja2 templates, large language models, embeddings, batch processing, DAGs, and evaluation rubrics.

liveBookmanning.com

Read AI Agents in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Generative AI in ActionPrompt Flow is a Microsoft open-source toolset for designing, evaluating, testing, and deploying workflows that combine prompts, language models, and code.

Compare how 2 books explain prompt flow →

Related concepts

• Jinjauses · Jinja is a Python templating engine that generates text by rendering templates containing static content, expressions, variables, and control structures with supplied data.
• directed acyclic graphuses · A directed acyclic graph is a graph with directed edges and no directed cycles.
• evaluationuses · Evaluation is the systematic assessment of a system, result, or approach against relevant criteria, evidence, or objectives.
• prompt versionuses · A prompt version is an identifiable revision or variant of a prompt that differs in wording, instructions, or model-specific conventions.
• prompt engineeringuses · Prompt engineering is the deliberate design and refinement of instructions, context, examples, or other inputs to guide an AI model’s behavior or outputs.

This explainer was generated by Manning's knowledge graph from the text of AI Agents in Action. The section links point to the parts of the book it draws on.