3 Mastering agent profiles with Prompt Flow
This chapter covers
- Understanding systematic prompt engineering
- Setting up your first prompt flow
- Crafting the effective profile/persona prompt
- Evaluating profiles: rubrics and grounding
- Grounding evaluation of an LLM profile
- Comparing profiles: Getting the perfect profile
In the last chapter, we covered using the basics of LLMs, connecting to LLMs, following helpful strategies of prompt engineering, and choosing the right LLM. This chapter examines an agent's first and primary components, the profile/persona, and the evaluation of that profile. An agent's profile describes what it does and how, but building a compelling profile requires iteration and assessment, as we will see in the next section.
3.1 Understanding systemic prompt engineering
For this chapter and section, we want to explore the prompt engineering strategy Test Changes Systematically. If you recall, we covered the grand strategies of the OpenAI Prompt Engineering Framework in Chapter 2. These strategies can help us build better prompts and, in turn, better agent profiles and personas.
Test Changes Systematically is such a core facet of prompt engineering that Microsoft developed a tool around this strategy called Prompt Flow. (Prompt Flow is a tool we will cover later in this chapter.) Before getting to Prompt Flow, we need to understand systemic prompt engineering and why we need it.