6 Testing and prompt engineering

 

This chapter covers

  • Understanding the importance of testing Copilot code
  • Using closed-box versus open-box testing
  • Addressing errors by Copilot by modifying prompts
  • Working through examples of testing Copilot-generated code

In chapter 3, we first started to see the importance of testing the code produced by Copilot. Testing is an essential skill for anyone writing software because it gives you confidence that the code is functioning properly. In this chapter, we’ll learn how to test our code thoroughly and how to help Copilot fix code that doesn’t work by modifying our prompts.

Testing is an essential skill that you’ll need to learn how to do well on your own, so that you are able to check that the code works correctly. Copilot can generate tests, and has been improving in the quality of tests it produces, but we encourage you to hold off on using Copilot to generate tests just yet, because you need to learn to do this well enough on your own to be able to verify that the tests Copilot produces are reasonable. This will be true of the next few chapters as well – testing, problem decomposition, and debugging are all skills that are essential to learn how to do on our own, before asking for Copilot’s help, as we need to know how to do it on our own to know if Copilot is doing something reasonable.

6.1 Why it is crucial to test code

6.2 Closed-box and open-box testing

6.2.1 Closed-box testing

6.2.2 How do we know which test cases to use?

6.2.3 Open-box testing

6.3 How to test your code

6.3.1 Testing using the Python prompt

6.3.2 Testing in your Python file (we won’t be doing it this way)

6.3.3 doctest

6.4 Revisiting the cycle of designing functions with Copilot

6.5 Full testing example

6.5.1 Finding the most students we can add to a row

6.5.2 Improving the prompt to find a better solution

6.5.3 Testing the new solution

6.6 Another full testing example—Testing with files

6.6.1 What tests should we run?

6.6.2 Creating the function

6.6.3 Testing the function

6.6.4 Common challenges with doctest

6.7 Exercises

6.8 Summary