6 Testing and prompt engineering

In chapter 3, we first started to see the importance of testing the code produced by Copilot. Testing is an essential skill for anyone writing software because it gives you confidence that the code is functioning properly. In this chapter, we’ll learn how to test our code thoroughly and how to help Copilot fix code that doesn’t work by modifying our prompts.

6.1 Why it is crucial to test code

Back in chapter 3, we mentioned that you should test code to make sure it is correct. Unfortunately, we’ve seen student after student be reluctant to test. Why? We think a couple of things are at play. The first is that there’s this well-documented problem nicknamed the Superbug, which is that humans, when first learning to code, think the computer can understand the intent of the code and respond accordingly [1]. Since they wrote the code and the code made sense to them, of course it should work. The second problem is compounded on the first: if you think your code is right, testing can only bring you bad news. If you don’t test, you can’t find out if the code is wrong. It’s like the old saying about putting your head in the sand.

6.2 Closed-box and open-box testing

6.2.1 Closed-box testing

6.2.2 How do we know which test cases to use?

6.2.3 Open-box testing

6.3 How to test your code

6.3.1 Testing using the Python prompt

6.3.2 Testing in your Python file (we won’t be doing it this way)

6.3.3 doctest

6.4 Revisiting the cycle of designing functions with Copilot

6.5 Full testing example

6.5.1 Finding the most students we can add to a row

6.5.2 Improving the prompt to find a better solution

6.5.3 Testing the new solution

6.6 Another full testing example—Testing with files

6.6.1 What tests should we run?

6.6.2 Creating the function

6.6.3 Testing the function

6.6.4 Common challenges with doctest

Summary