A passing test shows that software met an expected condition for a particular case; it does not establish correctness for every case. This distinction matters when AI-generated code handles a convincing happy path but fails on neighboring inputs. section 8.1.3 section 8.2
Tests execute behavior against expectations. Unlike static checks for syntax, types, or conventions, they can check whether a given input produces the required output. Write those expectations as contracts: specify what the software must do without tying the test to how it does it. section 8.2.6 section 8.3.7
A parser that passes the easy cases
A generated markup parser may convert simple bold text, headers, and links correctly while breaking nested formatting or links with parentheses. Test both the ordinary conversions and combinations or malformed inputs that challenge the parser’s assumptions. section 8.2 section 8.8.6
Listing 8.1 Initial version of markup parser in Python
def parse_markup(text: str) -> str:
result = text
lines = result.split('\n')
for i, line in enumerate(lines):
if line.startswith('###'):
lines[i] = f'<h3>{line[3:].strip()}</h3>'
elif line.startswith('##'):
lines[i] = f'<h2>{line[2:].strip()}</h2>'
elif line.startswith('#'):
lines[i] = f'<h1>{line[1:].strip()}</h1>'
result = '\n'.join(lines)
import re
result = re.sub(r'\*\*(.+?)\*\*', r'<strong>\1</strong>', result) # Bold
result = re.sub(r'\*(.+?)\*', r'<em>\1</em>', result) # Italic
result = re.sub(r'\[(.+?)\]\((.+?)\)', r'<a href="\2">\1</a>', result) # Links
return resultInspect the parser’s regular expressions and header handling alongside the cases they can miss. section 8.2
Use testing as a disciplined verification process:
- Before migrating legacy code, establish a baseline with unit, integration, end-to-end, and performance tests where applicable. section 2.4
- Ask an AI tool to identify and generate missing tests, then run and review them. Generated tests can be invalid, poorly designed, or produce false positives. section 2.4.1
- Describe requirements independently of the implementation. High coverage can exercise code without specifying the right behavior; snapshots can preserve wrong behavior. section 8.3 section 8.3.7
- Probe failure paths as well as expected cases, including empty, malformed, or malicious inputs and unavailable dependencies. section 8.8.6
MemberRepositoryTest.java:
@ApplicationScoped
public class MemberRepositoryTest {
@Inject
@Mock
private EntityManager em;
@InjectMocks
private MemberRepository memberRepository;
@BeforeEach
void setUp() {
MockitoAnnotations.openMocks(this);
}
@Test
void testFindById() {
Member mockMember = new Member();
mockMember.setId(1L);
when(em.find(Member.class, 1L)).thenReturn(mockMember);
Member result = memberRepository.findById(1L);
assertNotNull(result);
assertEquals(1L, result.getId());
verify(em).find(Member.class, 1L);
} // remaining tests omitted
RemoteMemberRegistrationIT.java:
@Test
public void testGetAllMembers() throws Exception {
HttpRequest request = HttpRequest.newBuilder(getHTTPEndpoint())
.GET()
.build();
HttpResponse response = HttpClient.newHttpClient().send(request, HttpResponse.BodyHandlers.ofString());
Assert.assertEquals(200, response.statusCode());
// Parse the JSON response
JsonReader jsonReader = Json.createReader(new StringReader(response.body().toString()));
JsonArray members = jsonReader.readArray();
Assert.assertNotNull("Response should be a JSON array", members);
// Log the members for debugging
log.info("Found " + members.size() + " members");
for (int i = 0; i < members.size(); i++) {
JsonObject member = members.getJsonObject(i);
log.info("Member " + i + ": " + member);
}Compare generated repository and integration tests when reviewing a migration baseline. section 2.4.1
Testing is one control, not a substitute for all others. An agent with production credentials can cause harm through its actions even when the software has excellent tests; permissions, environment boundaries, and review are separate safeguards. section 1.1.4


































































