Code coverage reports the proportion of source-code lines, branches, or paths executed during testing. High coverage can still conceal untested behavior: execution shows that code ran, not that assertions verified the right result. section 8.3.4
AI-generated coverage is especially risky when the same process produces both implementation and tests. Correlated tests may reproduce the implementation’s assumptions, exercise familiar cases, and miss adversarial inputs or edge cases. section 8.3.4 section 8.3.7
Use coverage as evidence, not as a release decision:
- Review tests for requirements and expected behavior, rather than accepting high percentages as proof of correctness. section 8.3.7
- Add cases for malformed input, boundary conditions, and mixed or multiline data that can traverse existing lines while exposing defects. section 8.3.4
- Review AI-generated tests for validity and false positives before using them to support migration work. section 2.4.1
- Combine coverage with property-based testing, mutation testing, contract testing, characterization testing, and runtime observability when the system warrants stronger evidence. section 8.9
A misleading 95% result
A markup parser can report roughly 95% coverage after tests for simple H1, H2, H3, and bold cases. That result does not establish correct handling of #### H4, a header without a space, or multiline input. The tests execute the parser, but they do not specify enough behavior to expose its blind spots. section 8.3.4
Listing 8.6 First set of test case
def parse_markup(text: str) -> str:
result = text
# Headers
lines = result.split('\n')
for i, line in enumerate(lines):
if line.startswith('###'):
lines[i] = f'<h3>{line[3:].strip()}</h3>'
elif line.startswith('##'):
lines[i] = f'<h2>{line[2:].strip()}</h2>'
elif line.startswith('#'):
lines[i] = f'<h1>{line[1:].strip()}</h1>'
result = '\n'.join(lines)
result = re.sub(r'\*\*(.+?)\*\*', r'<strong>\1</strong>', result)
result = re.sub(r'\*(.+?)\*', r'<em>\1</em>', result)
result = re.sub(r'\[(.+?)\]\((.+?)\)', r'<a href="\2">\1</a>', result)
return result
def test_headers():
assert parse_markup("# H1") == "<h1>H1</h1>"
assert parse_markup("## H2") == "<h2>H2</h2>"
assert parse_markup("### H3") == "<h3>H3</h3>"
def test_formatting():
assert parse_markup("**bold**") == "<strong>bold</strong>"
assert parse_markup("*italic*") == "<em>italic</em>"
def test_links():
assert parse_markup("[a](b)") == '<a href="b">a</a>'Inspect the parser and its first tests to see how straightforward examples can exercise code without testing adversarial behavior. section 8.3.4





















