Running every test on every change can make CI too slow for rapid iteration. Targeted test selection narrows the run to tests affected by a code or dependency change. The shortcut is useful only if the dependency information captures the change’s potential reach; testing the modified component alone can miss regressions in its consumers. section 10.3.4 section 8.7.1
A queryable dependency graph identifies changed targets and their consumers, then selects the relevant tests. Smaller build targets make the selection more precise: fewer unrelated tests are triggered, and the blast radius is easier to inspect. The tradeoff is more explicit build declarations and maintenance. section 10.2.4 section 10.3.4
Checkout change
A change to a checkout service runs its tests and those of its direct consumers. A leaf utility with no consumers might trigger 12 tests rather than a suite of three million. section 10.3.4
Selection also supports dependency upgrades. An agent can query which services consume a library, upgrade it, and run the consumers’ tests. Passing tests alone do not settle every merge decision: the described workflow also considers change complexity and routes some changes to human review. section 10.6.1
A related impact-analysis technique generates targeted tests rather than selecting existing ones. It identifies modified functions, finds their callers, and infers the behaviors those callers depend on, such as return shape or side effects. This aims between the cost of a full suite and the narrowness of testing only the modified function. section 8.5.5
Listing 8.21 Pseudo-code for impact analysis
import ast
def analyze_change_impact(diff_path: str, codebase_path: str):
"""Identify behaviors potentially affected by an AI-generated change"""
modified = extract_modified_function_names(diff_path)
callers = []
for py_file in Path(codebase_path).rglob("*.py"):
tree = ast.parse(py_file.read_text())
for node in ast.walk(tree):
if isinstance(node, ast.Call) and getattr(node.func, 'id', None) in modified:
callers.append((py_file, node.lineno, node.func.id))
affected_behaviors = []
for file_path, line, func_name in callers:
behavior = infer_dependency(file_path, line, func_name)
# e.g., "uses return value as string", "checks for None", "iterates over result"
affected_behaviors.append(behavior)
return generate_verification_tests(affected_behaviors)The pseudocode sketches the path from modified functions to callers, inferred dependencies, and verification tests. section 8.5.5
For AI-generated changes, tag tests by component and dependency so that a parser change, for example, can trigger content service, rendering, and API tests as well as parser tests. Some teams run the full suite instead, accepting longer CI times to catch unexpected regressions that selection might miss. section 8.7.1
