Copilot can complete a function body from nearby comments and code, which makes it feel like familiar editor completion with a far larger appetite for context. That speed is useful—but a completion is a draft, not evidence that the draft satisfies the real requirement. section 1.3 section 2.6
A good prompt can still yield a nearly right answer with a quiet mismatch in its definition of the problem. Ask for words beginning with x and ending with y, and a generated x\w*y pattern heads in the right direction while treating digits and underscores as possible word characters. A more explicit lowercase-word request may improve the pattern on one attempt, then be ignored on another. Context and phrasing affect the output, but not in a reliably human-like way. section 2.3 section 2.6
A regex that is clever, wrong, and therefore wrong
For detecting a pair in a poker hand, Copilot generated detailed comments and a compact backreference-based regex. It can find repeated adjacent non-suit characters in strings such as AK992, but formatted cards place ranks and suits between repetitions; the pattern cannot recognize the intended hand structure. Clever syntax does not rescue a mismatched model of the input. section 4.24
Review generated code at several levels:
- Check semantics, not merely broad success. An IPv4 pattern that accepts any dot-separated digit runs matches valid addresses and many invalid ones too. section 5.9
- Check unstated conventions. A card sorter can correctly order ranks yet omit suit ordering until that ordering is explicitly specified. section 4.12
- Check operational behavior. A mathematically correct alternating-block regex can catastrophically backtrack, while a proposed non-greedy revision can remain just as slow on a failing input. section 3.3
- Check whether the generated abstraction is warranted. A generic helper that compiles and searches a supplied regex can be less direct than calling search or match where needed. section 3.6
Copilot may contribute useful boilerplate, inline documentation, test scaffolding, or a genuinely apt completion when the relevant technique is already present in local context. It may also repeat comments indefinitely, supply irrelevant tests, or fail to emit code after producing plausible prose. Treat verbosity and apparent intention as separate from correctness. section 3.6 section 4.24 section 3.3
s1 = "EEEEDEDDEEDEDDDDDDDDEEDEDEDEE"
s2 = "EEEEDEDDEEDEDDDDDDDDEEDEDEDEEX"
from time import perf_counter as now
start = now()
print("s1 naive", check_alternating(s1) is not None,
"%.2f seconds" % (now() - start))
start = now()
print("s1 nocat", check_alternating_nocat(s1) is not None,
"%.2f seconds" % (now() - start))
start = now()
print("s2 naive", check_alternating(s2) is not None,
"%.2f seconds" % (now() - start))
start = now()
print("s2 nocat", check_alternating_nocat(s2) is not None,
"%.2f seconds" % (now() - start))
Timing both matching and failing strings exposes why a superficially improved regex still suffers catastrophic backtracking. section 3.3
The useful stance is neither awe nor dismissal: use Copilot to make candidate code quickly, then supply the domain assumptions, boundary cases, performance tests, and code-review judgment that decide whether it belongs in a program. section 2.6 section 5.9
