Exact-match accuracy measures how often generated responses match their reference answers exactly after whitespace normalization. Each response receives 1 for a match and 0 otherwise; the overall score is the average across examples. section 5.1.3 section 5.1.4
Exact match is strict: a helpful rephrasing can score 0. Use it alongside Token-F1, which rewards word overlap, rather than treating a low exact-match score alone as proof that an answer is unhelpful. Safety refusal rate measures a separate concern: whether the model refuses harmful prompts. section 5.1.4
For a base-model versus LoRA-adapter comparison, evaluate both on held-out examples that were not used earlier. Generate an answer for each example, compare it with the reference, and aggregate the binary scores. The evaluation can also report Token-F1 and safety refusal rate. section 5.1.3
## base - Overall exact match: 0.0% - Overall token-F1: 0.149 - Safety refusal rate: 100.0% ## adapter - Overall exact match: 0.0% - Overall token-F1: 0.154 - Safety refusal rate: 60.0% ## adapter (Improvement vs Base) - Token-F1 delta: +0.005 - Safety refusal rate delta: -40.0% - Per-category token-F1 (base to adapter): - networking: 0.169 to 0.170 (+0.001) - security: 0.199 to 0.185 (-0.014) - windows: 0.133 to 0.141 (+0.008) - linux: 0.129 to 0.144 (+0.014) - hardware: 0.152 to 0.188 (+0.037) - software: 0.122 to 0.111 (-0.011)
In this IT-support evaluation, both models scored 0% exact match on long, open-ended answers. The adapter’s safety refusal rate also fell from 100% to 60%, a separate warning that the exact-match score does not capture. section 5.1.4







