Exact-match accuracy reports the share of outputs accepted as correct under specified comparison rules. In a tool-call evaluation, those rules normalize case and whitespace, allow small coordinate differences, and accept reasonable supersets of expected text; the comparison functions themselves are not detailed. section 13.3
Fine-tuning comparison
Using exact match as the main metric, the original model scored 88.9% and the fine-tuned T specialist scored 96.3%. The remaining T failures involved incorrect arguments. section 13.3
Table 13.3 Pruning and distillation results for M→S
|
Model
(transformer
blocks / expansion)
|
Valid JSON
|
Exact match
|
Tolerant
match
|
No call
|
Wrong
function
|
Wrong args
|
|
M
(25, 2304)
|
100.0%
|
92.6%
|
92.6%
|
0
|
0
|
4
|
|
Depth
Pruning
(22, 2304)
|
100.0%
|
13.0%
|
20.4%
|
0
|
38
|
5
|
|
+ Width
pruning
(22, 1792)
|
100.0%
|
22.2%
|
24.1%
|
0
|
33
|
8
|
|
S, post-KD
(22, 1792)
|
100.0%
|
88.9%
|
88.9%
|
0
|
0
|
6
|
Compare exact and tolerant match alongside valid JSON and failure types across pruning and distillation stages. section 13.5







