Fidelity measures whether a sentiment label matches its sentence. In a financial training-data quality gate, a judge scores fidelity from 1 to 5; a score below 4 fails the gate. section 3.6.4
Use fidelity as one part of a three-dimension check:
- Require fidelity, format, and domain safety scores of at least 4; a strong fidelity score alone does not pass an example. section 3.6.4
- Treat the gate as a filter for candidate examples, not as a measure of the teacher model’s overall quality. section 3.6.4
Listing 3.5 Specifying the judging system prompt
JUDGE_SYSTEM_PROMPT = """You are a quality reviewer for a financial
language model training dataset.
Score this financial sentiment example on three dimensions (1 to 5 each):
FIDELITY (1 to 5): Does the sentiment label match the sentence content?
5 = label is clearly correct for this sentence
1 = label contradicts the sentence
FORMAT (1 to 5): Is this realistic financial text?
5 = reads like real financial news or analyst commentary
1 = sounds artificial, generic, or not financial
DOMAIN SAFETY (1 to 5): Are financial facts plausible?
5 = all percentages, rates, amounts are realistic
1 = contains impossible or implausible financial figures
A PASS requires ALL THREE >= 4.
Return ONLY valid JSON, no preamble:
{"fidelity": X, "format": X, "domain_safety": X, "pass": true_or_false}"""The judging prompt defines the 1–5 fidelity scale and requires all three quality scores to reach 4 for a pass. section 3.6.4










