Data curation prepares representative, high-quality examples for evaluation and model adaptation. When there are not yet a few hundred suitable examples, curate data before choosing a training technique. section 1.7
Start with the data needed to make and measure the adaptation decision:
- Build a held-out evaluation set and core task examples before selecting a technique; reuse the same evaluation set to judge each technique. section 1.7
- Size the evaluation set for the decision: 50 examples can reveal a large gap, while resolving differences of a few points typically takes 200 to 500. section 1.7
- Expect human effort in curation and evaluation to outweigh compute costs in the illustrated fine-tuning project. section 6.12.4
Clinical adaptation
A hospital team curates a few hundred clinical examples, measures a baseline with a self-hosted open model, and trains a LoRA adapter. section 1.7
Table 6.14 Training cost breakdown (illustrative)
|
Phase
|
Duration
|
Compute Cost
|
Human Cost
|
Total
|
|
Data curation and quality review
|
2 weeks
|
–
|
$8,000 (clinical reviewer time)
|
$8,000
|
|
Data formatting and preprocessing
|
3 days
|
$50 (CPU instances)
|
$2,000 (ML engineer time)
|
$2,050
|
|
Training (including failed experiments)
|
2 days
|
$400 (GPU hours)
|
$1,500 (ML engineer time)
|
$1,900
|
|
Evaluation and safety testing
|
1 week
|
$200 (inference costs)
|
$4,000 (clinical + ML engineer)
|
$4,200
|
|
Total
|
~3.5 weeks
|
$650
|
$15,500
|
$16,150
|
Compare illustrative staff time and compute costs across curation, preprocessing, training, and evaluation. section 6.12.4




