For a language-model market-signal extractor, a data pipeline turns raw financial news into a curated, labeled, cleaned fine-tuning dataset and structured signals an agent can use. section 12.5
Build the dataset in five stages, controlling costs and checking quality along the way:
- Collect news, curate out routine or repetitive headlines, label useful items, clean the results, and build the dataset. section 12.5
- Pilot labeling on 50 examples before scaling; run calls in parallel, make jobs resumable, calibrate labels, and validate outputs. section 12.5 section 12.11
Curate before labeling
Remove routine headlines before sending news to an expensive labeling oracle, so labeling effort goes to items more likely to carry market signals. section 12.5
Figure 12.5 The news-to-signal data pipeline, and where the cost falls.

The news-to-signal diagram shows the five-stage flow and highlights where labeling costs fall. section 12.5
