Preprocessing makes raw examples usable for LLM fine-tuning. It sits within data preparation, before model selection and training: collect examples, format them consistently, and validate what the model will learn from them. section 5.3
Clean without discarding information the model needs:
- Remove unwanted artifacts, such as strange Unicode and broken markup, that could confuse the model. section 5.3.1
- Preserve meaningful domain notation, such as legal section markers or code snippets, rather than stripping it out during cleaning. section 5.3.1
- Keep example structure consistent. In technical-support data, role labels and a newline distinguish the customer’s issue from the agent’s response; solutions retain the intended tone and product terms. section 5.3.1
{"prompt": "Customer: <issue description>\nAgent:", "completion":
" <step-by-step resolution>\n"}
A prompt/completion JSONL record shows the Customer-to-Agent pattern used to prepare technical-support examples. section 5.3.1
Cleaning is not the whole preparation job. Keep validation and test examples out of training data to avoid leakage, and decide whether the task calls for general patterns or deliberate memorization of a reference text. Validation performance can then help identify overfitting. section 5.3.1
