Data preprocessing prepares raw documentation, support material, or source code for model training by parsing it into a structured dataset. For instruction-based fine-tuning, records commonly contain instructions, inputs, and expected outputs. The central problem is preserving enough context for each record to teach a useful behavior. section 12.2.1 section 12.2.2
A practical preprocessing workflow includes:
- Choose a record structure that matches the desired training task, such as instruction, input, and output fields. section 12.2.2
- Parse unstructured documentation or code into records, using automation when the corpus is large and manual curation when its cost is acceptable. section 12.2.1
- Preserve logical context instead of splitting code indiscriminately into isolated lines such as `}` or `@Autowired`. section 12.2.1
- Design and encode the prompt format so the fine-tuning process can combine instruction and input data consistently. section 12.2.2
- Run the prepared data through a tokenizer before fine-tuning so text becomes tokens ready for training. section 12.2.4
Avoiding context-free code records
A line-by-line parser can produce a table containing isolated fragments: `1, @RequestMapping(...)`, `2, public ResponseEntity<Branding>...`, and `3, SQLException`. A context-preserving parser instead groups the controller declaration, injected service, endpoint annotation, method body, and return statement into a coherent record. That record can support an instruction such as “How does the method getBranding work?” with a code-based answer. section 12.2.1
{"instruction": "What is the class declaration for
➥BrandingServiceIT","output": "@ExtendWith
➥@SpringBootTest
➥@ActiveProfiles
public class BrandingServiceIT { }"}
{"instruction": "How does the method setup Restito work for
➥BrandingServiceIT?","output": "
➥@BeforeEach
➥public void setupRestito(){
➥whenHttp(server).match(post("/auth/validate"))
➥.then(status(HttpStatus.OK_200));
➥}"}
Browse paired instruction-and-output records showing how parsed Java test code becomes fine-tuning data. section 12.2.1
