A training dataset is the collection of examples a machine-learning model analyzes while learning patterns. For generative coding tools, those examples include existing source code in many languages. The model uses them to form predictions about what new code should look like, rather than retrieving a guaranteed-correct answer. section 1.3 section 2.1
A training dataset affects generated output in several practical ways:
- It teaches statistical patterns, such as common word combinations, code structures, and relationships between parts of text. section 1.5.2
- It transfers its weaknesses: errors, misleading information, incorrect code, outdated Python patterns, and common but suboptimal solutions can appear in later suggestions. section 1.5.2
- It supports probability-based prediction, so a model may produce convincing but incorrect output when flawed examples shaped its learning. section 1.5.2
GitHub Copilot
GitHub Copilot uses the OpenAI Codex model, trained on billions of lines of code from GitHub repositories and other publicly available code. When a developer writes code, Copilot uses the project context it can view to generate suggestions. The useful workflow is therefore predictive rather than authoritative: accept, reject, and verify suggestions instead of assuming that polished code is correct. section 2.1 section 1.3
Figure 1.1 Integrated tools use a sophisticated system to generate code. The process starts with your prompt, and the assistant gathers up documentation and source code to see whether your answer can come from these sources. It makes a best guess at what you’re looking for and generates a response. Your acceptance of these responses helps train the assistant in the future (unless you’ve blocked feedback).
Follow the feedback loop from a developer prompt through context gathering and prediction to generated code and feedback. section 1.3
