An ablation study removes or disables system components, then compares performance to determine which parts contribute to the result. Reliable comparisons use structurally similar models and an unseen test set. section 3.2.1 section 4.5.4
A useful ablation checks:
- Whether removing a feature causes a meaningful performance drop rather than a minor, potentially noisy gain or loss. section 3.2.1
- Whether accuracy changes justify added latency, memory, or index size. section 3.2.1
- Whether individual components improve evidence synthesis, retrieval coverage, or answer focus. section 4.5.4
RAG component removal
A RAG system scored 44.5% with its full configuration. Removing marginalization reduced performance to 41.5%; using pretrained models without fine-tuning reduced it to 37.2%; setting K=1 instead of K=5 reduced it to 39.8%. These comparisons attribute measurable contributions to the disabled components. section 4.5.4




