Model rearchitecting adapts a pretrained model by physically changing its architecture or components to meet deployment constraints or specialized requirements. Unlike prompt engineering or fine-tuning alone, it can remove layers, reconfigure attention, or reshape internal blocks. section 1.0 section 1.1.3
A domain-specific dataset can guide calibration and pruning decisions, then specialize the recovered model. A general dataset supports knowledge recovery after pruning. This structural approach can complement RAG by adapting how a model retrieves and uses external information. section 1.2.5 section 1.1.2
A rearchitecting workflow can include:
- Establish baseline capabilities and inference metrics. section 2.0
- Calibrate on relevant data, identify less useful components, and prune them. section 1.2.5
- Recover capabilities through distillation, then specialize with LoRA when needed. section 1.2.5
Financial reports
For quarterly earnings analysis, calibrate a 7B general-purpose model on financial reports, prune components that contribute least to the task, recover general capabilities through distillation, and specialize with LoRA. The resulting model is described as smaller, faster, and more accurate for the use case. section 1.2.5
Figure 1.2 Dataset integration in the rearchitecting pipeline. The domain-specific dataset guides calibration of the base model, informs structural optimization decisions, and enables final specialization through LoRA fine-tuning. A general dataset supports Knowledge Recovery, ensuring the pruned model retains broad capabilities before domain-specific specialization. This dual approach optimizes each phase for the project’s objectives.
The figure maps how domain-specific and general datasets support different stages of the rearchitecting pipeline. section 1.2.5
