Semantic adaptation changes what a model means or reasons about, rather than only changing how it expresses an answer. Common-sense reasoning, visual understanding, and multimodal instruction are examples of tasks that require it. section 7.4
DoRA separates magnitude and direction during training, a design that may help with deep semantic adaptation. By contrast, structured extraction mainly changes output form; QLoRA is the faster pragmatic choice for that task, while QDoRA’s semantic-task advantage remains a hypothesis to test. section 7.4 section 7.3.5
Testing for a semantic-task advantage
Replace a clinical extraction dataset with a reasoning dataset such as a subset of databricks-dolly-15k, keep the notebook’s code and hyperparameters unchanged, and compare QLoRA with QDoRA for general-capability retention and training stability. section 7.5
