Fine-tuning continues training a pretrained model, or selected components, on a task-specific dataset. It is a practical deep-learning baseline when a related model already exists, especially for computer vision and language problems. section 8.4 section 8.6.2
Choose fine-tuning within a baseline progression:
- Start with constant or rule-based baselines, then consider pretrained-model reuse, feature extraction with a shallow model, or fine-tuning. section 8.4 section 8.7
- Prefer fine-tuning when direct reuse or pretrained representations do not provide enough task-specific performance, and compare accuracy against development effort, interpretability, and computation time. section 8.4 section 8.7
- Avoid assuming that greater model complexity will pay off; a simple transfer-learning baseline can remain difficult to beat. section 8.4 section 8.7
PhotoStock image search
For PhotoStock, CLIP can first produce an image–query relevancy score without additional training. A later step can fine-tune CLIP or its components on the company’s dataset, potentially combining image and description signals. section 8.6.2
