liveBook · Manning

Fine-tuning

As explained in Applied Reinforcement Learning

Further training of a pretrained model on data or objectives suited to a particular task or use.

Fine-tuning adapts a general-purpose language model to a specific signal schema by training it on thousands of examples. In portfolio management, this turns a capable but poorly matched zero-shot model into a more useful signal generator. section 12.8

Measure whether the change helps: signal agreement with the oracle rose from 81% to 91% across two iterations, while the lead over other signal modes widened. Changing one component at a time helps trace whether improvements reach downstream performance. section 12.8

Table 12.2 Five modes on the test window, averaged over all episodes
Mode
Sharpe
Annualized Return
Annualized Volatility
Maximum Drawdown
Turnover
Final Value
Buy-and-hold SPY
0.77
26.6%
34.4%
−12.33%
1.00
1.0432
Equal-weight
0.91
16.0%
17.5%
−6.34%
1.16
1.0216
RL, no news
1.41
40.9%
29.1%
−9.76%
4.07
1.0571
RL + naive signal
2.52
67.1%
26.6%
−7.06%
6.09
1.0876
RL + base LLM signal
2.62
66.7%
25.5%
−7.07%
6.49
1.0871
RL + fine-tuned LLM signal
8.47
169.5%
20.0%
−2.18%
8.96
1.2251

Compare test-window results across portfolio strategies; the fine-tuned signal mode reports a Sharpe ratio of 8.47, versus 2.62 for the base LLM signal. section 12.8

How Applied Reinforcement Learning covers it

How deep it goes
Explains training on thousands of schema examples and reports measured results, but gives no training procedure.
Examples
Uses a portfolio signal schema, oracle agreement, and comparisons between fine-tuned, base, and naive signals.
What you can do after
Evaluate whether task-specific tuning improves signal agreement and downstream portfolio performance.
How it connects
Connects pretrained language models with signal generation and reinforcement learning for portfolio management.

Also explained in

Compare how 52 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.