chapter twelve
12 Building Reinforcement Learning systems: LLM-guided RL agent for portfolio management
This chapter covers
- Why a working reinforcement learning system is mostly engineering, not algorithm
- A practical framework for designing real-world RL systems
- Integrating LLM fine-tuning into an RL system
- Building, evaluating, and iterating an end-to-end LLM-guided trading system
In theory, there is no difference between theory and practice. In practice, there is.
Benjamin Brewster, American industrialist
Over the previous four chapters you have built a Deep Q-Network, a REINFORCE agent with a Transformer policy, a full PPO-based RLHF pipeline, and a GRPO fine-tuner with LoRA and verifiable rewards. You understand the algorithms. You could re-derive the policy gradient theorem on a whiteboard and explain why PPO clips its ratio. And now someone hands you a real problem — manage a portfolio, route a fleet, cool a building, align a model — and asks you to build something that actually works, that they can run every day, that will not embarrass them the first time reality deviates from the slide deck.