3 Working with LLMs
This chapter covers
- The base class for working with LLMs in our LLM agent framework
- Implementatingr an LLM class that enables the use of any open-source LLM with Ollama
- A complete demonstration of the tool-call process
In the previous chapter, we began our Stage 1 build of llm-agents-from-scratch by writing base classes for tools as well as the necessary data structures that they work with. We’ll continue our Stage 1 build here by similarly adding a base class for LLMs and the data structures that will enable the various modes of interacting with LLMs we want to support in our framework.
One such mode is the tool-calling process, which we’ll finally be able to execute in its entirety by the end of this chapter. Specifically, we’ll learn how to elicit a tool-call request from an LLM and how to submit the result of the tool invocation we covered in the previous chapter back to the LLM for synthesis and response.
After establishing our base class, BaseLLM, we’ll move on to the very exciting task of building an integration with Ollama, a highly popular open-source LLM inference framework. We’ll do this by implementing OllamaLLM, a subclass of BaseLLM, which will enable the use of any of the many open-source LLMs supported by Ollama, including those from the Llama and Qwen families of models. Figure 3.1 shows our updated build plan, highlighting the progress we’ve made so far and our current focus.