15 Tool use and function calling
Language models using tools is a natural way to expand their capabilities, especially for high-precision tasks where external tools contain the information or for agents that need to interact with complex web systems. These can be thought of in a few strategies where tool use is the general category. An AI model uses any external tools by outputting special tokens to trigger a certain endpoint. These can be anything from highly specific tools, such as functions that return the weather at a specific place, to code interpreters or search engines that act as fundamental building blocks of complex behaviors.
The exact origin of the term "tool use" is not clear, but the origins of the idea far predates the post ChatGPT world where RLHF proliferated. Early examples circ 2015 attempted to build systems predating modern language models, such as Neural Programmer-Interpreters (NPI) [1], "a recurrent and compositional neural network that learns to represent and execute programs." As language models became more popular, many subfields were using integrations with external capabilities to boost performance. To obtain information outside of just the weights many used retrieval augmented generation [2] or web browsing [3]. Soon after, others were exploring language models integrated with programs [4] or tools [5].