appendix E Open source LLMs
In earlier chapters, you worked with OpenAI’s public REST API. It’s a straightforward way to build large language model (LLM) applications because you don’t need to set up a local LLM host. After signing up with OpenAI and generating an API key, you can send requests to their endpoints and access LLM capabilities. This quick setup lets you work with state-of-the-art models efficiently. The main drawback is cost—running examples such as summarization might cost a few cents or even dollars. If you’re working on projects for your company, privacy might also be a concern. Some employers block OpenAI entirely to avoid the risk of leaking sensitive or proprietary data.
This appendix introduces open source LLMs, a practical solution for reducing costs and addressing privacy concerns. These models are especially appealing to individuals and organizations that prioritize data confidentiality or are new to AI. I’ll guide you through the most popular open source LLM families, their features, and the advantages they offer. The focus will be on running these models, ranging from high-performance, advanced setups to user-friendly tools that are ideal for learning and experimentation.
Finally, I’ll show you how to transition the summarization and QA systems you built earlier to a local open source LLM. By the end of this appendix, you’ll understand open source LLMs well and feel confident using them when they’re the right choice.