1 Small Language Models
This chapter covers
- The definition of Small Language Model
- An overview of the Transformers architecture
- The major areas of application for Language Models
- Limits and risks of closed source generalist Large Language Models
- Open Source alternatives to closed source LLMs
- A comparison between LLMs and Domain-specific Language Models
Whenever a new, potentially disruptive technology emerges, or an existing one undergoes a radical change, there's an inevitable hype about it. This hype can result in a swift drop in interest if expectations aren't met or, conversely, lead to a rapid increase in investments and projects before its true business potential and associated risks are fully understood. Either way, nothing good would come out from a new big thing. You can observe the same happening nowadays to Large Language Models (LLMs): huge hype about them which makes it difficult to understand what they really can do (and cannot do) and if they really could be a good fit for some of your own business needs. But hype is the only thing you wouldn’t find in this book. This chapter aims to clarify the LLM world by highlighting pros, risks and challenges of both generalist and domain-specific models.