3 Encoder models in action: Semantic-Based Retrieval Systems
This chapter covers
- A general overview and evolution of information retrieval systems
- Introduction to keyword-based search
- Building semantic search with encoders
- Semantic similarity metrics
The previous chapter provided a comprehensive overview of the transformer architecture, including explorations into the encoder and decoder models, followed by an overview of self-attention mechanisms, positional encodings, and multi-head attention. Building upon those core concepts, this chapter delves deeper into encoder models and one of their most impactful applications: semantic-based information retrieval (IR) systems.
An IR system is a software system that stores and manages information (documents, web pages, articles, and others) and provides interfaces for users to find and access the information they need. At a high level, information retrieval systems have three core functions: