3 Encoder models in action: Semantic-Based Retrieval Systems

 

This chapter covers

  • A general overview and evolution of information retrieval systems
  • Introduction to keyword-based search
  • Building semantic search with encoders
  • Semantic similarity metrics

The previous chapter provided a comprehensive overview of the transformer architecture, including explorations into the encoder and decoder models, followed by an overview of self-attention mechanisms, positional encodings, and multi-head attention. Building upon those core concepts, this chapter delves deeper into encoder models and one of their most impactful applications: semantic-based information retrieval (IR) systems.

An IR system is a software system that stores and manages information (documents, web pages, articles, and others) and provides interfaces for users to find and access the information they need. At a high level, information retrieval systems have three core functions:

3.1 Information Retrieval Systems: A Historic Overview

3.2 Keyword Search using Inverted Index and TF-IDF

3.2.1 Implementing keyword search using inverted index and TF-IDF

3.3 Semantic Search from Scratch

3.4 Implementing Semantic Search with Python and Sentence Transformers

3.5 Summary