An encoder transforms tokens or text sequences into dense numerical representations that preserve semantic and contextual relationships. In a Transformer, it forms one tower of a multi-layer encoder–decoder architecture; self-attention lets it relate positions across a sequence while processing them in parallel. section 2.2 section 2.5
Encoder-only models produce contextual embeddings rather than generated continuations. BERT learns bidirectional context by predicting masked words, making its representations useful for text classification, named entity recognition, sentiment analysis, and question answering. SBERT adapts this idea to whole-sentence similarity by producing embeddings that can be compared directly. section 2.3 section 2.3.1
For semantic retrieval, an encoder is the representation stage of a larger pipeline:
- Convert documents, reviews, and queries into vectors that capture meaning beyond exact keyword overlap. section 2.3.3 section 4.2
- Store and index document vectors, then compare a query vector with them using similarity or distance metrics. section 4.0
- Use a bi-encoder when query and documents must be encoded separately for efficient similarity search; use a cross-encoder when joint query-document interaction justifies greater computation. section 4.0
- Pass retrieved information to a decoder when natural-language recommendations or answers are needed in a RAG system. section 5.3.1
Encoding hotel reviews
A hotel finder can encode hotel descriptions and reviews alongside free-form requests such as “quiet hotels with ocean views and a spa.” Vector similarity supports matches based on concepts such as peacefulness even when the same keywords are absent. Retrieved hotel information can then support personalized generated recommendations. section 2.4 section 2.3.3
A practical sentence-embedding workflow loads a pretrained Sentence Transformers model, optionally moves it to CUDA, calls encode on a list of texts, and checks the resulting embedding shape. Model selection balances representation quality and multilingual support against cost, hardware control, scalability, and whether inference runs locally or through an API. section 4.2.1 section 4.2
Figure 4.2 An Overview of the various steps involved in creating Semantic Search from scratch for Hotel Finder in Paris: Travelle
Use the pipeline overview to place encoder-based embedding generation before distance-based retrieval in semantic search. section 4.0



























