Multilingual support enables a language model and retrieval system to understand and generate text across languages. It addresses a production failure in which English-heavy embedding models miss relevant non-English documents. section 7.4 section 2.3.4
A multilingual pipeline combines a generative LLM that writes a coherent hypothetical document in the query language with a multilingual encoder that maps queries and documents into a shared vector space. HyDE uses the generated document to reduce query-document mismatch without an explicit translation step. section 7.4 section 2.3.4
Korean question retrieval
For “인간은 언제 불을 사용했는가?” (“When did humans first use fire?”), a multilingual LLM generates a Korean hypothetical answer. paraphrase-multilingual-mpnet-base-v2 embeds that answer, compares it with Korean corpus vectors, and returns the closest document. The extra generation call improves retrieval potential but adds latency and computational cost. section 7.4
Listing 7.4 HyDE’s multilingual potential.
from sentence_transformers import SentenceTransformer
import numpy as np
query_kr = "인간은 언제 불을 사용했는가?" # "When did humans first use fire?" #A
korean_hypo = "고고학적 증거에 따르면, 인류는 약 백만 년 전에 불을 통제하여 사용하기 시작하였다." #B
encoder = SentenceTransformer( #C
'paraphrase-multilingual-mpnet-base-v2')
v_query = encoder.encode(korean_hypo)
corpus = ["불의 발견은 인간 진화의 전환점이었다.",
"도구 사용은 초기 인류의 특징이다.",
"고대 인류는 약 백만 년 전에 불을 사용했다."]
corpus_vecs = encoder.encode(corpus)
scores = np.dot(corpus_vecs, v_query) #D
best_idx = np.argmax(scores)
print("Retrieved:", corpus[best_idx])
Shows Korean hypothetical-document generation, multilingual encoding, vector scoring, and retrieval in Python. section 7.4

