liveBook · Manning

GPGPU

As explained in Retrieval Augmented Generation, The Foundational Ideas

GPU acceleration is the use of graphics processing units to execute computations faster or in greater parallelism than general-purpose processors.

GPU acceleration uses parallelism to speed up large-scale similarity search. In FAISS, it can help keep exact, brute-force search practical rather than requiring an approximate index. section 2.2.2 section 2.2.3

Move an IVF index to the GPU

FAISS can transfer an inverted-file index to a GPU, then train it and add embeddings: res = faiss.StandardGpuResources() gpu_index = faiss.index_cpu_to_gpu(res, 0, index) gpu_index.train(embeddings) gpu_index.add(embeddings) section 2.2.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Demonstrates GPU-backed IVF setup and relates GPU speedups to search scale without quantifying GPU-specific latency.
Examples
Demonstrates transferring and populating an IVF index on GPU, alongside a million-vector FAISS search example.
What you can do after
Apply GPU acceleration through FAISS by transferring, training, and populating a GPU-backed inverted-file index.
How it connects
Connects GPU parallelism with FAISS, vector similarity search, inverted-file indexes, and retrieval-augmented generation.

Also explained in

Compare how 6 books explain GPGPU →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.