liveBook · Manning

Memory bound

As explained in Enterprise RAG

A bottleneck is a component or operation that limits system throughput or causes excessive delay under load.

A performance bottleneck is a component or operation that limits throughput or creates excessive delay under load. In a RAG application, it may appear as a 429 rate-limit error, a missing API response, or answers that take around ten seconds to return. section 5.4.3

Diagnose bottlenecks through a load-testing and remediation loop:

  • Run Locust tests until failures or unacceptable latency reveal the constrained component. section 5.4.3 section 5.5
  • Inspect Web App logs to identify whether the problem is API rate limiting, insufficient application capacity, or another failing stage. section 5.4.3 section 5.5
  • Scale only the constrained resource: raise token quotas or use Azure OpenAI, add Web App CPU and memory, increase instances, or raise AI Search capacity. section 5.4.3 section 5.5
  • Reduce demand as well as adding capacity; shrinking search results from four chunks to two lowers token consumption and cost. section 5.4.3
Figure 5.7 If we experience any failures during out Locust tests, we first look at our logs in our web app. Next, once we have identified the error, we fix it by scaling out that particular part of our system. For example, if we are getting rate limits from OpenAI, we increase the amount of requests we can send.

Use the diagnostic flow to connect Locust failures with application logs, identify the failing subsystem, and scale that subsystem rather than the entire system. section 5.4.3

How Enterprise RAG covers it

How deep it goes
Explains symptoms, log-based diagnosis, and targeted capacity changes, but stops short of detailed implementation code.
Examples
Uses 429 errors, ten-second answers, missing API responses, excessive search chunks, and overloaded Web App instances as concrete cases.
What it assumes
Assumes a deployed RAG application with a Web App, external model or search services, logging, and Locust-based load tests.
What you can do after
Diagnose/debug with it by using Locust and logs to isolate constrained services, then apply targeted scaling or demand reduction.
How it connects
Connects performance bottlenecks with load testing, logging, rate limiting, token quotas, horizontal scaling, AI Search, and Web App capacity.

Also explained in

Compare how 18 books explain memory bound →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Enterprise RAG. The section links point to the parts of the book it draws on.