The RAG Stability Score (RSS) measures how consistently a reader model performs near its optimal retrieval depth. It matters when retrieval quality fluctuates and changing k can move a system away from its best configuration. section 15.1.1 section 15.1.2
Let k* be the retrieval depth with the highest answer-quality score. RSS divides the lowest score within a chosen window around k* by the peak score. Values near 1 indicate a forgiving configuration; lower values indicate brittleness. section 15.1.2
Use RSS to distinguish two retrieval-depth behaviors:
- Improve-then-plateau readers gain quality until roughly k=10, then remain stable; FLAN-T5 reached RSS 0.99. section 15.1.1
- Peak-then-decline readers perform best at small k and degrade with added distractors; LLaMA-3 scored about 0.63–0.64. section 15.1.1
- Treat retrieval depth as a production tuning parameter for noise-sensitive readers rather than retrieving many passages defensively. section 15.1.1
Robust versus sensitive readers
Illustrative scores produced RSS 0.945 for a robust reader and 0.444 for a sensitive reader. The robust reader stayed within 6% of its peak around k*, while the sensitive reader varied sharply near its optimum. section 15.1.2
Figure 15.1 Two distinct reader scaling behaviors from RAGGED.

Compare plateauing and declining reader-performance curves as retrieval depth increases. section 15.1.1
Listing 15.1 Computing RAG stability score and scalability coefficient.
import numpy as np
def compute_rag_metrics(
scores: list[float],
delta: int = 2,
epsilon: float = 0.5
) -> dict[str, float]:
"""Compute RSS and RSC from per-k performance scores."""
scores_arr = np.array(scores, dtype=float)
best = int(np.argmax(scores_arr)) #A
peak = scores_arr[best]
window_start = max(0, best - delta)
window_end = min(len(scores_arr), best + delta + 1)
window = np.concatenate([ #B
scores_arr[window_start:best],
scores_arr[best + 1:window_end]
])
rss = float(np.min(window) / peak) if peak > 0 else 0.0
k_last_gain = 0 #C
for i in range(1, len(scores_arr)):
if scores_arr[i] - scores_arr[i - 1] >= epsilon:
k_last_gain = i
rsc = 0.0
for i in range(k_last_gain):
rsc += 0.5 * (scores_arr[i] + scores_arr[i + 1])
return {
"k_star": best + 1,
"rss": round(rss, 3),
"rsc": round(rsc, 1),
}
robust_model = [20, 35, 42, 48, 52, 54, 55, 55, 55, 55] #D
sensitive_model = [20, 38, 45, 42, 38, 35, 33, 31, 30, 29]
print("Robust model:", compute_rag_metrics(robust_model))
print("Sensitive model:", compute_rag_metrics(sensitive_model))
Compute RSS from per-depth performance scores and compare it with the separate RAG Scalability Coefficient. section 15.1.2
