The RAG Scalability Coefficient (RSC) measures whether a model keeps extracting value as retrieval depth increases. It accumulates the area under the performance curve until the last retrieval depth that still produces a gain of at least epsilon. Higher values indicate useful context integration across a wider range of k, especially for multi-hop questions. section 15.1.2
Use RSC alongside the RAG Stability Score to separate two retrieval-depth behaviors:
- Peak-then-decline models benefit from a narrow retrieval range and may lose answer quality when additional passages introduce noise. section 15.9
- Improving-and-plateauing models continue gaining value across more retrieved passages, producing a higher RSC. section 15.1.2
- Evaluate RSC from per-depth answer-quality scores such as F1, exact match, or RAGAS faithfulness. section 15.1.2
Robust versus sensitive readers
A robust reader reaches its optimum at k=7, remains close to that peak, and scores RSC 268.5. A sensitive reader peaks at k=3, drops sharply around its optimum, and scores RSC 70.5. The RSC difference indicates how broadly each model benefits from additional retrieved context; RSS separately captures local brittleness. section 15.1.2
Listing 15.1 Computing RAG stability score and scalability coefficient.
import numpy as np
def compute_rag_metrics(
scores: list[float],
delta: int = 2,
epsilon: float = 0.5
) -> dict[str, float]:
"""Compute RSS and RSC from per-k performance scores."""
scores_arr = np.array(scores, dtype=float)
best = int(np.argmax(scores_arr)) #A
peak = scores_arr[best]
window_start = max(0, best - delta)
window_end = min(len(scores_arr), best + delta + 1)
window = np.concatenate([ #B
scores_arr[window_start:best],
scores_arr[best + 1:window_end]
])
rss = float(np.min(window) / peak) if peak > 0 else 0.0
k_last_gain = 0 #C
for i in range(1, len(scores_arr)):
if scores_arr[i] - scores_arr[i - 1] >= epsilon:
k_last_gain = i
rsc = 0.0
for i in range(k_last_gain):
rsc += 0.5 * (scores_arr[i] + scores_arr[i + 1])
return {
"k_star": best + 1,
"rss": round(rss, 3),
"rsc": round(rsc, 1),
}
robust_model = [20, 35, 42, 48, 52, 54, 55, 55, 55, 55] #D
sensitive_model = [20, 38, 45, 42, 38, 35, 33, 31, 30, 29]
print("Robust model:", compute_rag_metrics(robust_model))
print("Sensitive model:", compute_rag_metrics(sensitive_model))
Inspect the Python listing to compute RSC and RSS from a sequence of per-retrieval-depth performance scores, with configurable delta and epsilon parameters. section 15.1.2
