A sidecar is a companion component deployed beside a primary application. For LLM services, it moves monitoring and log processing out of the inference path, avoiding unnecessary latency on good requests while still capturing quality and safety signals. section 6.2.5
A practical sidecar arrangement:
- Runs a logging container in the same Kubernetes pod as the main application. section 6.2.5
- Processes monitoring data in parallel rather than adding expensive checks to the critical path. section 6.2.5
LLM text monitoring
A whylogs sidecar with LangKit metrics can evaluate incoming prompts and generated text for readability, complexity, and toxicity, while the primary inference service continues serving requests. section 6.2.5
