9 Observing AI operations

In chapter 6, I asked you to think about your last visit to see the doctor. Think about that visit once again.

I’ll bet that while you were there, the doctor or a nurse took all kinds of measurements, such as your temperature, blood pressure, and heart rate. They may have even taken blood and tested it for a variety of conditions. In some situations, you may have been given a CT scan to get an even deeper view into how your body and systems are functioning. The measurements and tests gave them a high-level view into your overall health and likely informed their thoughts on how best to treat you.

Your applications can be thought of similarly to your health. Just as a doctor uses your vital statistics and test results to better understand your overall health, you can better understand the health and behavior of your application by observing various metrics that the application produces. Building observability and tracing into your application can give you valuable insights and clarity into the inner-working of the application.

9.1 Enabling the Actuator metrics

9.1.1 Inspecting vector store operations

9.1.2 Examining AI model interaction

9.1.3 Counting token usage

9.1.4 Observing ChatClient operations

9.2 Viewing metrics in Prometheus

9.3 Creating AI dashboards

9.4 Tracing AI operations

9.5 Summary