chapter seven

7 Vibe performance engineering: when assumptions mislead

 

This chapter covers

  • Are LLM tools prone to the same mistakes as humans for performance engineering?
  • Optimizing to the hot-path vs automatically addressing broader performance issues
  • Identifying the implicit assumptions about the trade-offs that LLM tools can make

Performance work often goes wrong in two predictable ways: optimizing based on false assumptions, and failing to optimize the hot-path. The hot-path is a part of your code that does most of the work and is executed for almost every user request. Both mistakes lead to the same outcome: more complexity, more maintenance burden, and little real benefit in production.

Understanding these two concepts is essential before delving into vibe engineering techniques, as they directly impact the performance engineering process.

The first issue is premature optimization, specifically when code is optimized without input data on expected traffic or a service-level agreement (SLA), making it difficult to reason about the code's required performance. Optimizing random code paths in such a situation is like shooting in the dark. You complicate your code for no good reason.

7.1 Mistakes of LLMs over-engineering performance improvements

7.1.1 The human solution

7.1.2 Vibe performance engineering solution

7.1.3 Adding traffic expectations to the vibe performance engineering solution

7.2 Vibe performance engineering of the hot-path in code

7.2.1 Understanding the Pareto principle in the context of software systems

7.2.2 A word service with a potential hot-path

7.2.3 Hot-path detection in your code

7.3 Measuring the code

7.3.1 Measuring the code with vibe performance engineering techniques

7.3.2 Mistake of generating too much code

7.4 Improvements for hot-path performance

7.4.1 Improving performance with vibe performance engineering techniques

7.4.2 Comparing performance results based on Gatling simulations

7.5 Concluding thoughts

7.6 Summary