7 Vibe Performance Engineering: When assumptions mislead
This chapter covers
- Are LLM tools prone to the same mistakes as humans for performance engineering?
- Optimizing to the hot-path vs automatically addressing broader performance issues
- Identifying the implicit assumptions about the trade-offs that LLM tools can make
In this chapter, we will attempt to address two main problems related to improving the performance of our code, that is, optimizing based on false assumptions and not optimizing the hot-path.
Before delving into vibe engineering techniques, let’s describe those two main concepts that are impacting the performance engineering process. The first issue is premature optimization, specifically when code is optimized without input data on expected traffic and a service level agreement (SLA), making it difficult to reason about the code's required performance. Optimizing random paths in code in such a situation is like shooting in the dark. You will complicate your code without a sane reason. We want to prevent a situation where the code is prematurely or inaccurately (for a given traffic) optimized, resulting in over-engineered and hard-to-maintain code. Human engineers are often tempted to optimize code that may achieve better performance prematurely, but this performance may not be utilized since the traffic the code is expected to handle may be orders of magnitude lower compared to the traffic where the code actually demonstrates its improved performance.