4 Data design and performance models
This chapter covers
- Why real applications struggle to achieve performance
- Addressing kernels and loops that significantly underperform
- Choosing data structures for your application
- Assessing different programming approaches before writing code
- Understanding how the cache hierarchy delivers data to the processor
This chapter has two topics that are intimately coupled; the (1) introduction of performance models, increasingly dominated by data movement and thus necessarily (2) the underlying design and structure of data.
Although it may seem secondary to performance, the data structure and its design is critical. It must be determined in advance, as it dictates the entire form of the algorithms, code, and later the parallel implementation. The choice of data structures and thereby the data layout often determines the performance that can be achieved and in ways that are not always obvious when the design decisions are being made. Thinking about the data layout and its performance impacts is at the core of a new and growing programming approach called data-oriented design. This approach considers the patterns of how data will be used in the program and proactively designs around it. Data-oriented design gives us a data-centric view of the world which is also consistent with our focus on memory bandwidth rather than flops. In summary, for performance, our approach is to think about