chapter thirteen
13 Read-only data optimization
This chapter covers
- Why reading the same value repeatedly from global memory wastes cache space
- How constant memory’s dedicated cache avoids that waste
- The three-step pattern for putting read-only data into constant memory
- How a single broadcast serves all 32 threads of a warp in one fetch
- Why threads that read different addresses are serialized and up to 32× slower
Imagine 32 students who all need the same one-page answer key. The classroom is smart: the first student fetches it from the storage room and leaves a copy on a shared shelf just outside the classroom door, and the next 31 students grab it from there, no trip to the storage room needed. That sounds efficient, until the shelf fills up and gets cleared, tossing out that answer key to make room for something else before anybody notices. Now the next student who needs it has to make the slow trip to the storage room all over again.