GPU matrix-multiplication kernels operate on fixed-size tiles. When a pruned feed-forward width misses a tile boundary, the kernel pads it upward and still allocates, moves, and multiplies the extra columns. Pruning to an aligned width avoids paying for that discarded work. section 12.7.1
A 20% width cut
Cutting a width of 8,192 by 20% leaves 6,553 neurons after truncation. With a 128-column alignment, the kernel serves 6,656 columns, wasting 103; snapping down to 6,528 avoids that padding. section 12.7.1
Listing 12.4 Reporting the padding tax and snapping to an aligned width
def padding_report(target_width, multiple=128): #1
served = ((target_width + multiple - 1) // multiple) * multiple
return served, served - target_width
def snap_to_aligned(target_width, multiple=128): #2
return (target_width // multiple) * multiple
raw = int(8192 * 0.80)
served, waste = padding_report(raw)
aligned = snap_to_aligned(raw)
print(f"naive target {raw}: served {served}, wasted columns {waste}")
print(f"aligned target {aligned}: served {aligned}, wasted columns 0")Use the Python helpers to report a target width’s served width and padding waste, or snap the target down to an aligned boundary. section 12.7.1





















