liveBook · Manning

Epilogue fusion

As explained in GPU Programming with Triton

Epilogue fusion combines operations applied after a primary computation into the same kernel so intermediate results remain available without a memory round trip.

Epilogue fusion combines operations after a primary computation in one kernel, keeping intermediate values out of global memory. In SwiGLU, the kernel applies SiLU to the gate projection and multiplies by the value projection, with both operations performed in registers. This avoids intermediate writes and reads that can make the elementwise work memory-bound. section 3.3.1

Simplified SwiGLU
\[\mathrm{SwiGLU}(x) = \underbrace{\mathrm{SiLU}(G)}_{\text{Gate}} \odot \underbrace{U}_{\text{Value}}\]

The simplified formula shows the gate activation and value multiplication combined in the fused epilogue. section 3.3.1

Fused SwiGLU in Triton

The kernel splits a contiguous input into gate and value halves, loads both, computes SiLU and multiplies in registers, then stores the output. section 3.3.1

How GPU Programming with Triton covers it

How deep it goes
Details register-level activation and multiplication in Triton, assuming contiguous gate and value halves.
Examples
Uses a fused SwiGLU kernel that splits gate and value inputs, computes SiLU, and multiplies elementwise.
What it assumes
Assumes the input contains two contiguous halves for the gate and value projections.
What you can do after
Apply epilogue fusion with a Triton kernel that keeps intermediate activation values in registers.
How it connects
Connects SwiGLU and SiLU activation to matrix multiplication and global-memory traffic.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.