Compute capability identifies GPU architecture features that determine whether a serving runtime can execute a supported kernel. For AWQ weights, vLLM uses Marlin-family INT4 kernels on compute capability sm80 and newer; without a matching kernel, serving may fail or dequantize weights to FP16. section 12.1 section 12.5
L4 versus T4
An Ada L4 has compute capability sm89, so it runs the four-bit kernel. A Turing T4 has sm75, below the kernel floor, and may fall back to FP16. Quantization and serving therefore have different hardware requirements. section 12.1 section 12.5
