GPU kernel
A GPU kernel is an executable program or function that performs computation on a graphics processing unit, commonly using parallel threads.
LLMs in Production
A kernel is a small program or function executed on a parallel processor to perform a computational operation.
- How it is used
- Supports model compilation and deployment tuning by mapping neural-network operations onto efficient processor routines.
- How deep it goes
- Explains kernel selection and tuning factors, but stops short of implementation code or detailed hardware execution mechanics.
- Examples
- Uses convolution algorithms and fused convolution-bias-ReLU operations to illustrate alternative kernels and graph optimization.
- What you can do after
- Evaluate/understand how hardware and tensor characteristics influence kernel selection and deployed model performance.
Fast Python: High performance techniques for large datasets
- How it is used
- Dispatches per-element GPU computation from a CPU entry point, distributing work across blocks and threads.
- How deep it goes
- Implements Numba CUDA kernels, launch configuration, and bounds checks; defers shared-memory algorithms as advanced.
- Examples
- Doubles array elements and matrix entries, then compares Numba and CUDA C Mandelbrot kernel timings.
- What you can do after
- Build it from scratch by writing a Numba CUDA function, assigning threads to elements, and launching a configured grid.
Read how Fast Python: High performance techniques for large datasets explains GPU kernel →
Deep Learning with JAX
A compiled low-level executable unit for performing an operation on target hardware, not a convolution kernel.
- How it is used
- Uses weighted matrices to blur, denoise, or sharpen images through convolution.
- How deep it goes
- Explains windowed multiplication, summation, channel handling, and clipping, but does not cover compiled executable kernels.
- Examples
- Demonstrates blur, Gaussian denoising, sharpening, per-channel color convolution, and comparisons of noisy, blurred, restored, and original images.
- What you can do after
- Apply it via 2D convolution to construct blur, denoising, and sharpening image filters.
Related concepts
- GPUuses · A graphics processing unit (GPU) is a processor designed for highly parallel computation, especially graphics and data-parallel workloads.
- threaduses · A thread is an independently schedulable sequence or context of execution within a program or process.
- functionkind of · A function is a callable program unit that performs an operation, may accept inputs, and can produce a result or side effect.
- PyTorchused by · PyTorch is an open-source machine-learning framework for Python that provides tensor computation, automatic differentiation, and tools for building and training neural networks.
- boundary checkuses · A boundary check is a test that verifies whether a value or operation lies within an allowed range or valid data extent.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.