liveBook · Manning

GPU

As explained in Inside Deep Learning: Math, Algorithms, Models

A graphics processing unit used as a parallel compute device for computation.

A graphics processing unit (GPU) is specialized hardware that can also accelerate tensor calculations. In PyTorch, this makes GPUs useful for the compute-intensive operations in neural networks. section 1.2.1

GPU work has a transfer cost: data must move from the CPU to the GPU, and results may need to move back. For small calculations, this overhead can make CPU execution faster; benchmarking helps compare the two for a given workload. section 1.2.1

When using a GPU with PyTorch:

  • Check that CUDA is available and move tensors to the chosen device before computing. section 1.2.1
  • Keep tensors in an operation on the same device; move a CUDA tensor back to the CPU before converting it to NumPy. section 1.2.1
def moveTo(obj, device): 
    """ 
    obj: the python object to move to a device, or to move its
     contents to a device
    device: the compute device to move objects to 
    """
    if isinstance(obj, list): 
        return [moveTo(x, device) for x in obj] 
    elif isinstance(obj, tuple): 
        return tuple(moveTo(list(obj), device)) 
    elif isinstance(obj, set): 
        return set(moveTo(list(obj), device)) 
    elif isinstance(obj, dict): 
        to_ret = dict() 
        for key, value in obj.items(): 
            to_ret[moveTo(key, device)] = moveTo(value, device) 
        return to_ret 
    elif hasattr(obj, "to"): 
        return obj.to(device) 
    else: 
        return obj

some_tensors = [torch.tensor(1), torch.tensor(2)] 

print(some_tensors) print(moveTo(some_tensors, device)) 
[tensor(1), tensor(2)]
[tensor(1, device='cuda:0'), tensor(2, device='cuda:0')]

A recursive helper moves tensors inside nested Python containers to a selected compute device. section 1.2.1

Google Colab offers limited-time GPU access, avoiding the need to buy a GPU just to get started with deep learning. section 1.1

How Inside Deep Learning: Math, Algorithms, Models covers it

How deep it goes
Explains tensor acceleration, transfer overhead, device mismatches, and benchmarking, but does not detail GPU hardware internals.
Examples
Demonstrates matrix-multiplication timing, CUDA availability, device-mismatch errors, and moving nested objects to a device.
What you can do after
Apply it via PyTorch by checking CUDA, moving tensors to a device, and benchmarking workloads against CPU execution.
How it connects
Connects GPU use to PyTorch tensors, neural networks, CPU transfers, and CUDA device placement.

Also explained in

Compare how 15 books explain GPU →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.