A graphics processing unit (GPU) is specialized hardware that can also accelerate tensor calculations. In PyTorch, this makes GPUs useful for the compute-intensive operations in neural networks. section 1.2.1
GPU work has a transfer cost: data must move from the CPU to the GPU, and results may need to move back. For small calculations, this overhead can make CPU execution faster; benchmarking helps compare the two for a given workload. section 1.2.1
When using a GPU with PyTorch:
- Check that CUDA is available and move tensors to the chosen device before computing. section 1.2.1
- Keep tensors in an operation on the same device; move a CUDA tensor back to the CPU before converting it to NumPy. section 1.2.1
def moveTo(obj, device):
"""
obj: the python object to move to a device, or to move its
contents to a device
device: the compute device to move objects to
"""
if isinstance(obj, list):
return [moveTo(x, device) for x in obj]
elif isinstance(obj, tuple):
return tuple(moveTo(list(obj), device))
elif isinstance(obj, set):
return set(moveTo(list(obj), device))
elif isinstance(obj, dict):
to_ret = dict()
for key, value in obj.items():
to_ret[moveTo(key, device)] = moveTo(value, device)
return to_ret
elif hasattr(obj, "to"):
return obj.to(device)
else:
return obj
some_tensors = [torch.tensor(1), torch.tensor(2)]
print(some_tensors) print(moveTo(some_tensors, device))
[tensor(1), tensor(2)]
[tensor(1, device='cuda:0'), tensor(2, device='cuda:0')]
A recursive helper moves tensors inside nested Python containers to a selected compute device. section 1.2.1
Google Colab offers limited-time GPU access, avoiding the need to buy a GPU just to get started with deep learning. section 1.1














