A graphics processing unit (GPU) accelerates machine-learning computation through massively parallel processing. PyTorch uses CUDA to move tensor operations onto supported NVIDIA GPUs, speeding training and inference; it can also distribute large models across multiple GPUs. section 2.2.1
Run a Transformers pipeline on CUDA
from transformers import pipeline classifier = pipeline( model="distilbert/distilbert-base-uncased-finetuned-sst-2-english", device="cuda") classifier("The service was excellent and the food was memorable.") Setting device="cuda" allocates the pipeline to a CUDA-compliant GPU; the default device is "cpu". The pipeline returns a sentiment label and confidence score. section 3.2.4
