liveBook · Manning

Multi-gpu environment

As explained in LLMs in Production

A multi-GPU environment is a computing setup that provides access to multiple graphics processing units for distributed or parallel machine-learning workloads.

A multi-GPU environment provides several graphics processors for machine-learning work that exceeds one GPU’s memory or would take too long to train. Large language models often need many more GPUs for training than inference, making cloud access a practical alternative to buying specialized hardware. section 5.1

Use a multi-GPU environment when:

  • The largest models do not fit on a single GPU. section 5.1
  • Parallel training speed matters more than local convenience. section 5.1 section 5.5
  • You can control hourly costs by shutting down or deleting unused environments. section 5.1.1
  • A smaller model is sufficient for experimentation, since single-GPU services or local hardware may be cheaper. section 5.1.1

Google Compute Engine setup

A practical route is to create a Google Cloud account and project, configure billing, install the gcloud CLI, and create a GPU-backed VM. A typical workflow initializes gcloud, creates an instance with a selected GPU machine type and boot disk, connects through SSH, copies requirements or application files, and deletes the instance when finished. section 5.1.1

How LLMs in Production covers it

How deep it goes
Explains motivation, cloud provisioning, file transfer, access, and cost control without detailing distribution algorithms.
Examples
Uses Google Compute Engine, gcloud commands, GPU-backed VMs, SSH, file copying, and instance deletion as operational examples.
What it assumes
Assumes access to cloud account creation, project billing, operating-system-specific SDK installation, and basic command-line use.
What you can do after
Operate/configure a cloud VM for larger LLM experiments, transfer project files, connect remotely, and stop unnecessary billing.
How it connects
Connects multi-GPU environments with LLM training, inference, cloud computing, virtual machines, and GPU parallelization.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.