Ray Tune
Ray Tune is a Ray library for distributed hyperparameter tuning and experiment management.
Designing Deep Learning Systems: A guide for software engineers
Ray Tune is a library for distributed hyperparameter tuning and experiment execution built on the Ray platform.
- How it is used
- Automates hyperparameter searches for model training on local machines, managed clusters, or multinode Ray deployments.
- How deep it goes
- Explains objective functions, search spaces, schedulers, and distributed execution, while stopping short of detailed algorithm internals.
- Examples
- Demonstrates convolutional-network training with learning rate and momentum searches, ASHA scheduling, and multinode cluster execution.
- What you can do after
- Build it from scratch by defining an objective function, search space, scheduler, and distributed Tune run.
Read how Designing Deep Learning Systems: A guide for software engineers explains Ray Tune →
Transformers in Action
Ray Tune is a Ray library for distributed hyperparameter tuning and experiment management.
- How it is used
- Orchestrates distributed hyperparameter searches and experiment management for model training.
- How deep it goes
- Implements dataset conversion, tokenization, distributed training, search scheduling, checkpoint selection, retraining, and model loading.
- Examples
- Uses SuperGLUE CommitmentBank classification with searched learning rates, epochs, batch sizes, and weight decay.
- What you can do after
- Apply it via Ray Tune to compare configurations, select the lowest-loss trial, retrain, restore, and publish a model.
Related concepts
- ASHA schedulerused by · The ASHA scheduler is an asynchronous successive-halving algorithm that allocates resources to promising hyperparameter trials while stopping underperforming trials early.
- hyperparameter optimization servicekind of · A software service that coordinates automated searches for effective hyperparameter values by running and comparing multiple training trials.
- librarykind of · A software library is a reusable collection of code or components that provides functionality for other programs to incorporate or invoke.
- Rayuses · Ray is an open-source framework for building and scaling distributed applications, including Python and machine-learning workloads.
- resource isolationcontrasts with · Resource isolation separates or limits computational, storage, or communication resources so one workload cannot materially interfere with another.
- distributed programmingimplements · Distributed programming is the design of software that coordinates computation across multiple networked or otherwise separate machines, processors, or processes.
Questions about Ray Tune
What is Ray Tune?
Ray Tune is a Ray library for distributed hyperparameter tuning and experiment management.
Which Manning books explain Ray Tune?
Designing Deep Learning Systems: A guide for software engineers and Transformers in Action.
What is the difference between Ray Tune and resource isolation?
Ray Tune is a Ray library for distributed hyperparameter tuning and experiment management. Resource isolation separates or limits computational, storage, or communication resources so one workload cannot materially interfere with another.
What is Ray Tune a kind of?
Ray Tune is a kind of hyperparameter optimization service. A software service that coordinates automated searches for effective hyperparameter values by running and comparing multiple training trials.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

