Ray Tune is a Ray library for distributed experiment execution and hyperparameter tuning. It coordinates training, evaluation, scheduling, checkpoint selection, and final model training around a PyTorch workflow. section 8.2
Ray Tune
Ray Tune is a Ray library for distributed hyperparameter tuning and experiment management.
How Transformers in Action covers it
- How deep it goes
- Implements dataset conversion, tokenization, distributed training, search scheduling, checkpoint selection, retraining, and model loading.
- Examples
- Uses SuperGLUE CommitmentBank classification with searched learning rates, epochs, batch sizes, and weight decay.
- What you can do after
- Apply it via Ray Tune to compare configurations, select the lowest-loss trial, retrain, restore, and publish a model.
- How it connects
- Connects Ray Data, TorchTrainer, PyTorch, Hugging Face Transformers, checkpointing, and ASHA scheduling.
Also explained in
-
Designing Deep Learning Systems: A guide for software engineersRay Tune is a library for distributed hyperparameter tuning and experiment execution built on the Ray platform.
Compare how 2 books explain Ray Tune →
Related concepts
- Rayuses · Ray is an open-source distributed-computing framework for scaling Python, artificial-intelligence, and machine-learning applications across processes, machines, and clusters.
- librarykind of · A software library is a reusable collection of code or components that other programs can incorporate or invoke to provide functionality.
- hyperparameter optimization servicekind of · A software service that coordinates automated searches for effective hyperparameter values by running and comparing multiple training trials.
- ASHA schedulerused by · The ASHA scheduler is an asynchronous successive-halving algorithm that allocates resources to promising hyperparameter trials while stopping underperforming trials early.
- resource isolationcontrasts with · Resource isolation separates or limits computational, storage, or communication resources so one workload cannot materially interfere with another.
- distributed programmingimplements · Distributed programming is the design of software whose computation is coordinated across multiple networked machines, processors, or processes.
This explainer was generated by Manning's knowledge graph from the text of Transformers in Action. The section links point to the parts of the book it draws on.