Ray distributes Python work across multicore CPUs or cluster nodes. It is especially useful when machine-learning pipelines or models exceed one GPU, turning distributed execution into a small code change rather than a multithreading or asynchronous setup. section 3.3.2
Ray organizes parallel work around two primitives:
- Tasks are functions that Ray schedules across available resources. section 3.3.2
- Actors are stateful objects that can be invoked and run concurrently. section 3.3.2
- A cloud Ray cluster can place separate LLM pipelines on nodes with the required GPUs, simplifying infrastructure setup. section 3.3.2
Listing 3.3 Example Ray parallelization task
import ray
import time
ray.init() #1
def slow_function(x): #2
time.sleep(1)
return x
@ray.remote
def slow_function_ray(x): #3
time.sleep(1)
return x
results = [slow_function(i) for i in range(1, 11)] #4
results_future = [slow_function_ray.remote(i) for i in range(1, 11)] #5
results_ray = ray.get(results_future)
print("Results without Ray: ", results)
print("Results with Ray: ", results_ray)
ray.shutdown()
A minimal comparison shows ordinary calls running sequentially and @ray.remote calls submitted concurrently, with results gathered through futures. section 3.3.2
