A Kubernetes service provides a stable DNS address for reaching a pod whose IP address may change. In distributed training, the job tracker creates a service for the master pod so worker pods can initialize communication without depending on an ephemeral pod IP. section 4.3.3
A distributed-training service coordinates the following responsibilities:
- Uses the service DNS name as MASTER_ADDR and exposes MASTER_PORT to every training pod. section 4.3.3
- Provides network connectivity between the master pod and worker pods while Kubernetes schedules the pods. section 4.3.2
Three-pod training group
A three-instance run creates one master pod and two worker pods. All receive WORLD_SIZE=3 and MASTER_PORT=12356; the master has RANK=0, while workers use ranks 1 and 2. Each pod reaches the master through the DNS name intent-classification-1-master-service.
