A persistent volume claim (PVC) is a Kubernetes storage request made by a containerized application. It provides workloads with access to storage managed by cluster infrastructure, either through dynamic provisioning or an existing claim. section 7.1.2
PVCs solve production pipeline problems caused by downloading datasets repeatedly, wasting storage, and limiting data sharing. A pipeline can create one claim, mount it at /data, and reuse the dataset across download, split, training, and validation components. section 9.1 section 9.1.7
Configure and operate a pipeline PVC by:
- Choosing an access mode such as ReadWriteOnce, a requested size such as 1Gi, and a storage class such as local-path. section 9.1.3
- Mounting the claim into each component with a consistent path such as /data. section 9.1.5
- Declaring execution order explicitly with .after() when shared-volume usage does not create an inferred data dependency. section 9.1.5
- Removing direct MinIO downloads and multiple input-path arguments from training and validation components. section 9.1.6 section 9.1.7
Shared dataset for object detection
A Kubeflow Pipeline creates yolo-pipeline-pvc with ReadWriteOnce access, 1Gi capacity, and the local-path storage class. The download component writes the dataset, split_dataset_op reorganizes files under /data, and training and validation mount the same claim. TensorBoard can later mount the completed claim to read training logs. section 9.1.3 section 9.1.5 section 9.2.1
Listing 9.1 VolumeOp to create a Pipeline PVC
@dsl.pipeline(name="YOLO Object Detection Pipeline") def pipeline( epochs: int = 1, batch: int = 8, random_state: int = 42, yolo_model_name: str = "yolov8n_custom" ): pvc_name = 'yolo-pipeline-pvc' pvc = kubernetes.CreatePVC( #1 pvc_name=pvc_name, access_modes=['ReadWriteOnce'], #2 size='1Gi', #3 storage_class_name='local-path', #4 )
Inspect the VolumeOp configuration that creates a pipeline PVC and specifies its name, access mode, size, and storage class. section 9.1.3
