Parameterization exposes configurable values so a component can change behavior without changing implementation code. In production ML pipelines, it prevents environment-specific settings from becoming hardcoded dependencies. section 5.1.1
Apply parameterization to values that vary across runs or environments:
- Pass hyperparameters, file paths, and configuration settings as component inputs. section 5.1.1
- Declare inputs and their types in the component definition, then forward values to the container command. section 5.1.1
- Use values such as mlflow_host, bucket_name, and minio_host to reuse components across environments. section 5.1.1
Parameterized data component
A read-data component accepts minio_host, access_key, secret_key, bucket_name, file_name, and data_output_path. Its Python implementation remains unchanged while deployment-specific values flow through the component definition. section 5.1.1
Listing 5.1 Read data component
def get_data(
minio_host: str,
access_key: str,
secret_key: str,
bucket_name: str,
file_name: str,
data_output_path: str,
):
client = Minio(
endpoint=minio_host,
access_key=access_key,
secret_key=secret_key,
secure=False,
)
local_temp_file = os.path.join("/tmp", file_name)
print(f"Downloading {file_name} from bucket {bucket_name}...")
client.fget_object(
bucket_name,
file_name,
local_temp_file,
) #1
print(f"Reading the downloaded file {local_temp_file}...")
df = pd.read_parquet(local_temp_file) #2
print(f"Saving the processed data to {data_output_path}..."
Inspect the read-data implementation to see configurable storage values used alongside a fixed data-processing workflow. section 5.1.1
