`@dsl.component` turns a Python function into a reusable Kubeflow Pipelines component. The function can then be invoked inside a `@dsl.pipeline` definition. At execution time, each component runs in a container on a Kubernetes pod, so its Python dependencies must be available there—not merely in the environment where the pipeline was written. section 7.2.1 section 7.3.1
Pass a simple value between components
A joke generator can return text for a word-counting component. Configure its external dependency on the decorator and import it inside the function: import kfp.dsl as dsl @dsl.component(packages_to_install=["pyjokes"], base_image="python:3.11") def generate_joke() -> str: import pyjokes return pyjokes.get_joke() @dsl.component def count_words(input: str) -> int: return len(input.split()) @dsl.pipeline(name="joke_pipeline") def pipeline(): joke = generate_joke() count_words(input=joke.output) The first component’s output supplies the second component’s input; the decorator defines each function as a component, while `@dsl.pipeline` connects their invocations. section 7.2.1
Choose an explicit way to provide dependencies:
- Use `base_image` when the required packages are already installed in the image. The illustrated lightweight components use `python:3.11`. section 7.2.1
- Use `packages_to_install` to name packages the component needs installed in its execution environment, such as `pyjokes` or `requests`, `boto3`, and `tqdm`. section 7.2.1 section 7.3.1
- Keep imports inside the function so its dependencies are encapsulated for container packaging. Installing packages from within the function is possible, but less explicit and not recommended. section 7.2.1 section 7.3.1
The same pattern scales beyond returned strings and integers. A dataset-download component can declare an `Output[Dataset]`; a splitting component can accept an `Input[Dataset]` and declare separate training, validation, and test outputs. Invoke these components in a pipeline and connect the resulting dataset outputs to downstream inputs. Inspect the compiled pipeline YAML when you need to see how component execution is specified. section 7.3.1
Listing 7.19 Using multiple Output[Dataset] and inner imports
@dsl.component(
packages_to_install=["scikit-learn"],
base_image="python:3.11"
)
def split_dataset(
random_state: int,
input_dataset: Input[Dataset], #1
train_dataset: Output[Dataset], #2
validation_dataset: Output[Dataset], #2
test_dataset: Output[Dataset] #2
):
import os
import glob
import shutil
from sklearn.model_selection import train_test_split
The dataset-splitting signature shows `@dsl.component` paired with explicit package and image settings, inner imports, and typed dataset inputs and outputs. section 7.3.1
