Data analysis examines datasets to guide reliable machine-learning work. In production, it must stay close to the data, support interactive exploration, and connect to reproducible preparation pipelines. section 7.0 section 7.1
A production analysis workflow must provide:
- Compute near large datasets so massive data can be processed efficiently. section 7.1
- Ad hoc access to GPUs and high-resource machines that can be started and stopped as needed. section 7.1
- An environment that resembles deployment while supporting versioning and CI/CD. section 7.1
Kubeflow notebooks provide browser-based Jupyter, VS Code, or RStudio environments on the same cluster as pipelines. Shared base images and CI-built notebook images improve reproducibility and deployment alignment. section 7.1
Inspecting an object-detection dataset
Launch a Kubeflow notebook, inspect training results, optionally load the object-detection model, and run ad hoc inference before building the data-preparation pipeline. section 7.0 section 7.1
Figure 7.1 The mental map where we’re now focusing on the first step/component of our pipeline—data collection and preparation (3)
Use the pipeline mental map to locate data collection and preparation as the first operational focus. section 7.0
