TabularDataset is a managed Vertex AI dataset resource that gives a machine-learning training job a named, reusable handle for tabular data. Instead of passing a local CSV directly to the job, upload the processed file to Cloud Storage, create a Vertex AI dataset, and retain its dataset ID for pipeline configuration. section 11.2.7 section 11.5
A managed dataset supports the training workflow by:
- Referencing the source data through a resource path assembled from the project, region, and dataset ID. section 11.3.6
- Making the dataset available through Cloud Storage for the training script. section 11.2.7
- Dividing the data into training, validation, and test subsets according to the pipeline configuration, then splitting those subsets into CSV files. section 11.2.7
- Serving as the dataset argument when a CustomTrainingJob runs the training script. section 11.3.6
Using a Kuala Lumpur real-estate dataset
After uploading the cleaned Kuala Lumpur real-estate CSV to a Cloud Storage location such as gs://first-project-ml-tabular-bucket/processed_dataset/kl_real_estate_output.csv, create a tabular regression or classification dataset in Vertex AI and record its ID. The pipeline can then construct and use it as follows: dataset_path = 'projects/' + project_id + '/locations/' + region + '/datasets/' + dataset_id; ds = aiplatform.TabularDataset(dataset_path); model = job.run(dataset=ds, training_fraction_split=training_fraction, validation_fraction_split=validation_fraction, test_fraction_split=test_fraction). section 11.2.6 section 11.2.7 section 11.3.6
dataset_path = \ 'projects/'+config['project_id']+\ '/locations/'+config['region']+\ '/datasets/'+config['dataset_id'] ds = aiplatform.TabularDataset(dataset_path)
Inspect the Python listing that builds the managed dataset resource path and creates the TabularDataset object used by the pipeline. section 11.3.6
The resource does not replace pipeline preparation: the cleaned file must already be uploaded, and the training container must use Cloud Storage locations because it cannot access the surrounding file system. section 11.5
