gRPC is a remote procedure call framework for communicating between distributed services. Its concise service definitions make it practical for exposing APIs without introducing unnecessary framework detail. section 2.2.3 section 3.1.1
A service declares callable methods with request and response types. A dataset management API uses CreateDataset, UpdateDataset, GetDatasetSummary, and ListDatasets, with the last method returning a stream of summaries. section 2.2.3
A production-oriented gRPC workflow can separate file transfer from service communication:
- Upload data to cloud object storage and obtain a downloadable URL. section 2.2.3
- Send the URL and dataset metadata in a CreateDataset request. section 2.2.3
- Let the service download, transform, store, and summarize the data before returning a response. section 2.2.3
Dataset ingestion API
rpc CreateDataset (CreateDatasetRequest) returns (DatasetSummary); rpc UpdateDataset (CreateCommitRequest) returns (DatasetSummary); rpc GetDatasetSummary (DatasetPointer) returns (DatasetSummary); rpc ListDatasets (ListQueryOptions) returns (stream DatasetSummary).









