liveBook · Manning

BigQuery Storage API

As explained in Build Python Web Apps with Streamlit

The BigQuery Storage API is Google's high-performance interface for reading BigQuery data into client applications.

The BigQuery Storage API lets a Python application read BigQuery data into a Pandas dataframe faster than using the BigQuery API alone. In a Streamlit dashboard, it works alongside BigQuery: BigQuery runs the query, while the Storage API helps convert the returned data for dataframe use. section 7.7.2 section 7.7.3

A practical setup follows these steps:

  • Enable both the BigQuery API and BigQuery Storage API in the Google Cloud project. section 7.7.2
  • Install google-cloud-bigquery, google-cloud-bigquery-storage, and db-dtypes. section 7.7.3
  • Authenticate with service-account credentials stored in Streamlit secrets, then reuse the BigQuery client credentials for the Storage API client. section 7.7.2

Loading dashboard data

A modular data_loader.py can create a BigQuery client, initialize BigQueryReadClient with the same credentials, run SELECT * against a project.dataset.table reference, and return result.to_dataframe(bqstorage_client=storage_client). Updating only load_data lets the rest of the dashboard continue working while its data source changes from a static CSV file to BigQuery. section 7.7.3

How Build Python Web Apps with Streamlit covers it

How deep it goes
Explains enablement, packages, credentials, client construction, querying, and dataframe conversion, but not Storage API internals.
Examples
Uses a Streamlit CEO dashboard, a data_loader.py function, a sales_data table, and SELECT * query.
What it assumes
Assumes a Google Cloud project and BigQuery table, while teaching service-account authentication and API enablement.
What you can do after
Apply it via google-cloud-bigquery-storage to replace static CSV loading with authenticated BigQuery dataframe loading.
How it connects
Connects BigQuery, Pandas, service accounts, Streamlit secrets, SQL, and separation of concerns.
Cover of Build Python Web Apps with Streamlit
liveBookmanning.com
Read Build Python Web Apps with Streamlit on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build Python Web Apps with Streamlit. The section links point to the parts of the book it draws on.