BigQuery is a fully managed Google Cloud data warehouse for storing and analyzing massive datasets with SQL. It removes the need to manage infrastructure or plan for scaling, making it a practical replacement for a static CSV when an app needs a larger or more queryable data source. section 7.7.1 section 7.7.3
Use the BigQuery interface to prepare data in a project:
- Create or select a Google Cloud project, then open BigQuery. section 7.7.1
- Create a dataset, which organizes tables within the project. section 7.7.1
- Create a table by uploading a CSV file and selecting CSV as its format. section 7.7.1
A Python application connects through Google Cloud APIs and a service account. Enable the BigQuery and BigQuery Storage APIs, create a service account with Viewer access, generate a JSON key, and store its credentials in Streamlit secrets rather than embedding them in application code. section 7.7.2
Loading a BigQuery table into Pandas
Install google-cloud-bigquery, google-cloud-bigquery-storage, and db-dtypes. Create a BigQuery client from st.secrets, reuse its credentials for the Storage client, reference the table as project.dataset.table, run SELECT * FROM that table, and convert the result to a Pandas dataframe. Keeping this work inside load_data lets the rest of the dashboard remain unchanged. section 7.7.3
For Streamlit Community Cloud deployment, omit the CSV from the repository when BigQuery supplies the data, configure the GCP credentials in the deployment settings, and list required package versions in requirements.txt. section 7.7.4
