liveBook · Manning

SQLite

As explained in Software Engineering for Data Scientists

SQLite is an embedded relational database engine that stores a complete database in a local file.

SQLite is a lightweight relational database engine suited to prototyping machine-learning pipelines. It stores prediction outputs in a database that can be queried outside the Python session. section 9.2.1

A practical churn-prediction workflow uses SQLite to:

  • Create or open a local customers database with sqlite3. section 9.2.1
  • Create a customer_churn_predictions table with customer ID, model prediction, and prediction date columns. section 9.2.1
  • Upload model predictions from a pandas DataFrame, then retrieve them with sqlite3, pandas, or SQL. section 9.2.1

Querying stored predictions

After connecting with sqlite3.connect("data/customers.db"), execute SELECT * FROM customer_churn_predictions and fetch the returned records. Alternatively, use pandas.read_sql(...) to load the query result into a DataFrame. section 9.2.1

Figure 9.2 This diagram extends Figure 9.1 to also show the sample output table, which can be stored in an RDMS, like MySQL, SQL Server, SQLite, etc. The view shows a sample record of the data, with values for customer_id, model_prediction, and date (prediction date).
ch9 database diagram

View how a machine-learning workflow writes customer_id, model_prediction, and prediction_date into a relational database output table. section 9.2.1

How Software Engineering for Data Scientists covers it

How deep it goes
Explains database creation, table definition, prediction upload, and querying, while deferring scheduled production jobs.
Examples
Demonstrates a customer-churn prediction table, sqlite3 queries, pandas read_sql, SQL statements, and DB Browser views.
What it assumes
Assumes basic Python, pandas, SQL, and model-pipeline familiarity while introducing database and relational-table usage.
What you can do after
Apply it via sqlite3, pandas, or SQL to persist and retrieve model predictions in a local database.
How it connects
Connects SQLite with relational databases, SQL, Python, pandas, and machine-learning pipelines.

Also explained in

Compare how 16 books explain SQLite →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Software Engineering for Data Scientists. The section links point to the parts of the book it draws on.