A data manipulation routine transforms raw or combined datasets into forms suitable for analysis, visualization, statistical investigation, or an interactive proof of concept. Typical operations include filtering, sorting, grouping, and reshaping. section 6.2.4 section 10.2.4
A practical routine usually moves through these tasks:
- Load one or more CSV or Excel datasets, including large files containing millions of rows. section 6.2.4 section 8.2.4
- Combine datasets and inspect or reshape them before analysis. section 6.2.4 section 5.2.4
- Filter records to match the analytical question, then calculate summary measures or prepare visualizations. section 8.2.4 section 11.2.2
- Pass the transformed data into statistical analysis, forecasting, dashboards, or interactive applications when needed. section 8.2.4 section 10.2.4
Filtering a house-price explorer
An interactive application lets users select a county, town, and street. The routine builds a progressively specific query, retrieves the matching records, calculates record count and median sale price, and supplies charts and a raw-data table. section 11.2.2 section 11.2.2
house_filter_query = "county == @county_select"
filter_message = f"Results for {county_select}"
if town_select and town_select != TOWN_NULL_VALUE:
house_filter_query += " and town_city == @town_select"
filter_message += f", {town_select}"
if street_select and street_select != STREET_NULL_VALUE:
house_filter_query += " and street == @street_select"
filter_message += f", {street_select}"
selected_data = wales.query(house_filter_query)
Inspect an executable pandas pattern for constructing conditional geographic filters and retrieving the selected data. section 11.2.2
