Before deciding what customer or order data means, find out what is actually in the files. pandas is a Python library for working with tabular data in DataFrames. It provides a practical starting point for loading business records, inspecting their structure, and checking assumptions before analysis. section 2.3
A useful first pass through a DataFrame is:
- Run `head()` to see sample rows and column names, then check `shape` for the number of rows and columns. section 2.3
- Use `info()` for data types and non-null counts, `describe()` for numerical summaries, and `isna().sum()` to count missing values. section 2.3
- Use `value_counts()` to inspect category frequencies. When identifiers have uncertain meanings, compare their distinct counts with `nunique()` and examine their relationship with `groupby()`. section 2.3
Check what a customer identifier represents
A customer table contains both `customer_id` and `customer_unique_id`. Counting distinct values and grouping one identifier by the other reveals that a single unique customer can have multiple customer IDs—up to 17 in the reported results. That check matters before treating either column as a count of people. section 2.3
Listing 2.1 Preparing variables for further analysis later in the chapter
import pandas as pd
df_customers = pd.read_csv('olist_customers_dataset.csv')
df_order_items = pd.read_csv('olist_order_items_dataset.csv')
df_order_payments = pd.read_
csv('olist_order_payments_dataset.csv')
df_order_reviews = pd.read_csv('olist_order_reviews_dataset.csv')
df_orders = pd.read_csv('olist_orders_dataset.csv')
df_products = pd.read_csv('olist_products_dataset.csv')Jupyter
df_sellers = pd.read_csv('olist_sellers_dataset.csv')
df_product_category_translation =
pd.read_csv('product_category_name_translation.csv')
See how `pd.read_csv()` loads separate customer, order, payment, review, product, and seller CSV files into DataFrames for subsequent analysis. section 2.3
pandas also supports a simple review-search workflow: apply a Portuguese positive-keyword check to a review column, filter matching rows, and save them as a CSV. Treat the resulting file as candidates, not verified positive sentiment. A review may contain a positive word while expressing a complaint, and the basic generated function does not handle missing review text. Inspect matches and address such cases before relying on the result. section 5.5.6
import pandas as pd
# Load your dataframe
df = pd.read_csv('olist_order_reviews_dataset.csv')
# List of positive keywords in Portuguese
positive_keywords = ['ótimo', 'excelente', 'bom', 'incrível',
'maravilhoso', 'perfeito', 'gostei', 'satisfeito', 'recomendo',
'amei']
# Define a function to check if a review contains positive keywords
def is_positive(review):
for keyword in positive_keywords:
if keyword.lower() in review.lower():
return True
return False
# Apply the function to the 'review_comment_message' column
df['is_positive'] = df['review_comment_message'].apply(is_positive)
# Create a new dataframe with only positive reviews
positive_reviews_df = df[df['is_positive']]
# Save the positive reviews to a new CSV file
positive_reviews_df.to_csv('positive_reviews.csv', index=False)
The generated keyword-search listing shows `apply()`, boolean row filtering, and `to_csv()` together; its missing-value handling needs attention before use on real reviews. section 5.5.6
