Preprocessing transforms raw, unevenly organized data into a usable dataset for machine learning. Begin by locating the target output, then identify the predictors that will be mapped to it. Because data preparation and pipelining often consume substantial effort, understanding the data scheme and combining stored data correctly are central parts of the work. section 2.3
A practical preprocessing workflow includes:
- Inspect columns and distributions to distinguish numeric-looking data from genuinely numeric features. section 9.1
- Use unique-value counts as a useful indication of categorical or continuous features, while investigating ambiguous columns rather than applying the rule mechanically. section 9.1
- Measure dataset dimensions and missing values before cleanup, then address columns according to their specific data problems. section 9.1 section 9.2
- Initially clean the complete feature set when practical, because examining all available columns can reveal invalid values and preserve data for later projects. section 9.2
Kuala Lumpur real estate data
A real estate dataset contains 53,883 rows and eight columns. Location, Property Type, and Furnishing appear categorical; Price, Bathrooms, Car Parks, and Size appear continuous; Rooms requires further investigation. Although Price, Rooms, and Size contain values that look numeric, descriptive statistics show that only Bathrooms and Car Parks are numeric until those fields are processed. Missing values also require attention: Car Parks has 17,567 missing entries, Furnishing 6,930, and Bathrooms 2,013. section 9.1 section 9.2
Listing 9.1 Getting the count of unique values in each column
counts = df.nunique() #1
print("unique value counts:\n",counts)
Inspect unique-value counts with a compact DataFrame operation, then compare the counts across columns to investigate likely feature types. section 9.1
