Data cleaning detects, corrects, removes, or manages inaccurate, incomplete, inconsistent, or irrelevant data before analysis or machine learning. It includes parsing, normalization, handling malformed records, and deciding whether unusual values represent real cases or broken data. section 4.0 section 4.1
The appealing shortcut is to use a dataset that already looks clean, especially synthetic data. That can produce unrealistically accurate analyses or models because it avoids outliers, malformed records, and other real-world difficulties. Treat cleaning decisions as model assumptions, and verify the resulting numbers against business context: plausible output can still hide flawed logic. section 4.0 section 4.1
A practical cleaning workflow for price prediction includes:
- Load a prepared feature file and confirm that the preceding preparation step completed; otherwise, the workflow fails. section 4.5.4
- Remove rows with null Dosage_Value or Salt_Name values when the training process requires complete columns. section 4.5.4
- Exclude prices at or above the 99th percentile to reduce the influence of outliers or potentially broken values. section 4.5.4
- Separate features from the price label, then encode categorical fields such as Salt_name, Manufacturer, and Dosage_Unit numerically. section 4.5.4
- Evaluate predictions with R², mean absolute error, and root mean squared error rather than trusting a successful run alone. section 4.5.4
|
Use Case
|
Dataset A (Indian)
|
Dataset B (Synthetic)
|
|
Exploratory Data Analysis (EDA)
|
✅ Strong — real
distributions, price trends
|
⚠️ Limited — synthetic distributions
|
|
Supervised ML for Prediction
|
⚠️ Limited (no target label by
default)
|
⚠️ Same — likely no target labels
|
|
Clustering / Unsupervised learning
|
✅ Yes — cluster by
manufacturer, composition
|
⚠️ Yes but less real-world
meaning
|
|
NLP/Categorization
|
⚠️ Yes — text fields (names/composition)
|
⚠️ Yes (names) but synthetic
semantics
|
|
Recommender systems
|
⚠️ Possible but needs
augmentation
|
❌ Limited
|
|
Benchmarking models / DEMO
|
⚠️ Possible but raw
|
✅ Ideal due to cleaner,
controlled data
|
|
Aspect
|
Indian Medicine Dataset (A)
|
Kaggle Medicine Dataset (B)
|
|
Source
|
Scraped real data
|
Synthetic generated data
|
|
Size
|
~250 K records
|
~50 K records
|
|
Real-world
|
✅ Yes
|
❌ No
|
|
Cleanliness
|
⚠️ No (requires cleaning)
|
✅ Likely
|
|
Usability for Research
|
High
|
Moderate
|
|
ML Power
|
Good for real analysis
|
Good for ML practice
|
Compare real and synthetic datasets by cleanliness, realism, exploratory-analysis value, and machine-learning use before choosing what to clean. section 4.1

















