liveBook · Manning

Normalization

As explained in Vibe Engineering

Normalization is the process of transforming data into a consistent representation or scale suitable for comparison and analysis.

Normalization turns inconsistent business data into a representation that supports comparison and analysis. It follows data selection, cleaning, and parsing, but it must be verified: plausible numbers and charts can still conceal flawed logic. section 4.0

For drug analytics, use normalization as a deliberate feature-engineering workflow:

  • Prefer structured short_composition fields for analysis and machine learning; retain salt_composition for display, metadata, and simple text searches. section 4.4
  • Reshape ingredient information into long-form records such as drug_id, ingredient, and strength when multiple composition fields require consistent analysis. section 4.4
  • Derive explicit features: mark Is_Multi_Salt from the presence of a second composition, then parse salt names, dosage values, and dosage units. section 4.4
  • Check missing values and inspect outputs against domain meaning before trusting downstream statistics or models. section 4.0 section 4.4

Normalizing medicine composition

For Amoxycillin (500mg), extract Salt_Name = Amoxycillin, Dosage_Value = 500, and Dosage_Unit = mg. Set Is_Multi_Salt according to whether a second composition exists; keep the original text for display or validation. section 4.4

Listing 4.4 Output of feature engineering notebook
 
[1] Extracted 'Salt_Name'. Showing top 5 unique salts:
Salt_Name
Cefixime       8992
Domperidone    8869
Aceclofenac    8683
Amoxycillin    8550
Ofloxacin      6768
Name: count, dtype: int64
 
[2] Extracted 'Dosage_Value' and 'Dosage_Unit'.
Products with Dosage Value: 223054/253973
 
--- Feature Engineering Results (First 10 Rows) ---
| name                            | short_composition1        | Salt_Name     |   Dosage_Value | Dosage_Unit   | Is_Multi_Salt   |
|:--------------------------------|:--------------------------|:--------------|---------------:|:--------------|:----------------|
| Augmentin 625 Duo Tablet        | Amoxycillin  (500mg)      | Amoxycillin   |            500 | mg            | True            |
| Azithral 500 Tablet             | Azithromycin (500mg)      | Azithromycin  |            500 | mg            | False           |
| Ascoril LS Syrup                | Ambroxol (30mg/5ml)       | Ambroxol      |            nan | nan           | True            |
| Allegra 120mg Tablet            | Fexofenadine (120mg)      | Fexofenadine  |            120 | mg            | False           |
| Avil 25 Tablet                  | Pheniramine (25mg)        | Pheniramine   |             25 | mg            | False           |
| Allegra-M Tablet                | Montelukast (10mg)        | Montelukast   |             10 | mg            | True            |
| Amoxyclav 625 Tablet            | Amoxycillin  (500mg)      | Amoxycillin   |            500 | mg            | True            |
| Azee 500 Tablet                 | Azithromycin (500mg)      | Azithromycin  |            500 | mg            | False           |
| Atarax 25mg Tablet              | Hydroxyzine (25mg)        | Hydroxyzine   |             25 | mg            | False           |
| Ascoril D Plus Syrup Sugar Free | Phenylephrine (5mg)       | Phenylephrine |              5 | mg            | True            |

New data with engineered features saved to 'features_engineered_medicine_data.csv'.

Inspect the generated feature table to compare original composition fields with normalized salt, dosage, and multi-salt features. section 4.4

How Vibe Engineering covers it

How deep it goes
Moves from a normalization requirement to structural ingredient normalization and parsed features, without detailing a standalone normalization algorithm.
Examples
Uses drug composition fields, Amoxycillin dosage parsing, Is_Multi_Salt, and a generated feature-engineering table.
What it assumes
Assumes basic programming knowledge while teaching field selection and parsing from drug records.
What you can do after
Apply structural normalization by selecting short_composition, reshaping ingredients, and parsing salt and dosage fields.
How it connects
Connects normalization with data cleaning, parsing, feature engineering, machine learning, and statistical analysis.

Also explained in

Compare how 24 books explain normalization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.