liveBook · Manning

Feature engineering

As explained in Vibe Engineering

A feature engineering pipeline is a sequence of transformations that creates model inputs from raw data.

Feature engineering transforms raw data into inputs a model can use. The shortcut is to ask an LLM to generate the transformations; the risk is that plausible parsing code silently loses information. Check its output before relying on the resulting features. section 4.4 section 4.7

Extract medicine features

For a medicine dataset, parse short_composition1 such as “Amoxycillin (500mg)” into Salt_Name, numeric Dosage_Value, and Dosage_Unit. Set Is_Multi_Salt to true when both composition fields are present and false when short_composition2 is missing. A pandas script can use a regular expression for dosage parsing and save the resulting columns to a CSV for later modeling. section 4.4

Specify the input fields, output columns, and diagnostic output when requesting a script. Inspect the most frequent extracted salts, count rows with dosage values, and review sample results. A coverage count is not an accuracy test: compare extracted values with the original strings, especially for unusual formats. section 4.4 section 4.7

| Alex Syrup                      | Phenylephrine (5mg/5ml)   | Phenylephrine |            nan | nan           | True            |

A dosage expressed as 5mg/5ml produced missing dosage values, illustrating a format that the extraction missed. section 4.4

In one run, 223,054 of 253,973 products had an extracted dosage value. Investigate missing values and refine the parsing rather than assuming that generated code captures every usable record. Real datasets can contain messy distributions and outliers that cleaner synthetic data does not reflect. section 4.4 section 4.7

How Vibe Engineering covers it

How deep it goes
Works through field extraction, diagnostic counts, sample inspection, and saving processed data, while emphasizing verification of LLM-generated parsing.
Examples
Uses medicine composition strings to extract salt names and dosages and identify multi-salt products.
What you can do after
Apply it via Python and pandas to create and inspect features from medicine records.
How it connects
Connects feature engineering with string parsing, regular expressions, unit normalization, and predictive modeling.

Also explained in

Compare how 24 books explain feature engineering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.