liveBook · Manning

Feature engineering

As explained in Software Engineering for Data Scientists

Feature engineering is the creation or transformation of input variables to make them suitable or informative for machine learning.

Feature engineering creates new model inputs from existing variables so the data is more useful for prediction. It can improve model performance through transformations, and it is sometimes required because a raw value cannot be used directly by a model. section 1.2.7

Figure 1.4 A sample data science workflow. Data science workflows involve key steps like gathering data, exploring and cleaning the data, feature engineering, and model development. The model evaluation piece is also a highly important step, that we will cover in detail when we discuss model monitoring later in this book.
sample data science workflow

Feature engineering sits between data cleaning and model development in a broader workflow that also includes gathering, exploring, and evaluating data. section 1.2.7

Common ways to engineer features include:

  • Transform a variable when a model benefits from inputs with a normal distribution. section 1.2.7
  • Parse a transaction date into day of week, hour of day, or month for a credit-fraud prediction task. section 1.2.7
  • Bucket a customer’s age into ranges such as under 18, 18–24, 25–30, and 31–40 for customer-churn prediction. section 1.3.1
  • Create indicator columns from a categorical site-category field, such as one column for each selected category. section 8.1.2

Choose where the transformation runs

A bucketed_age feature can be created in the data pipeline as a table column, or after extraction in preprocessing code before model input. The important practical choice is to make the transformation part of the path from source data to the finalized training dataset. section 1.3.1 section 1.4.2

Listing 8.10 We can create new columns in Dask the same way we do in pandas.
#1
ad_frame["site_category_28905ebd"] =
     ad_frame.site_category.map(lambda val:
     1 if val == "28905ebd" else 0)

ad_frame["site_category_50e219e0"] =
     ad_frame.site_category.map(lambda val:
     1 if val == "50e219e0" else 0)

This Dask example creates two new indicator columns from site_category with map operations, illustrating feature creation in a dataframe workflow. section 8.1.2

Treat feature engineering as an operational component, not only an experiment. It may live in data-pipeline tables or preprocessing code, may be split across multiple files, and needs monitoring because corrupted or missing feature logging can reduce prediction accuracy. Extra feature steps can also affect memory consumption in chunked workflows. section 1.4.2 section 1.4.6 section 7.7

How Software Engineering for Data Scientists covers it

How deep it goes
Defines feature engineering and demonstrates date parsing, age bucketing, and categorical columns without detailing transformation algorithms.
Examples
Uses transaction dates for credit fraud, bucketed customer ages for churn, and site-category indicator columns for ad-click data.
What you can do after
Apply it via Dask or a data pipeline to create columns and transformations before training a model.
How it connects
Connects feature engineering to data cleaning, preprocessing, data pipelines, machine learning pipelines, model training, and model monitoring.

Also explained in

Compare how 24 books explain feature engineering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Software Engineering for Data Scientists. The section links point to the parts of the book it draws on.