liveBook · Manning

Independent variable

As explained in Timeless Algorithms, The Foundational Ideas

A feature is an input attribute whose values are considered by a predictive model when making predictions.

An independent variable is an input attribute whose values a predictive model considers when making predictions. In linear regression it is also called an explanatory variable or predictor; decision trees and classification use the term feature. section 3.4.1 section 5.3.1 section 8.1.1

yi=β0+β1xi+εi

In simple linear regression, xᵢ is the input for observation i; the model relates it to response yᵢ. The slope describes the expected response change associated with a one-unit increase in xᵢ, not necessarily a causal effect. section 3.4.1

A feature’s role varies by model. A decision tree tests features at internal nodes to divide observations into groups, with each root-to-leaf path forming a sequence of decision rules. In one-dimensional classification, a feature places observations along an axis, where a decision boundary separates predicted classes. section 5.3.1 section 8.1.1

Temperature as a feature

Temperature readings serve as the single feature for classifying manufacturing conditions as safe or unsafe. A threshold marks the decision boundary; when several thresholds separate the observations, the maximum-margin approach chooses the one farthest from the nearest observation in either class. section 8.1.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines the feature through regression notation and illustrates its roles in regression, tree splits, and one-dimensional classification.
Examples
Uses regression inputs, income or credit history for tree splits, and temperature for safe-or-unsafe classification.
What you can do after
Evaluate/understand how a model uses an input attribute to estimate a response, split observations, or classify cases.
How it connects
Links features to linear regression, decision trees, classification, support vector machines, response variables, and decision boundaries.

Also explained in

Compare how 5 books explain independent variable →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.