Pruning

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Pruning removes selected decision-tree splits after or during construction to reduce tree complexity.

Pruning removes decision-tree splits after construction, especially partitions containing too few training examples, to reduce overfitting. It produces a simpler tree that may fit training data less perfectly but can classify new examples more reliably. section 5.1

Use pruning when a fully grown tree has become too specific:

• Work backward through the tree and remove splits enclosing very few training examples. section 5.1
• Compare pruning with limiting growth in advance, such as imposing a maximum number of splits. section 5.1

Figure 5.3 The problem handled by a simpler decision tree, obtained by pruning or by limiting its growth

Shows how pruning produces a simpler decision tree with fewer data partitions. section 5.1

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes

Explains backward split removal and its trade-off, but stops short of pruning algorithms or implementation steps.

Examples

Uses fully grown and simpler partition diagrams, including regions containing only one training case.

What it assumes

Assumes familiarity with decision-tree splits and terminal nodes while explaining overfitting through data partitions.

What you can do after

Evaluate whether to simplify an overfit tree by pruning backward or limiting growth before construction.

How it connects

Connects decision trees with splits, data partitions, terminal nodes, and overfitting.

liveBookmanning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  LLMs in ProductionStructured pruning removes whole structural components of a neural network, such as filters, channels, or layers.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsPruning is the removal of branches or subtrees from a decision tree to simplify the model and limit unnecessary complexity.
• 

  liveBookmanning.com
  Advanced Algorithms and Data StructuresPruning is the elimination of search-tree branches that cannot contain a valid or better candidate.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionPruning is the removal or restriction of branches in a decision tree to reduce excessive specificity and improve generalization.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA model-compression technique that removes less important weights or connections to reduce model size and cost.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionPruning is the removal of selected model parameters or computational structures to reduce model size or computation.

Compare how 7 books explain pruning →

Related concepts

• baggingcontrasts with · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
• decision treeuses · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
• overfittingaddresses · Overfitting is a modeling condition in which a model fits patterns or idiosyncrasies in its training data too closely and consequently generalizes poorly to new data.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.