liveBook · Manning

XGBFIR

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

XGBFIR is a project that extracts and ranks feature interactions from XGBoost models.

XGBFIR automatically discovers and ranks potentially useful feature interactions in an XGBoost model, reducing the need to inspect every feature pair manually. section 6.1.4

A practical workflow is:

  • Install xgbfir and export interactions from the fitted XGBoost estimator, including transformed feature names. section 6.1.4
  • Read the Interaction Depth 1 worksheet, sort interactions by Gain, and inspect the strongest candidates. section 6.1.4
  • Test promising pairs with a conjoint partial dependence plot rather than treating rankings as final evidence. section 6.1.4

Numeric interaction candidate

The combination of minimum_nights and calculated_host_listings_count illustrates how specific values of two numeric features can associate strongly with a positive target response. Multiplying the features can improve a gradient boosted decision-tree model more efficiently. section 6.1.4

Listing 6.8 Discovering interactions by XGBFIR
import xgbfir
xgbfir.saveXgbFI(
    model_pipeline['XGBoost'],
    feature_names=(
        model_pipeline['processing']
        .get_feature_names_out()
    ),
    OutputXlsxFile='fir.xlsx')                                    #1
fir = pd.read_excel('fir.xlsx', sheet_name='Interaction Depth 1') #2
result = fir[["Interaction", "Gain"]].sort_values(by="Gain", 
  
ascending=False).head(10).round(2)                                #3
for index, row in result.iterrows():
    print(f"{row['Interaction']}")
 
PartialDependenceDisplay.from_estimator(
    model_pipeline,
    X=data,
    kind='average',
    features=[(
        'minimum_nights',
        'calculated_host_listings_count')])                       #4

Use the listing as a compact template for exporting, ranking, printing, and validating XGBFIR interaction candidates. section 6.1.4

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes
Explains installation, export, gain ranking, and PDP validation but does not detail XGBFIR internals.
Examples
Demonstrates minimum_nights and calculated_host_listings_count as a ranked numeric interaction candidate.
What it assumes
Assumes a fitted XGBoost pipeline and access to transformed feature names.
What you can do after
Apply xgbfir to rank candidate interactions and validate them with conjoint partial dependence plots.
How it connects
Connects XGBoost with feature interactions, partial dependence plots, and gradient boosted decision trees.
Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
liveBookmanning.com
Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.