chapter seven

7 Learning with Continuous and Count Labels

 

This chapter covers

  • An introduction to regression in machine learning
  • Understanding loss and likelihood functions for regression
  • Understanding when to use different loss and likelihood functions
  • Adapting parallel and sequential ensembles for regression problems
  • Using ensembles for regression in practical settings

Many real-world modeling, prediction and forecasting problems are best framed and solved as regression problems. Regression has a rich history predating the advent of machine learning and has long been a part of the standard statistician’s toolkit.

Regression techniques have been developed and widely applied in many areas. Here are just a few examples:

  • Weather forecasting: to predict the precipitation tomorrow using data from today, including temperature, humidity, cloud cover, wind and more.
  • Insurance analytics: to predict the number of automobile insurance claims over a period of time, given various vehicle and driver attributes.
  • Financial forecasting: to predict stock prices using historical stock data and trends.
  • Demand forecasting: predict the residential energy load for the next three months using historical, demographic and weather data.

Whereas Chapters 2-6 introduced ensembling techniques for classification problems, in this chapter, we will see how to adapt ensembling techniques to regression problems.

7.1 A Brief Review of Regression

7.1.1 Linear Regression for Continuous Labels

7.1.2 Poisson Regression for Count Labels

7.1.3 Logistic Regression for Classification Labels

7.1.4 Generalized Linear Models

7.1.5 Nonlinear Regression

7.2 Parallel Ensembles for Regression

7.2.1 Random Forest and ExtraTrees

7.2.2 Combining Regression Models

7.2.3 Stacking Regression Models

7.3 Sequential Ensembles for Regression

7.3.1 Loss and Likelihood Functions for Regression

7.3.2 Gradient Boosting with LightGBM and XGBoost

7.4 Case Study: Demand Forecasting

7.4.1 The UCI Bike Rental Data Set

7.4.2 Generalized Linear Models and Stacking

7.4.3 Random Forest and ExtraTrees

7.4.4 XGBoost and LightGBM

7.5 Summary