A random forest combines predictions from multiple decision trees. It can classify ad clicks, but training choices depend on data size: Dask parallelizes training locally or across machines, while PySpark scales it across a Spark cluster. section 8.1.3 section 8.2.6
A practical random-forest workflow includes:
- Read the ads dataset and create numeric features from site categories. section 8.1.3
- Assemble multiple numeric columns into a single feature vector when using PySpark. section 8.2.6
- Split the data into training and test sets, such as 70% and 30%. section 8.2.6
- Choose hyperparameters such as the number of trees and maximum tree depth, then fit the classifier and generate predictions. section 8.2.6
Predicting ad clicks
A PySpark workflow creates a RandomForestClassifier with 100 trees and a maximum depth of 2, fits it on the training data, and transforms the test data to produce predictions. The resulting output can show actual clicks, predicted probabilities, and predicted classes.
Listing 8.12 In this code, we train a random forest model on the ads dataset using Dask. This code combines previous examples of reading in the ads dataset, connecting to a local Dask cluster, creating new features, splitting the ads data into train and test, and finally training a random forest model to predict ad clicks.
import time #1
import joblib
from dask.distributed import LocalCluster, Client
import dask.dataframe as dd
from sklearn.ensemble import RandomForestClassifier
from dask_ml.model_selection import train_test_split
cluster = LocalCluster() #2
client = Client(cluster) #3
ad_frame = dd.read_parquet( #4
"data/ads_train_data.parquet",
columns=("click", "banner_pos",
"site_category"))
ad_frame["site_category_28905ebd"] = #5
ad_frame.site_category.map(lambda val:
1 if val == "28905ebd" else 0)
ad_frame["site_category_50e219e0"] =
ad_frame.site_category.map(lambda val:
1 if val == "50e219e0" else 0)
inputs = ["banner_pos", #6
"site_category_28905ebd",
"site_category_50e219e0"]
train, test = train_test_split(ad_frame, #7
train_size = 0.7,
shuffle = True,
random_state = 0)
with joblib.parallel_backend("dask"): #8
forest = RandomForestClassifier(
n_estimators = 100,
max_depth = 2,
min_samples_split = 40,
verbose = 1)
start = time.time()
forest.fit(train[inputs], train.click)
end = time.time()
print("Time to train model = ",
end - start, " seconds")
Inspect a complete Dask example that reads the ads data, creates features, splits the data, and trains a random forest locally with parallel joblib execution. section 8.1.3
















