concept parameter in category machine learning

This is an excerpt from Manning's book Machine Learning with TensorFlow, Second Edition MEAP V08. Login to get full access to this book.

For any integer n, the formula 2n + 1 produces an odd number. Moreover, any odd number can be written as 2n + 1 for some value n. The number 3 can be written as 2(1) + 1. And the number 5 can be written as 2(2) + 1.

INFO:tensorflow:Restoring parameters from ./spikes.ckpt
[False False  True False False  True False  True]

copy

Figure 3.1 Different values of the parameter w result in different linear equations. The set of all these linear equations is what constitutes the linear model M.

This is an excerpt from Manning's book Machine Learning Bookcamp MEAP V06. Login to get full access to this book.

Often, we don't have two models to try, but a lot more. Logistic regression, for example, has a parameter, C, and depending on the value we set, the results can vary dramatically. Likewise, a neural network has many parameters, and each may have a great effect on the predictive performance of the final model. What's more, there are other models, each with its own set of parameters. How do we select the best model with the best parameters?

The parameters for our final model are:

This is an excerpt from Manning's book Machine Learning with R, the tidyverse, and mlr. Login to get full access to this book.

Figure 1.4. A hypothetical algorithm for learning the parameters of a straight line. This algorithm takes two continuous variables as inputs and fits a straight line through the mean. It iteratively rotates the line until it finds a solution that minimizes the sum of squares. The parameters of the line are output as the learned model.

The resulting plot is shown in figure 11.17. Notice that our elastic net model’s parameter estimates are something of a compromise between those estimated by ridge regression and those estimated by LASSO. The elastic net model’s parameters are more similar to those estimated by pure LASSO, however, because our tuned value of alpha was close to 1 (remember that when alpha equals 1, we get pure LASSO).

coefTibInts <- tibble(Coef = rownames(ridgeCoefs),
                  Ridge = as.vector(ridgeCoefs),
                  Lm = as.vector(lmCoefs))
coefUntidyInts <- gather(coefTibInts, key = Model, value = Beta, -Coef)

ggplot(coefUntidyInts, aes(reorder(Coef, Beta), Beta, fill = Model)) +
  geom_bar(stat = "identity", col = "black") +
  facet_wrap(~Model) +
  theme_bw()  +
  theme(legend.position = "none")

# The intercepts are different. The intercept isn't included when
# calculating the L2 norm, but is the value of the outcome when all
# the predictors are zero. Because ridge regression changes the parameter
# estimates of the predictors, the intercept changes as a result.

copy

This is an excerpt from Manning's book Machine Learning with TensorFlow. Login to get full access to this book.

Models may also have hyperparameters, which are extra ad hoc properties about a model. The term hyper in hyperparameter seems a bit strange at first. If it helps, a better name could be metaparameter, because the parameter is akin to metadata about the model.

When a line appears to fit some data points well, you might claim that your linear model performs well. But you could have tried out many possible slopes instead of choosing the value 2. The choice of slope is the parameter, and the equation containing the parameter is the model. Speaking in machine-learning terms, the equation of the best-fit curve comes from learning the parameters of a model.

As another example, the equation y = 3x is also a line, except with a steeper slope. You can replace that coefficient with any real number, let’s call it w, and the equation will still produce a line: y = wx. Figure 3.1 shows how changing the parameter w affects the model. The set of all equations you can generate this way is denoted as M = {y = wx | w ∈ }. This is read, “All equations y = wx such that w is a real number.”

Figure 3.1. Different values of the parameter w result in different linear equations. The set of all these linear equations is what constitutes the linear model M.

This is an excerpt from Manning's book Real-World Machine Learning. Login to get full access to this book.

Tuning the model parameters— ML algorithms are configured with parameters specific to the underlying algorithm, and the optimal value of these parameters often depends on the type and structure of the data. The value of each parameter, or any of them combined, can have an impact on the performance of the model. We introduce various ways to find and select the best parameter values, and show how this can help in determining the best algorithm for the dataset in question.

You’ve seen how cross-validation is used to test models and some of the performance metrics you can use to evaluate the results. For the simplest models, this is a matter of training, testing, and computing the appropriate performance metric(s). More-sophisticated algorithms have tuning parameters—knobs that can be turned by the user—that affect how they’re trained and applied. Each combination of settings yields a different mode. In the next section, you’ll see how sometimes a small adjustment can make a big difference in the results.

Listing 10.5. Random forest regression

from sklearn.ensemble import RandomForestRegressor from sklearn import cross_validation features = ['exposure', 'meanViewTime', 'nImps', 'reach', 'reachRate', #1 'vImps', 'vRate', 'vReach', 'vReachRate'] X_train, X_test, y_train, y_test = cross_validation.train_test_split( #2 df[features], df.CTR, test_size=0.40, random_state=0) reg = RandomForestRegressor(n_estimators=100, n_jobs=-1) #3 model = reg.fit(X_train, y_train) scores = cross_validation.cross_val_score(model, X_train, y_train) #4 print(scores, scores.mean()) ([ 0.62681533, 0.66944703, 0.63701492]), 0.64442575999999996) model.score(X_test, y_test) #5 0.6135074515145226 plt.rcParams["figure.figsize"] = [12.0, 4.0] plt.bar(range(len(features)), model.feature_importances_, align='center') _ = plt.xticks(range(len(features)), features) #1 - features are simple aggregates by pub #2 - Splits data into test and train, features and targets; trains on 60% of the data, holds out 40% for test #3 - Runs the random forest regression with 100 trees; n_jobs parameter tells RF to use all available cores #4 - Cross-validation splits training set to evaluate the model #5 - Runs the model on the test set

copy

This is an excerpt from Manning's book Mahout in Action. Login to get full access to this book.

Note how the final parameter to evaluate() is 0.05. This means only 5 percent of all the data is used for evaluation. This is purely for convenience; evaluation is a time-consuming process, and using this full data set could take hours to complete. For purposes of quickly evaluating changes, it’s convenient to reduce this value. But using too little data might compromise the accuracy of the evaluation results. The parameter 0.95 simply says to build a model to evaluate with 95 percent of the data, and then test with the remaining 5 percent.

For running k-means clustering, our mandatory list of parameters includes: