What you will learn in Grokking Machine Learning

by Luis Serrano

Read it on liveBook Get the book at manning.com

Chapter by chapter

Chapter 1: What is machine learning? It is common sense, except done by a computer

1.1 Do I need a heavy math and coding background to understand machine learning? · 1.2 OK, so what exactly is machine learning? · 1.3 How do we get machines to make decisions with data? The remember-formulate-predict framework · 1.4 Summary

What you learn in chapter 1 →

Chapter 2: Types of machine learning

2.1 What is the difference between labeled and unlabeled data? · 2.2 Supervised learning: The branch of machine learning that works with labeled data · 2.3 Unsupervised learning: The branch of machine learning that works with unlabeled data · 2.4 What is reinforcement learning? · 2.5 Summary · 2.6 Exercises

What you learn in chapter 2 →

Chapter 3: Drawing a line close to our points: Linear regression

3.1 The problem: We need to predict the price of a house · 3.2 The solution: Building a regression model for housing prices · 3.3 How to get the computer to draw this line: The linear regression algorithm · 3.4 How do we measure our results? The error function · 3.5 Real-life application: Using Turi Create to predict housing prices in India · 3.6 What if the data is not in a line? Polynomial regression · 3.7 Parameters and hyperparameters · 3.8 Applications of regression · 3.9 Summary · 3.10 Exercises

What you learn in chapter 3 →

Chapter 4: Optimizing the training process: Underfitting, overfitting, testing, and regularization

4.1 An example of underfitting and overfitting using polynomial regression · 4.2 How do we get the computer to pick the right model? By testing · 4.3 Where did we break the golden rule, and how do we fix it? The validation set · 4.4 A numerical way to decide how complex our model should be: The model complexity graph · 4.5 Another alternative to avoiding overfitting: Regularization · 4.6 Polynomial regression, testing, and regularization with Turi Create · 4.7 Summary · 4.8 Exercises

What you learn in chapter 4 →

Chapter 5: Using lines to split our points: The perceptron algorithm

5.1 The problem: We are on an alien planet, and we don’t know their language! · 5.2 How do we determine whether a classifier is good or bad? The error function · 5.3 How to find a good classifier? The perceptron algorithm · 5.4 Coding the perceptron algorithm · 5.5 Applications of the perceptron algorithm · 5.6 Summary · 5.7 Exercises

What you learn in chapter 5 →

Chapter 6: A continuous approach to splitting points: Logistic classifiers

6.1 Logistic classifiers: A continuous version of perceptron classifiers · 6.2 How to find a good logistic classifier? The logistic regression algorithm · 6.3 Coding the logistic regression algorithm · 6.4 Real-life application: Classifying IMDB reviews with Turi Create · 6.5 Classifying into multiple classes: The softmax function · 6.6 Summary · 6.7 Exercises

What you learn in chapter 6 →

Chapter 7: How do you measure classification models? Accuracy and its friends

7.1 Accuracy: How often is my model correct? · 7.2 How to fix the accuracy problem? Defining different types of errors and how to measure them · 7.3 A useful tool to evaluate our model: The receiver operating characteristic (ROC) curve · 7.4 Summary · 7.5 Exercises

What you learn in chapter 7 →

Chapter 8: Using probability to its maximum: The naive Bayes model

8.1 Sick or healthy? A story with Bayes’ theorem as the hero · 8.2 Use case: Spam-detection model · 8.3 Building a spam-detection model with real data · 8.4 Summary · 8.5 Exercises

What you learn in chapter 8 →

Chapter 9: Splitting data by asking questions: Decision trees

9.1 The problem: We need to recommend apps to users according to what they are likely to download · 9.2 The solution: Building an app-recommendation system · 9.3 Beyond questions like yes/no · 9.4 The graphical boundary of decision trees · 9.5 Real-life application: Modeling student admissions with Scikit-Learn · 9.6 Decision trees for regression · 9.7 Applications · 9.8 Summary · 9.9 Exercises

What you learn in chapter 9 →

Chapter 10: Combining building blocks to gain more power: Neural networks

10.1 Neural networks with an example: A more complicated alien planet · 10.2 Training neural networks · 10.3 Coding neural networks in Keras · 10.4 Neural networks for regression · 10.5 Other architectures for more complex datasets · 10.6 Summary · 10.7 Exercises

What you learn in chapter 10 →

Chapter 11: Finding boundaries with style: Support vector machines and the kernel method

11.1 Using a new error function to build better classifiers · 11.2 Coding support vector machines in Scikit-Learn · 11.3 Training SVMs with nonlinear boundaries: The kernel method · 11.4 Summary · 11.5 Exercises

What you learn in chapter 11 →

Chapter 12: Combining models to maximize results: Ensemble learning

12.1 With a little help from our friends · 12.2 Bagging: Joining some weak learners randomly to build a strong learner · 12.3 AdaBoost: Joining weak learners in a clever way to build a strong learner · 12.4 Gradient boosting: Using decision trees to build strong learners · 12.5 XGBoost: An extreme way to do gradient boosting · 12.6 Applications of ensemble methods · 12.7 Summary · 12.8 Exercises

What you learn in chapter 12 →

Chapter 13: Putting it all in practice: A real-life example of data engineering and machine learning

13.1 The Titanic dataset · 13.2 Cleaning up our dataset: Missing values and how to deal with them · 13.3 Feature engineering: Transforming the features in our dataset before training the models · 13.4 Training our models · 13.5 Tuning the hyperparameters to find the best model: Grid search · 13.6 Using K-fold cross-validation to reuse our data as training and validation · 13.7 Summary · 13.8 Exercises

What you learn in chapter 13 →

Concepts explained in this book

inputA property or characteristic of a data example that a model can use to make predictions.
accuracyA classification metric equal to the proportion of evaluated data points for which the predicted label matches the correct label.
classifierA supervised-learning technique that assigns an example to one or more discrete categories.
overfittingOverfitting is a modeling condition in which a model captures noise or idiosyncrasies in training data and generalizes poorly to new data.
linear regressionA regression technique that models a numerical target as a linear function of input features.
objective functionTraining error is a measure of a model's prediction error on the data used to train it.
parameterA numerical model parameter that multiplies a corresponding feature and determines its contribution to the prediction.
logistic regressionA perceptron-like model that outputs a continuous value between 0 and 1 rather than a discrete class label.
Gini impurityA decision-tree impurity measure that quantifies the heterogeneity of class labels within a node.
biasSystematic distortion in data or model behavior that can produce unfair or unrepresentative outcomes.
recallA classification metric equal to the proportion of positive-labeled data points that are correctly predicted as positive.
perceptronA processing unit or model that computes a weighted combination of inputs, adds a bias, and applies an activation function.
decision treeAn algorithm that recursively splits labeled data using feature tests and stopping conditions to construct a decision tree.
entropyEntropy is an information-theoretic impurity measure that is higher when class probabilities are more diverse and zero for a pure set.
labelA target feature attached to an example that identifies the value a supervised model should predict.
naive bayes classifierA probabilistic classification model that estimates class probabilities from observed features using the naive Bayes assumption.
cross entropy lossA multiclass classification loss that compares one-hot class labels with predicted class probabilities using cross-entropy.
epochOne iteration of a training loop that applies an update procedure to a selected training example.
interceptThe value of a linear function when its input features are zero, corresponding to where the line crosses the y-axis.
slopeThe rate at which a line's vertical coordinate changes relative to its horizontal coordinate, also called rise divided by run.
weak learnerA weak learner is a predictive model that performs only modestly better than chance but can contribute to a strong learner when combined with other weak learners.
hyperparameterA hyperparameter is a quantity set before training that controls how a machine-learning model is trained or specified.
classification scoreA signed numerical output of a linear classifier whose sign indicates the predicted class and whose magnitude indicates distance from the decision boundary.
leaf nodeA terminal decision-tree node with no outgoing branches that contains the resulting decision or prediction.
bayesian statisticsA probability identity that expresses a conditional probability in terms of a reversed conditional probability and the relevant prior probabilities.
f-scoreAn F-score is a classification metric that combines precision and recall into a single value.
precisionA classification metric measuring the proportion of examples predicted as positive that are actually positive.
thresholdA boundary value of a numerical feature that determines how data points are divided by a decision-tree split.
perceptron error functionAn error metric for a perceptron that assigns zero to correctly classified points and the absolute value of the perceptron's score to misclassified points.
probability theoryA numerical measure of the likelihood that an event occurs.
regularizationRegularization is a model-training technique that penalizes complexity while optimizing predictive performance.
posterior probabilityA posterior probability is the probability assigned to an event or hypothesis after incorporating observed evidence.
prior probabilityA prior probability is the initial probability assigned to an event or hypothesis before new evidence is considered.
kernel methodA machine-learning technique that maps data into a higher-dimensional feature space so linear methods can model nonlinear relationships.
perceptron algorithmA training algorithm that adjusts a perceptron's parameters to improve its predictions on labeled examples.
strong learnerA strong learner is an ensemble model formed by combining multiple weak learners to achieve substantially improved predictive performance.
recommender systemA recommendation system is a software system that suggests items, content, or actions to users based on available information about them or their context.
machine learningMachine learning is a field of artificial intelligence in which computers make decisions based on data rather than solely on explicitly specified rules.
gradient descentAn iterative optimization technique that changes parameters in the direction of greatest local decrease of an objective function.
stochastic gradient descentA gradient-descent variant that updates model parameters using one training data point at a time.
activation functionA function that transforms a model's score into an output or prediction.
learning rateA learning rate is a small positive training parameter that controls the size of parameter updates made during model optimization.
L1 regularizationL1 regularization adds the L1 norm of a model's non-bias coefficients as a complexity penalty to its regression error.
weight decayRidge regression is a linear regression technique that uses an L2 coefficient penalty, shrinking coefficients toward zero without usually making them zero.
decision stumpA shallow decision tree consisting of a single feature-based split followed by leaf predictions.
gradient boostingAn ensemble model that sequentially combines decision trees, with each tree improving the ensemble's fit to the data.
false negativeA false negative is a classification error in which a model predicts the negative class for a positive example.
false positiveA false positive is a classification error in which a model predicts the positive class for a negative example.
expressive powerModel complexity is the degree of flexibility or parameter magnitude that a model has for representing relationships in data.
polynomial kernelA kernel function that computes polynomial similarities between input examples and enables polynomial decision regions.
radial basis function kernelA kernel based on a radial function whose value depends on the distance between an input point and a center point.
kernel functionA mathematical function that represents an inner product in a transformed feature space, allowing a model to learn nonlinear decision boundaries.
backpropagationA neural-network training technique that computes loss derivatives with respect to model parameters by propagating error information backward through the network.
dropoutA neural-network regularization technique that randomly omits units during training to reduce co-adaptation and overfitting.
step functionA threshold function that maps an input to one of discrete constant values according to whether the input crosses a specified threshold.
root nodeThe top node of a decision tree associated with the complete input dataset before any split.
feature mapA mapping that sends data points from an original input space into another feature space, usually of higher dimension.
linearly separableThe property of a labeled dataset whose classes can be separated by a single linear boundary.
radial basis functionA function whose value depends only on the distance from its input to a specified center.
similarity metricA numerical measure of the closeness between two data points.
conditional probabilityThe probability of an event given that another event or condition is known to occur.
l2 normThe L2 norm of a vector is the square root of the sum of the squared values of its components.
decision nodeAn internal decision-tree node that represents a yes-or-no question and has two outgoing branches.
sentiment analysis classifierA classifier that predicts the affective polarity or sentiment category expressed by text.
specificityA classification metric equal to the proportion of actual negative examples correctly identified, also called the true negative rate.
ham emailAn email that is not spam or otherwise unwanted.
spamAn unwanted email, including unsolicited promotions or chain letters.
sigmoidA function that maps a real-valued input to a value between zero and one, commonly interpreted as a binary class probability.
ReLUAn activation function that returns zero for negative inputs and the input itself for nonnegative inputs.
confusion matrixA tabular representation of classification outcomes divided into true-positive, true-negative, false-positive, and false-negative counts.
datasetA structured collection of data points intended for analysis or machine-learning tasks.
perceptron trickA perceptron update technique that adjusts weights and bias according to the signed difference between a point's label and prediction.
absolute trickA book-specific line-adjustment technique that updates a model's parameters with sign-based corrections derived from whether a point lies above or below the line.
square trickA book-specific line-adjustment technique that updates a model's slope and intercept using the signed prediction error and a feature value.
hidden layerA neural-network layer positioned between the input and output layers whose units transform intermediate representations.
receiver operating characteristicA receiver operating characteristic curve is a plot of classifier sensitivity against its false-positive rate across decision thresholds.
stopping criterionA criterion that prevents a decision-tree node from being split further.
training datasetThe portion of a dataset used to fit a machine-learning model.
AdaBoostAn adaptive boosting algorithm that combines sequentially trained weak learners by weighting their contributions according to predictive performance.
independenceA probabilistic relationship in which the occurrence of one event does not change the probability of another event.
evaluation datasetA held-out subset of labeled data used to compare models or training configurations during development.
area under the curveA scalar performance metric equal to the area under a receiver operating characteristic curve.
root mean squared errorA regression metric equal to the square root of the average squared difference between observed and predicted target values.
model architectureThe structural specification of a neural network, including its layers, nodes, and activation functions.
regularization parameterA training hyperparameter that controls the strength of the complexity penalty relative to predictive error.
mean squared errorA regression error metric equal to the average squared difference between observed and predicted target values.
unlabeled dataData whose observations do not have associated target labels or outcome annotations.
unsupervised learningA machine-learning approach in which a model discovers structure or patterns from data without target outputs.
input layerThe neural-network layer that receives the input features or input representation.
output layerThe neural-network layer that produces the model's final outputs or predictions.
underfittingA model whose complexity is insufficient to capture the relevant structure of its dataset.
normalizationA mathematical operation that rescales nonnegative quantities by their sum so that the resulting quantities sum to one.
multiplication ruleA rule stating that the probability of independent events occurring together equals the product of their individual probabilities.
categorical dataData whose values belong to a finite set of named categories or states.
vanishing gradient problemA neural-network training problem in which gradients become extremely small as they are propagated through layers, slowing or preventing learning.
grid searchA model-selection technique that evaluates an exhaustive set of specified hyperparameter combinations.
boostingAn ensemble technique that trains learners sequentially so later learners focus on weaknesses of earlier learners and combines their predictions.
eventIn Bayesian reasoning, an event is an occurrence or observation that provides information for updating a probability.
categorical encodingA data-encoding technique that represents each category of a categorical feature with a separate binary indicator feature.
XGBoostAn open-source software library that implements optimized gradient-boosted decision trees for machine-learning tasks.
support vector machineA model that separates classes or fits targets using an optimized margin and support vectors.
softmaxA function that converts multiple real-valued scores into positive values whose normalized values sum to one and represent class probabilities.
logistic trickAn iterative logistic-regression parameter-update technique that adjusts weights and bias using prediction error, features, and a learning rate.
preprocessingData preprocessing is the transformation of raw data into a form suitable for computational analysis or model training.
tanhA smooth S-shaped activation function that maps real-valued inputs to values between negative one and one.
branchA decision-tree path segment connecting a node to a child node or resulting subtree.
random forestAn ensemble model that combines predictions from many decision trees, typically trained with randomized data or feature selection.
depthThe number of neural-network layers excluding the input layer.
linear layerA neural-network architecture in which every node in a layer connects to every non-bias node in the next layer, with no connections between nonconsecutive layers.
feature selectionFeature selection is a technique that chooses relevant input features and removes unnecessary or harmful ones before model training.
labeled dataData containing a designated target feature or label that a model is intended to predict.
supervised learningA machine-learning approach that learns from labeled data to predict labels for previously unseen data.
clusteringAn algorithm that partitions data points into groups based on similarity.
dimensionality reductionA technique that simplifies data by reducing or combining features while preserving as much information as possible.
binary treeA tree data structure in which each node has at most two child branches.
nodeA structural element of a tree that can contain a question or prediction and connect to branches.
gradientA vector of partial derivatives that indicates how a multivariable function changes with respect to its parameters.
information gainInformation gain is the reduction or change in entropy resulting from a data split.
absolute errorA regression error metric equal to the sum of the absolute differences between observed and predicted target values.
mean absolute errorA regression error metric equal to the average absolute difference between observed and predicted target values.
squared lossA regression error metric equal to the sum of squared differences between observed and predicted target values.
remember-formulate-predict frameworkThe remember-formulate-predict framework is a machine-learning workflow that examines previous data, builds a model from it, and applies the model to future data.
similarity scoreA similarity score is a metric defined here as the squared sum of a set's elements divided by the number of elements, optionally regularized by a denominator parameter.
tree pruningTree pruning is the removal of decision-tree branches that fail a specified split-quality condition.
hyperplaneA flat geometric subspace of one dimension less than its surrounding space that can separate points in higher-dimensional space.
image recognitionThe computational task of identifying or classifying objects, patterns, or other content in images.
Turi CreateAn open-source library for building machine-learning models through high-level interfaces.
discretizationBinning is a technique that partitions numerical values into intervals or buckets so they can be represented as categorical groups.
c parameterA support-vector-machine hyperparameter that controls the trade-off between a wide margin and errors on training examples.
testThe evaluation of a trained model on data used to assess its predictive performance.
generative adversarial networkA generative adversarial network is a generative model composed of competing generator and discriminator networks that learn to produce realistic data.
residual errorA residual is the difference between an observed target label and a model's predicted target value.
weighted majority voteA prediction aggregation rule in which each learner's vote contributes according to an assigned weight.
baggingAn ensemble technique that trains successive learners on random bootstrap subsets of data and aggregates their predictions.
learner scoreA learner score is a numerical coefficient that determines how strongly a weak learner contributes to an ensemble prediction.
oddsOdds are the ratio of the probability of an event occurring to the probability of it not occurring.
rescaling factorA rescaling factor is a multiplicative ratio used to increase the weights of incorrectly classified data points in the next boosting round.
polynomial regressionA regression model whose predicted outcome is expressed as a polynomial function of its input features.
sentiment analysisThe computational task of determining an expressed opinion or affective polarity in text.
test errorTesting error is a measure of a model's prediction error on data held out from training for evaluation.
classification error functionAn error metric for a two-boundary classifier equal to the sum of the perceptron errors from its two boundary lines.
distance error functionAn error metric for parallel classifier boundaries equal to the sum of the squared weights, which penalizes boundaries that are close together.
compile()A Keras model method that configures a model's loss function, optimizer, and evaluation metrics before training.
KerasAn open-source deep-learning library providing high-level APIs for constructing and training neural networks.
optimizationAn optimization algorithm that updates model parameters to minimize a loss or objective function.
tensor reshapingAn array operation that changes the dimensions of stored values without changing their underlying elements.
model complexity graphA plot of model complexity against training and validation error for comparing candidate models and identifying an appropriate complexity level.
SFrameA tabular data structure in Turi Create for storing and manipulating rows and columns of data.
validationA model-development procedure that evaluates candidate models or settings on held-out data without using the final test data for those decisions.
pandasPandas is an open-source Python library for manipulating and analyzing structured, especially tabular, data.
fully connected neural networkA neural-network layer in which each unit receives input from all units in the preceding layer.
reinforcement learningA machine-learning approach in which an agent learns to choose actions by interacting with an environment and receiving rewards or punishments.
matrix factorizationAn algorithmic technique that represents a matrix through products of lower-dimensional matrices to reduce its row and column dimensions.
singular value decompositionA matrix decomposition that expresses a matrix as the product of orthogonal matrices and a diagonal matrix of singular values.
nonlinear classifierA classifier whose decision boundary is not representable as a single affine linear function of the input features.
data cleaningData cleaning is the process of detecting, correcting, removing, or imputing invalid, inconsistent, or missing data.
scikit-learnAn open-source Python library providing classical machine-learning algorithms and utilities.
data frameA DataFrame is a two-dimensional labeled tabular data structure provided by Pandas for storing and manipulating heterogeneous columns.
unseen dataData whose examples were not included in a model's training experience.
golden ruleThe model-evaluation rule that testing data must not be used to train a model or make decisions about its model or hyperparameters.
feedforward networkA neural network composed of multiple layers of perceptron-like processing units.
greedy tree constructionA decision-tree construction strategy that selects the locally best feature or split at each node without guaranteeing a globally best tree.
missing valueA missing value is an absent, unreadable, or otherwise unavailable entry in a dataset.
convolutional layerA convolutional layer is a neural-network layer that produces a filtered representation by applying convolutions across an input.
convolutional neural networkA neural-network architecture that uses convolutional operations to learn localized and hierarchical patterns, especially in structured data such as images.
discriminatorA discriminator is the component of a generative adversarial network that distinguishes real data from synthetic data.
generatorA generator is the component of a generative adversarial network that produces synthetic data intended to resemble real data.
GRUA gated recurrent unit is a recurrent neural-network unit that uses gates to control the retention and update of sequence state.
LSTMA long short-term memory network is a recurrent neural network architecture with gated memory cells designed to learn long-range dependencies in sequences.
natural language processingNatural language processing is the computational analysis, transformation, or generation of human language.
recurrent neural networkA recurrent neural network is a neural network architecture that processes sequences by retaining and updating state across successive inputs.
transfer learningTransfer learning is a technique that adapts a pretrained model to a new task or dataset by continuing its training, often by modifying only some parameters.
DecisionTreeRegressorA scikit-learn estimator class that fits regression decision trees to numerical targets.
non-binary categorical featureA categorical feature whose possible values include more than two distinct classes.
weighted average Gini impurityA split-quality measure equal to the size-weighted average of the Gini impurity indices of the resulting subsets.
artificial intelligenceArtificial intelligence is the field concerned with computer systems making decisions or solving problems autonomously.
modeA prediction aggregation rule that selects the class receiving the greatest number of learner votes.
fillnafillna is a Pandas operation that replaces missing entries with a specified value or computed replacement.
is_nais_na is a Pandas operation that identifies missing or null entries in a data structure.
medianThe median is the middle value of an ordered set of values, or the average of the two middle values when the set has even size.
nanNaN is a special marker representing a value that is missing, undefined, or not representable as a numeric value.
cross-validationA resampling technique that partitions a dataset into k portions and repeatedly trains on k−1 portions while validating on the remaining portion, then averages the validation scores.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.