liveBookmanning.com

What you will learn in Outlier Detection in Python

by Brett Kennedy

Read it on liveBook Get the book at manning.com

Chapter by chapter

Concepts explained in this book

• outlier detectionAn outlier detection system is a system that identifies unusually divergent items in data and may provide scores or decisions about them.
• classifierThe assignment of an input observation to one of a set of predefined categories.
• ensembleAn ensemble is a combination of multiple models or detectors whose outputs are combined to obtain more robust results than an individual model or detector.
• anomalyAn extreme value is an observation far from the typical level of a time series, either unusually small or unusually large.
• anomaly scoreA numerical estimate of how unusual an item is relative to a specified reference group.
• signalSignal is variation in data regarded as meaningful or relevant within a particular context.
• synthetic dataArtificially generated data designed to have controlled properties rather than being collected from real-world observations.
• embeddingAn embedding is a numerical representation of an item in a continuous vector space that preserves selected properties or relationships of the original data.
• histogramA histogram is a graphical representation that partitions numerical values into intervals and displays the frequency or density in each interval.
• ruleA rule is an explicitly specified condition or procedure that classifies or responds to data according to predetermined criteria.
• collective outlier detectionA collective outlier test identifies unusual collections of records by analyzing aggregate, temporal, or relational patterns rather than only individual rows.
• labeled dataLabeled data consists of data items accompanied by target labels or annotations used for supervised learning.
• surrogate modelA proxy model is an approximate model that imitates the behavior of another model or detector for analysis or explanation.
• reconstruction lossReconstruction error is a measure of the discrepancy between an observation and the output reconstructed from its encoded or generated representation.
• kernel density estimateA kernel density estimate is a nonparametric probability-density estimate formed by smoothing contributions from sample observations.
• subspaceA subspace is a space formed by considering only a subset of the dimensions or features of a larger space.
• interpretabilityExplainable artificial intelligence (XAI) comprises methods and systems that make an artificial-intelligence model's predictions or decisions understandable to people.
• feature scalingA preprocessing technique that transforms feature values so their numerical ranges or dispersions are comparable.
• autoencoderAn autoencoder is a neural network trained to encode an input into a representation and reconstruct the original input.
• clusteringAn unsupervised learning technique that divides items into groups whose members are more similar to one another than to members of other groups.
• self-supervised learningSelf-supervised learning is a machine-learning technique that derives training targets from the data itself rather than requiring manually assigned labels.
• parameterA configurable value that controls the behavior or fitting of a model or algorithm.
• KNN detectorA nearest-neighbor outlier detector that scores observations using distances or relationships to their k closest neighbors.
• Gaussian mixture modelA Gaussian mixture model represents a probability distribution as a weighted combination of multiple Gaussian component distributions.
• local outlier factorLocal outlier factor is a density-based outlier detection algorithm that compares an observation's local density with the local densities of neighboring observations.
• decision treeA decision tree is a hierarchical predictive model that recursively partitions observations using feature-based decision rules.
• training datasetData supplied to a machine-learning model or algorithm for fitting its parameters or learning patterns.
• univariate statistical testA univariate statistical test evaluates one feature or variable at a time to assess a statistical property or detect unusual values.
• pseudo-ground truthPseudo-ground truth is an approximate set of labels considered reliable enough for evaluation or ensemble construction despite possible labeling errors.
• isolation forestIsolation Forest is an ensemble outlier-detection algorithm that identifies observations that are isolated quickly by random partitioning.
• point anomalyA point anomaly is an individual time-series observation whose value is unusual relative to the surrounding or expected values.
• local outlierAn internal outlier is an unusual data item located within the overall range of a multimodal distribution or a sparse gap between dense regions.
• semi-supervised learningSemisupervised learning is a machine-learning paradigm that trains with labeled data while also exploiting substantially larger amounts of unlabeled data.
• simulationA simulation is a computational representation of a process or environment that generates observations under specified rules and conditions.
• labelA value associated with a data item that identifies or describes its target category or numeric outcome.
• thresholdA threshold is a boundary value beyond which a monitored count or measurement triggers a specified response.
• feature engineeringFeature engineering is the process of modifying or combining original data attributes to create inputs that are more useful for analysis or modeling.
• curse of dimensionalityThe curse of dimensionality is a set of difficulties caused by increasing dimensionality, including data sparsity, less informative distances, and rapidly increasing computational demands.
• one-class svmOneClassSVM is a scikit-learn estimator that learns a boundary around training data to identify observations outside the learned region.
• language modelA large language model is a neural model trained on very large text corpora to process and generate natural language.
• false positiveA prediction that an issue exists when no issue is present.
• covariance matrixA covariance matrix is a square matrix containing the variances of variables on its diagonal and their pairwise covariances elsewhere.
• nearest-neighbor measurementThe kth-nearest-neighbor metric is the distance from an observation to its kth closest other observation.
• probability distributionA distribution describes how values or observations are apportioned across possible outcomes or regions of a data space.
• variational autoencoderA variational autoencoder is a generative neural-network model that learns a probabilistic latent representation and reconstructs or generates observations from it.
• false negativeA prediction that no issue exists when an issue is present.
• out-of-distributionData containing observations whose distribution differs from the distribution represented by a model's training data.
• deep learningDeep learning is machine learning based on neural networks with multiple computational layers that learn hierarchical representations.
• dimensionality reductionDimensionality reduction is the transformation of data into fewer dimensions while retaining relevant information.
• categorical dataCategorical data consists of values drawn from discrete categories rather than measured on a continuous numerical scale.
• importance scoreFeature importance is a quantitative indication of how much an input feature contributes to a model’s prediction, score, or decision.
• generative adversarial networkA generative adversarial network is a model consisting of a generator and discriminator trained competitively so the generator produces data resembling a reference distribution.
• principal component analysisPrincipal component analysis is a linear dimensionality-reduction technique that represents data using orthogonal directions of decreasing variance.
• inlierA data item that falls within the range regarded as consistent with the main population under an outlier criterion.
• data dopingData doping is the deliberate injection or alteration of observations in a dataset to test a detector under controlled conditions.
• lagA time lag is a temporal offset by which one time series follows or precedes another.
• PRISMA rule-discovery algorithm that constructs ordered class-specific rules according to coverage and support criteria.
• encoderAn encoder is the component of an autoencoder that transforms an input into a compressed representation.
• alibi-detectAn open-source Python library for detecting data drift, outliers, and other distributional or prediction anomalies across multiple data modalities.
• cluster-based outlier detectionAn outlier-detection approach that applies clustering to identify observations that are unusual relative to their cluster structure.
• k-nearest neighborsA k-nearest-neighbors algorithm scores a record using its average distance to its k nearest neighboring records.
• expected outlierA known outlier is an outlier belonging to a type of record that is expected or can be anticipated, even if it is rare.
• quantitative dataA numeric feature is a data attribute whose values are numerical quantities suitable for quantitative comparison and computation.
• centroidA cluster center is a representative location or point associated with a cluster and used to measure observations' proximity to that cluster.
• PyODPyOD is an open-source Python library dedicated to detecting anomalous or outlier observations with a collection of detection algorithms.
• ball treeBallTree is a tree-based data structure for efficient nearest-neighbor and distance queries in metric spaces.
• data segmentationData segmentation is the division of data items into groups intended to contain similar items for analysis or comparison.
• Spearman's rank correlation coefficientSpearman correlation is a rank-based measure of the strength and direction of a monotonic association between two variables.
• missing valueThe absence of an expected value or observation in a dataset.
• meanA mean is a measure of central tendency calculated as the arithmetic average of numerical observations.
• one-dimensional outlierA univariate outlier is a data item with a value that is unusual when considered individually.
• specific outlierA specific outlier, or specific anomaly, is a data item matching a predefined object, event, error, or condition of interest, whether or not it is statistically unusual.
• one-hot encodingOne-hot encoding represents each category of a categorical feature as a separate binary indicator feature.
• categorical outlier detectorAn outlier-detection model that analyzes features whose values belong to discrete categories.
• DBSCANDBSCAN is a density-based clustering algorithm that forms clusters from regions of sufficiently dense observations and can leave some observations unclustered.
• discretizationBinning is the process of assigning continuous or otherwise detailed values to discrete intervals or categories.
• correlationCorrelation is a measure of statistical association between paired numerical quantities, such as outputs from different detectors.
• normal distributionA normal, or Gaussian, distribution is a continuous probability distribution with a symmetric bell-shaped density determined by its mean and variance.
• machine learningMachine learning is the computational approach of learning patterns or predictive relationships from data rather than specifying all rules explicitly.
• distanceA function that quantifies dissimilarity or separation between observations in a data space.
• visualizationA visualization is a visual representation of data designed to reveal distributions, relationships, or unusual observations.
• random forestRandom forest is an ensemble learning method that combines predictions from multiple decision trees built using randomized data and feature selection.
• time seriesA time series is a sequence of observations indexed by or ordered according to time.
• evaluationThe assessment of a model or system against known outcomes, criteria, or reference results.
• mahalanobis distancemahalanobis() is an estimator method that computes Mahalanobis distances for observations.
• manual evaluationManual inspection is the direct review of individual data records or flagged observations to identify errors, categories, or meaningful patterns.
• forecastingThe process of predicting future values or events from historical or current data.
• pretext taskA pretext task is an automatically constructed training task whose solution helps a model learn transferable representations without manual target labels.
• bucketA bin is an interval or category into which observations are grouped for a histogram or related distribution summary.
• one-class methodA one-class method is an approach trained on data from a single class to characterize that class and identify observations that do not fit it.
• DeepODDeepOD is a Python library providing unsupervised and semisupervised outlier-detection algorithms for tabular and time-series data.
• marginal probabilityA marginal distribution describes the probability behavior of one variable after accounting for or aggregating over the other variables.
• trainingThe process of fitting a model to data so that it learns parameters or patterns.
• reference datasetA reference dataset is a collection of observations used as a basis for comparing other observations or assessing what is typical.
• unsupervised learningMachine learning that extracts structure, information, or patterns from data without labeled target values.
• joint probability distributionA joint distribution describes the combined distribution of two or more variables and their dependence across values.
• classifier-adjusted density estimationClassifier-adjusted density estimation is a synthetic-data outlier-detection technique that trains a classifier to distinguish observed data from samples generated from an estimated reference distribution.
• median absolute deviationMedian absolute deviation is a robust measure of statistical spread equal to the median of the absolute deviations from the data's median.
• indicator variableA data column whose values indicate membership using two possible states, commonly 0 and 1.
• latent spaceA latent space is a lower-level representation space whose coordinates encode hidden factors or inputs associated with observations.
• extreme outlierAn unexpected outlier is an extreme data value considered beyond what the underlying phenomenon is normally expected to produce.
• counterfactual explanationA counterfactual explanation is a hypothetical minimally changed input whose model outcome differs from that of the original input.
• area under the curveAUROC is the area under the receiver operating characteristic curve, measuring a scoring model's ability to rank positive cases above negative cases across classification thresholds.
• generative modelingA generative model represents a process or probability distribution capable of assigning likelihoods to observations or generating new observations.
• z-scoreA z-score is the signed number of standard deviations by which a numeric value differs from the mean of a reference set.
• distance-based algorithmAn outlier-detection approach that identifies unusual observations using distances between data points.
• ECODAn outlier detection algorithm that estimates feature-wise tail extremeness from empirical cumulative distributions and aggregates it into an outlier score.
• maximumAn aggregation method that assigns a record an aggregate score equal to its highest detector score.
• structured dataTabular data is data organized into rows and columns, typically representing records and their attributes.
• CountsOutlierDetectorCountsOutlierDetector is a multidimensional histogram-based outlier detector that evaluates records across feature subspaces of limited dimensionality.
• outlier processAn outlier process is a simulated generating mechanism that operates infrequently and produces records that are rare and significantly different from normal records.
• elliptic envelopeEllipticEnvelope is a scikit-learn estimator that detects outliers by fitting a robust covariance-based elliptical model to the data.
• maskingMasking is an outlier effect in which extreme observations cause other outliers to appear insufficiently unusual for detection.
• swampingSwamping is an outlier effect in which extreme observations cause otherwise typical observations to be incorrectly identified as outliers.
• contextual anomalyA contextual outlier is an observation that is anomalous only relative to its temporal or contextual conditions.
• decoderA decoder is a model component that transforms a learned representation back into an output representation or reconstruction.
• multivariate outlierOutlier detection applied jointly to multiple features represented in tabular data.
• trendA trend is a systematic directional change in observations across an ordered sequence, especially over time.
• principal componentA principal component is an orthogonal direction representing a decreasing amount of variance in data after successive components are selected.
• PyCaretAn open-source Python library that provides low-code interfaces for machine learning workflows, including preprocessing, modeling, clustering, time-series analysis, and outlier detection.
• samplingSampling is the selection of a subset of observations from a larger dataset for analysis or modeling.
• accuracyAccuracy is a metric measuring the proportion of a model's predictions that match the correct outcomes.
• arimaARIMA is a time-series forecasting method that models a value using autoregression, differencing, and moving-average error terms.
• DartsDarts is a Python library for time-series forecasting and related model evaluation and manipulation tasks.
• forecast errorA forecast error is the difference between an observed value and the value predicted for the same time point.
• mean absolute percentage errorMean absolute percentage error is the average absolute forecast error expressed as a proportion of the corresponding actual value.
• ODINODIN (outlier detection using in-degree number) is an outlier-detection algorithm that scores observations by their incoming nearest-neighbor relationships.
• supervised learningA machine learning paradigm in which models learn from data accompanied by labels.
• isolation treeAn isolation tree is a tree that recursively partitions feature space to isolate observations within an isolation forest.
• label encodingOrdinal encoding is a categorical encoding technique that maps discrete categories to integer codes representing an order.
• domain knowledgeDomain knowledge is application-specific knowledge about entities, processes, relationships, and conditions relevant to data analysis.
• random subspacesRandom subspace selection is a technique for constructing feature subsets by choosing dimensions randomly.
• sparse regionAn area in a two-dimensional plane containing relatively few observations compared with surrounding areas.
• histogram-based outlier scoreHistogram-based outlier score (HBOS) is an outlier-detection algorithm that scores observations by combining the rarity of their values in independently binned feature distributions.
• z-score testA z-score test is an outlier-detection procedure that flags values whose absolute z-scores exceed a selected cutoff.
• statistical outlierA statistical outlier is a data record that differs statistically from the majority of records in a dataset.
• BICThe Bayesian information criterion evaluates a statistical model by balancing goodness of fit against model complexity.
• parametric modelA parametric model assumes a specified structural form and estimates a finite set of parameters describing that form.
• frequent item setA frequent item set is a value or combination of values that occurs together in a substantial fraction of records.
• myth of averageThe myth of average is the idea that objects or observations can be typical in each individual dimension without any one being typical across all dimensions.
• entropy-based outlier detectionEntropy-based outlier detection identifies observations whose removal produces unusually large reductions in the combined entropy of feature values.
• preprocessingThe preparation and transformation of raw data into a form suitable for analysis or machine-learning models.
• hyperparameterA hyperparameter is a configuration value set before model training or algorithm execution that controls the model or algorithm's behavior.
• density-based outlier detectionA density-based method identifies unusual observations by comparing the density around observations or regions of the data.
• k-d treeKDTree is a binary space-partitioning tree that recursively divides points by coordinate dimensions to support nearest-neighbor and range queries.
• Real versus Fake detectorA book-specific outlier-detection approach that uses classification to distinguish real observations from synthetically generated observations.
• contamination parameterA contamination parameter specifies the expected proportion of outliers in a dataset, often to determine an outlier-classification threshold.
• data artifactA data artifact is an unusual recorded value caused by the data-collection process rather than by the phenomenon being monitored.
• Apriori algorithmThe Apriori algorithm discovers frequent item sets by progressively extending item combinations that satisfy a minimum support criterion.
• frequent item set-based outlier detectionFrequent pattern outlier factor is an outlier-detection method that scores records from the number and support of frequent item sets they contain.
• minimum supportMinimum support is the lowest fraction of records in which an item set must occur to qualify as frequent.
• interquartile rangeThe interquartile range is a measure of statistical dispersion equal to the difference between the third and first quartiles of a distribution.
• topic modelingTopic modeling is an unsupervised technique for inferring recurring semantic topics and their distributions in a collection of documents.
• statisticsA data-analysis technique that uses statistical properties such as location and dispersion to identify unusual values.
• entropyEntropy is a measure of uncertainty or information content in a probability distribution.
• local explanationA local explanation describes why a model produced a particular prediction or score for an individual observation.
• time series analysisTime-series analysis is the analysis of observations ordered over time to characterize trends, cycles, dependencies, or unusual temporal behavior.
• empirical cumulative distribution functionA function that assigns each value the proportion of observed data points less than or equal to it.
• tail probabilityA tail probability is the probability that a random variable takes a value at least as extreme as a specified boundary in a distribution's tail.
• high-dimensional dataA high-dimensional dataset is a dataset whose observations are represented by a large number of feature dimensions.
• expectation maximizationExpectation maximization is an iterative algorithm for estimating parameters in models with latent or incomplete assignments by alternating parameter evaluation and adjustment.
• soft clusteringSoft clustering assigns each observation a degree or score of membership for each cluster instead of assigning it to only one cluster.
• unlabeled dataData whose instances do not have assigned target labels or annotations.
• prediction averagingScore averaging is the aggregation of multiple detector scores or scores obtained under multiple parameter values by computing their mean.
• min-max normalizationA feature-scaling transformation that linearly maps values to a specified interval, commonly from zero to one.
• RobustScalerRobust scaling transforms feature values using robust location and spread statistics, typically the median and interquartile range, to reduce sensitivity to extreme values.
• data aggregationAggregation is the process of combining multiple observations into summary values according to specified grouping dimensions and time intervals.
• common apiA common API is a shared interface that exposes consistent operations across different detector implementations.
• decision_functiondecision_function is an estimator method that returns a numeric decision or anomaly score for each supplied record.
• decision_scores_decision_scores_ is a PyOD estimator attribute containing numeric outlier scores for the training records.
• fit methodA model-fitting method that estimates a model’s parameters or structure from supplied data.
• labels_labels_ is a fitted PyOD estimator attribute containing binary inlier or outlier labels for observations.
• predict_probapredict_proba is an estimator method that returns normalized class-membership scores for each supplied record.
• radius detectorA radius detector identifies or scores observations according to the number or arrangement of neighboring observations within a specified distance radius.
• cluster-based local outlier factorCluster-based local outlier factor is an outlier detection algorithm that scores observations using cluster membership, cluster size, and distance to relevant clusters.
• data annotationLabeling is the process of assigning annotations or target values to data records.
• retrainingRetraining is the process of fitting a model again using new, updated, or expanded training data.
• training timeTraining time is the duration required to fit a model or detector to data.
• digital imageImage data is visual data represented as arrays of pixel values or derived visual features.
• cellA cell is an interval-defined region of a feature subspace containing records with corresponding binned values.
• feature baggingFeature bagging is an ensemble technique that trains multiple detectors on randomly selected feature subsets and combines their outputs.
• scikit-learnscikit-learn is an open-source Python machine-learning library providing estimators, preprocessing utilities, model-selection tools, and related functionality.
• feature associationA feature association is a statistical relationship or dependence between feature values across observations.
• jitterJitter is a small perturbation added to values to spread overlapping observations or avoid repeated discrete outputs.
• multivariate testingA multivariate test evaluates multiple features jointly rather than assessing each feature independently.
• prediction timePrediction time is the duration required for a trained model or detector to produce outputs for input records.
• association rule miningAn association rule is an implication describing how the presence of one set of categorical values is associated with another set of values.
• rolling windowA rolling window is a selection of the most recent fixed-size or fixed-duration observations used for analysis.
• varianceVariance is a statistical measure of dispersion equal to the average squared deviation from the mean.
• data frameA dataframe is a tabular data structure with labeled rows and columns for storing and manipulating heterogeneous data.
• noiseVariation in data regarded as irrelevant or undesired within a particular context.
• numeric detectorA numeric detector is an outlier detector designed to analyze numerical features or numeric data.
• KDD Cup datasetThe KDD Cup dataset is a large labeled network-connection dataset containing normal records and records associated with network attacks for intrusion and outlier detection research.
• global explanationA global explanation describes how an outlier detector behaves across the dataset, including the patterns it identifies and how it assigns scores.
• approximate nearest neighbor algorithmA search technique that finds points near a query point with lower computational cost than exact nearest-neighbor search by allowing limited approximation.
• Manhattan distanceManhattan distance is the sum of the absolute coordinate-wise differences between two points.
• fraudFinancial fraud is the deliberate deception or misrepresentation of financial activities, records, or positions to obtain an improper benefit.
• decision boundaryA decision boundary is a surface in feature space that separates regions assigned different model decisions or classes.
• missing combinationA missing combination is an absent occurrence of a feature-value combination that is expected or possible in a dataset.
• interpretable modelAn interpretable model is a model whose decision process or outputs can be understood by people in terms of meaningful inputs and relationships.
• proxy taskA proxy task is an auxiliary prediction task created to train a model to learn useful representations rather than to solve the task of ultimate interest.
• representation learningRepresentation learning is the process of learning useful numerical representations of objects from data.
• rare featureA categorical value that occurs substantially less often than other values in the same feature.
• data cleaningData cleaning is the process of identifying and correcting, removing, or excluding inaccurate, corrupted, irrelevant, or anomalous data.
• stacked ensembleA predictive model that learns from the outputs of multiple component models or detectors, often using those outputs as input features.
• feature selectionFeature selection is the process of choosing a subset of input features that are most useful for a specified analysis or prediction task.
• percentileA percentile is a value or rank indicating the proportion of observations at or below a specified point in an ordered dataset.
• count encodingCount encoding transforms each categorical value into the frequency with which that value occurs in its column.
• consensus votingDetector consensus is an agreement-based method that derives labels from the combined decisions of multiple outlier detectors.
• shared neighbors distanceShared nearest neighbors is a neighborhood-similarity method that measures similarity by the overlap between observations' nearest-neighbor sets.
• clustering assumptionThe classification assumption that each class occupies one cluster or a small number of localized clusters in feature space.
• unclassified outlierAn outlier that is not assigned to any recognized category of useful or known outliers.
• smoothness assumptionThe classification assumption that nearby records in the input space tend to belong to the same class.
• categorical encodingCategorical encoding is the transformation of categorical values into numerical representations suitable for computational analysis.
• retrieval rankingAn ordering of observations from lowest to highest or highest to lowest according to their values.
• z-score normalizationZ-score scaling transforms values by subtracting the mean and dividing by the standard deviation.
• cross-sectional dataCross-sectional data consists of observations on multiple entities without a temporal sequence being the primary organizing dimension.
• panel dataPanel data records multiple entities at multiple points in time, combining cross-sectional and temporal dimensions.
• GoodhartingThe principle that a measure ceases to be a reliable measure when it becomes the target of optimization.
• DML detectorA DML detector is an outlier detector based on a DML modeling approach and capable of using internally trained classifiers.
• contrastive learningA representation-learning technique that brings embeddings of similar items closer together and separates embeddings of dissimilar items.
• adversarially learned anomaly detectionAdversarially learned anomaly detection is a GAN-based outlier-detection method using a bidirectional GAN architecture and reconstruction error scoring.
• AnoGANAnoGAN is a GAN-based anomaly-detection method that estimates a test observation's latent representation and combines reconstruction discrepancy with discriminator output.
• bidirectional generative adversarial networkA bidirectional generative adversarial network is a GAN augmented with an encoder that maps observations back into the generator's latent representation.
• discriminatorA discriminator is the component of a generative adversarial network that distinguishes real observations from generated observations.
• generatorA generator is the component of a generative adversarial network that produces synthetic outputs from input noise or latent representations.
• normalized scoreA unitless value obtained by scaling an observed count relative to a reference count or expected count.
• association rule-based outlier detectionAn outlier-detection technique that identifies records by evaluating whether categorical associations expected from their present item sets are violated.
• spaCyspaCy is an open-source Python library for efficient natural language processing and linguistic feature extraction.
• industrial processAn industrial process is a production or operational procedure involving machinery, materials, and monitored process conditions.
• sensor dataA sensor reading is a measured value produced by a sensor monitoring a physical condition or process.
• rank transformationA rank transformation replaces numerical values with their order ranks while preserving ordering and altering distances between values.
• DaskDask is a Python library for parallel and distributed computing with larger-than-memory and partitioned data.
• H2OH2O is a software platform for scalable machine learning and predictive analytics on large datasets.
• HNSWHNSW is a graph-based approximate nearest-neighbor data structure that organizes navigable connections between data points.
• SparkApache Spark is an open-source distributed computing engine for processing and analyzing large datasets.
• black boxA black-box model is a predictive model whose internal decision process is not readily understandable from its inputs and outputs.
• importance scoreA numeric value representing the significance or urgency assigned to an outlier based on applicable explanations or categories.
• rare combinationA rare combination is a combination of feature values that occurs relatively few times in a dataset.
• mean of squared scoresThe mean of squared scores is an aggregation technique that averages detector scores after squaring them, giving greater weight to larger scores.
• probability theoryA numerical quantity representing the degree of likelihood assigned to an event or class prediction.
• cardinalityThe number of distinct values or categories represented by a variable or column.
• sparsityData sparsity is a condition in which observed records occupy only a very small portion of the possible data space.
• volatilityVolatility is the degree and rate of fluctuation in a quantity across observations over time.
• seasonality effectA pattern of variation that recurs at a regular seasonal or calendar-related period.
• rare categoryA minority class is a class containing substantially fewer observations than the majority class or classes in a classification dataset.
• beeswarm plotA beeswarm plot displays individual observations as positioned dots, often using color to encode feature values or another variable.
• equal-frequency binningEqual-count binning divides ordered observations into intervals containing approximately the same number of observations.
• equal-width binningEqual-width binning divides a numeric range into intervals having the same width.
• feature interactionA feature interaction is a dependency in which the effect of one feature on a model output varies according to the value of another feature.
• SHAPSHAP is an open-source Python library for explaining machine-learning model outputs using feature attribution methods based on Shapley values.
• combined scoreAn aggregate value produced from multiple detector scores for the same record.
• real versus fake outlier detectorAn outlier detector that distinguishes observed records from artificially generated records and treats records classified as artificial as outliers.
• canberra distanceCanberra distance is a normalized distance metric that sums coordinate-wise absolute differences divided by the corresponding absolute-value sums.
• chebyshev distanceChebyshev distance is the maximum absolute difference between corresponding coordinates of two points.
• minkowski distanceMinkowski distance is a parameterized distance measure that aggregates powered coordinate differences and includes Manhattan and Euclidean distance as special cases.
• COPODCOPOD is a copula-based outlier-detection algorithm that uses empirical copulas and tail probabilities to score observations without assuming a parametric data distribution.
• parallelismParallelism is the execution of multiple computational operations concurrently rather than one after another.
• heat mapA color-encoded matrix visualization representing values for combinations of two categorical or numeric dimensions.
• frequencyWord frequency is the number or rate of occurrences of a particular word in a text collection.
• data groupinggroupby is a pandas method that partitions rows into groups sharing specified key values for subsequent aggregation or transformation.
• rare wordA rare word is a word that occurs substantially less often than other words in a reference text collection.
• subspace outlier detectionSOD is a subspace-based outlier detection algorithm that identifies observations anomalous in selected feature subsets.
• angle-based outlier detectionAngle-based outlier detection is an outlier-detection algorithm that evaluates observations using the variation of angles formed with pairs of other observations, with distance playing a secondary role.
• DataConsistencyCheckerDataConsistencyChecker is a data-quality and outlier-detection tool that identifies exceptions to consistency patterns within and across columns.
• LARGER testThe LARGER test checks whether values in one column are typically larger than values in another column and flags exceptions.
• time scaleA temporal resolution or interval over which observations are grouped and analyzed.
• ExplainerExplainer is a SHAP class that computes explanations for predictions or scores produced by a model.
• AEGMMAEGMM is a model that jointly trains an autoencoder and a Gaussian mixture model with a shared objective for representation learning and outlier detection.
• credit card fraudUnauthorized use of a credit card, including use after loss, theft, or counterfeiting.
• mean squared errorMean squared error is the average of the squared differences between corresponding predicted and observed values.
• OutlierAEOutlierAE is an alibi-detect detector class that identifies outliers using an autoencoder's reconstruction error.
• VAEGMMVAEGMM is a model that combines a variational autoencoder with a Gaussian mixture model for representation learning and outlier detection.
• ongoing outlier detectionA technique for continuously monitoring incoming or evolving data to detect outliers over an extended period.
• convex hullAn iterative outlier-detection method that assigns scores according to the successive layers of a convex hull around the data points.
• cycleA recurring temporal pattern in which values or behavior repeat over successive periods.
• residual errorA time-series residual is the portion of an observation remaining after modeled trend and seasonal components are removed.
• seasonal_decomposeseasonal_decompose is a statsmodels function that decomposes a time series into trend, seasonal, and residual components.
• fine-tuningFine-tuning is the continued training of a pretrained model, usually on a new task or domain, by adjusting some or all of its parameters.
• transfer learningTransfer learning is a technique that adapts knowledge learned by a model on one task or dataset to another task or dataset.
• smoothingSmoothing is a technique that reduces short-term fluctuations in an ordered sequence by averaging neighboring observations.
• univariate scoring methodA technique that assigns an outlier score to each observation using one feature at a time.
• class imbalanceClass imbalance is a distribution in which one or more target classes contain far fewer observations than another class.
• network attackA network attack is a malicious action or traffic pattern intended to compromise, disrupt, or misuse a computer network or its services.
• novel attackA novel attack is a network attack whose form was not represented among the attacks used to train a classifier.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.