liveBookmanning.com

What you will learn in Natural Language Processing in Action, Second Edition

by Hobson Lane and Maria Dyshel

Read it on liveBook Get the book at manning.com

Chapter by chapter

Concepts explained in this book

• embeddingA vector representation is a numerical vector that encodes the properties or meaning of a text, statement, word, or document.
• language modelA large language model is a neural language model trained on very large text corpora to model and generate language.
• recurrent neural networkA recurrent neural network is a neural network architecture that processes sequences while maintaining a state influenced by earlier inputs.
• conversational aiA dialog engine is a software system that manages conversational interaction and generates responses to user statements.
• convolutional layerA convolutional layer is a neural-network layer that applies learned filters across local regions of its input to produce feature maps.
• vocabularyA lexicon is a collection or index of words and other lexical sequences known to a language-processing system.
• convolution kernelA convolutional kernel is a parameter matrix or tensor whose sliding weighted operation detects a local pattern in an input sequence.
• similarity searchInformation retrieval, or search, is the process of finding relevant information or documents in a collection in response to a query.
• fine-tuningFine-tuning is the continued training of a pretrained model on a narrower dataset or task so its parameters adapt to that target domain or behavior.
• transformer architectureA transformer is a neural network architecture that uses attention mechanisms to process sequences and model relationships among their elements.
• convolutional neural networkA convolutional neural network is a neural network that applies learned convolutional filters to structured inputs to detect local patterns and construct hierarchical representations.
• parameterA weight is a learned numerical parameter that controls the contribution of an input or connection in a neural network.
• pipelineAn NLU pipeline is a sequence of natural-language-understanding stages that transform text into an interpretation or decision.
• multi-head attentionMulti-head attention computes multiple attention representations in parallel and combines them, potentially using distinct query and source sequences.
• stateA hidden state vector is the numerical state carried through a recurrent network to summarize information from earlier sequence inputs.
• topic vectorA topic vector is a numerical representation whose components encode the strength or proportion of topics associated with a document.
• regular expressionA regular expression is a formal pattern notation for matching and extracting strings that satisfy specified character or sequence constraints.
• bag-of-words modelBag-of-words is a text representation that records word occurrences while disregarding the order and syntactic relationships of the words.
• guardrailA guardrail is a constraint or control that limits a language-model system's behavior to improve safety, reliability, or adherence to requirements.
• padding tokenA padding token is a special token used to extend shorter sequences to a common length within a batch.
• positional embeddingA positional embedding is a learned vector representing a token's position within an input sequence.
• groundingGrounding is the process of anchoring a language model’s generated response in relevant external facts, sources, or user-specific information.
• toxicityToxicity is the presence of abusive, hateful, threatening, or otherwise harmful language in generated or user-provided text.
• generationNatural language generation, or text generation, is the computational production of human-readable natural-language text.
• Word2VecWord2Vec is a family of neural language models that learns dense vector representations of words from nearby-word prediction in text.
• objective functionLoss is a numerical measure of discrepancy between a model's prediction and a desired target, with lower values generally indicating better agreement.
• natural language processingNatural language processing is the field of computing concerned with enabling machines to analyze, understand, generate, and interact using natural language.
• tokenizerA tokenizer is a text-processing component that divides a character sequence into tokens such as words, subwords, or punctuation.
• classifierClassification is the assignment of an input to one or more predefined categories based on its properties.
• encoder-decoder architectureA neural architecture that encodes an input sequence into a representation and decodes that representation into an output sequence.
• recallRecall is the proportion of relevant items that a retrieval or classification system successfully identifies.
• neuronAn artificial neuron is a parameterized computational unit that combines inputs to produce an output, typically through an activation function.
• intent classificationIntent classification assigns text to a category representing the user's purpose or communicative intent.
• knowledge graphA knowledge graph is a graph-structured representation of entities, concepts, and their relationships.
• cosine similarityCosine similarity measures the angular similarity between two nonzero vectors by dividing their dot product by the product of their magnitudes.
• token count vectorA token count vector represents text by recording the number of occurrences of each token in a defined vocabulary.
• case foldingCase folding is the normalization of text by mapping letters with different capitalization to a common case.
• prompt engineeringPrompt engineering is the deliberate design and revision of prompts to influence a language model's behavior and outputs.
• precisionPrecision is the proportion of retrieved or classified items that are relevant or correct.
• byte pair encodingByte pair encoding is a subword tokenization algorithm that iteratively merges frequent adjacent symbol pairs to form a vocabulary.
• machine learningMachine learning is a computational approach in which algorithms derive patterns or behavior from data rather than relying solely on explicitly programmed rules.
• vector space modelA model that represents documents or other language objects as vectors and uses relationships among those vectors for analysis or retrieval.
• tensorA tensor is a multidimensional numerical array used to represent and transform structured data in machine-learning computations.
• lemmatizationA lemmatizer is a text-processing component that maps inflected word forms to their dictionary base forms.
• retrieval-augmented generationRetrieval-augmented generation is a language-generation technique that retrieves external information and conditions generation on the retrieved information.
• multilabel classificationMulti-label classification assigns zero or more independent labels to each input example.
• n-gramAn n-gram is a contiguous sequence of up to n extracted elements, such as words, characters, syllables, or symbols.
• evaluation datasetA validation set is a held-out portion of data used to compare model configurations during development.
• conversation flow diagramA dialog tree is a branching representation of possible conversational states, user inputs, and system responses.
• term frequencyNormalized term frequency is term frequency divided by the number of terms in the document.
• latent semantic analysisLatent semantic indexing, also called latent semantic analysis, is a dimensionality-reduction technique that represents words and documents in a lower-dimensional semantic space derived from term-document statistics.
• beginning-of-sequence tokenA start-of-sequence token is a special token marking the beginning of a sequence and signaling a generative decoder to begin output.
• pattern matchingA pattern-matching algorithm selects behavior or responses by comparing an input with predefined textual patterns.
• categorical encodingA one-hot encoder is a component that maps categorical items to binary vectors with one active position per item.
• stemmingStemming is the reduction of inflected or derived word forms to a shared stem, often by removing suffixes without requiring the result to be a valid word.
• temperatureTemperature is a decoding parameter that rescales token scores to control the randomness of a generated distribution.
• curse of dimensionalityThe curse of dimensionality is the difficulty of analyzing data as the number of dimensions grows, including sparse representations and increased computational cost.
• scalabilityScalability is the ability of a computational system or model to increase capacity, data volume, or workload while remaining practically trainable or usable.
• few-shot learningFew-shot learning is adapting a model to perform a task from a small number of examples supplied at inference time, often in the prompt.
• training datasetTraining data is the collection of examples used to fit the parameters of a machine-learning model.
• spaCy MatcherThe spaCy Matcher is a rule-based component that matches token sequences specified by token attributes such as part-of-speech tags.
• CounterA Python Counter is a mapping type that counts hashable objects and stores each object with its frequency.
• dependency parsingA software component that identifies dependency relations between words in a sentence.
• generative chatbotA generative chatbot is a chatbot that dynamically produces conversational responses from learned language representations rather than selecting only from fixed responses.
• natural languageA natural language is a human language that develops and is used for communication rather than being defined solely by formal syntactic rules.
• latent Dirichlet allocationLatent Dirichlet allocation is a probabilistic topic-modeling technique that represents documents as mixtures of latent topics and topics as distributions over words.
• GPT-2GPT-2 is an autoregressive transformer language model developed by OpenAI and trained to generate text from preceding context.
• batch sizeBatch size is the number of examples processed together in one training or evaluation step.
• dimensionality reductionDimension reduction is the transformation of data into a representation with fewer features or dimensions while retaining relevant information.
• perceptronA perceptron is a single-layer computational model that computes a weighted sum of inputs and applies a thresholding activation to make a decision.
• vocabulary normalizationVocabulary normalization is the transformation of semantically or functionally similar lexical forms into shared representations to reduce the number of distinct vocabulary items.
• attention matrixAn attention matrix is a matrix of learned or computed weights representing how strongly elements in a sequence should attend to one another.
• PyTorchPyTorch is an open-source machine-learning framework for building and training neural-network models with tensor computation and automatic differentiation.
• out-of-vocabulary wordAn out-of-vocabulary token is a token that is absent from the vocabulary or pretrained representation available to a model.
• hybrid chatbotA hybrid chatbot combines generative language models with rules, templates, or other conventional conversational techniques.
• promptA prompt is an input sequence of instructions or tokens supplied to a language model to guide its generated output.
• prompt injectionA jailbreak is an input or interaction strategy that induces an AI system to bypass its intended behavioral restrictions.
• semantic routingSemantic routing is the classification-driven selection of an appropriate language model or processing path for a natural-language input.
• LangChainLangChain is a software framework for developing applications that combine language models with tools, data sources, and orchestration logic.
• feature engineeringFeature engineering is the deliberate construction or transformation of input representations to make them useful for predictive modeling.
• activation functionAn activation function is a nonlinear function applied to a neural-network signal to transform its value before further processing.
• inner productA dot product is the scalar obtained by multiplying corresponding components of two equal-length vectors and summing the products.
• hyperparameterA hyperparameter is a user-specified setting that controls a machine-learning algorithm or processing pipeline rather than being learned from its training data.
• fallbackA fallback response is a predefined chatbot reply provided when no recognized intent meets the required confidence level.
• principal component analysisPrincipal component analysis is a statistical technique that transforms correlated variables into a smaller set of orthogonal components capturing as much variance as possible.
• coreference resolutionCoreference resolution is the task of determining which entity or expression a referring expression, such as a pronoun, denotes in text.
• part of speechA part of speech is a grammatical category such as noun, verb, or adverb assigned to a word according to its syntactic role.
• transformersHugging Face Transformers is an open-source library providing pretrained transformer models and utilities for natural-language and other machine-learning tasks.
• subword tokenizationA subword is a word fragment, character sequence, or complete word treated as a unit in subword-based language modeling.
• BERTBERT is a transformer language model pretrained with bidirectional contextual objectives for representing and understanding text.
• attentionAn attention mechanism is a sequence-processing operation that assigns different importance to input elements when computing a representation or output.
• accuracyAccuracy is the proportion of evaluated predictions that are correct.
• transfer learningTransfer learning is reusing knowledge learned from one task or dataset to improve learning on another task or dataset.
• analogy queryingAnalogy reasoning is a technique that combines or compares vector representations arithmetically to model relationships among concepts.
• gradientA gradient is the vector of partial derivatives that indicates how a function changes with respect to its parameters or inputs.
• sparse representationA sparse representation stores only nonzero or otherwise present values instead of explicitly storing the many default values in a larger representation.
• tokenA discrete lexical unit obtained by splitting source text into identifiable pieces, including individual characters in character-based processing.
• SeriesA pandas Series is a one-dimensional labeled data structure provided by the pandas Python library.
• vector databaseA vector store is a software component that stores and retrieves numerical vectors, typically with similarity-search indexes.
• knowledge baseA knowledge base is an organized collection of facts, entities, and relationships representing information about a domain or the world.
• character sequenceA character sequence is an ordered sequence of textual characters treated as a discrete symbolic representation.
• term frequency-inverse document frequencyA TF-IDF model represents text using term-frequency and inverse-document-frequency weights for terms in documents.
• Guardrails AIGuardrails AI is an open-source Python package for specifying and validating constraints on large-language-model inputs and outputs.
• NLP filterAn NLP filter is a language-processing mechanism that detects or removes text violating specified content constraints.
• wordpieceWordPiece tokenization is a subword tokenization technique that represents text using frequent word fragments and character sequences.
• l2 normThe L2 norm of a vector is the square root of the sum of the squares of its components.
• approximate nearest neighbor algorithmAn approximate nearest neighbor algorithm finds items near a query vector with lower computational cost than exhaustive exact search while allowing some accuracy loss.
• generalizationGeneralization is a model's ability to produce useful predictions for previously unseen inputs, including inputs outside its training examples.
• stop wordA stop word is a frequent function word that usually contributes relatively little substantive information to a phrase's meaning.
• backpropagationBackpropagation is an algorithm for computing gradients of a model's loss with respect to its parameters by propagating error signals backward through the computation.
• emergent behaviorAn emergent capability is a behavior or performance characteristic that appears to arise only when a system reaches a particular scale or complexity.
• overfittingOverfitting is a modeling failure in which a model relies too heavily on peculiarities of its training data and performs poorly on new data.
• sparse vectorA sparse vector is a vector in which most elements are zero or otherwise empty.
• GloVeGloVe is a word-embedding model that learns vector representations from global word co-occurrence statistics.
• dataset splitA train-test split partitions examples into separate subsets for model fitting and evaluation.
• early stoppingEarly stopping is a training regularization technique that terminates optimization when validation performance stops improving.
• multi-hot tensorA multi-hot vector is a binary vector with any number of active positions indicating multiple selected labels from a fixed set.
• simpletransformerssimpletransformers is a Python library that provides high-level wrappers for transformer models and task-specific training workflows.
• LSTMLong short-term memory is a recurrent neural network architecture that uses gated state updates to preserve and control information over long sequences.
• channelA convolution channel is an independent stream of input features processed alongside other streams by a convolutional layer.
• inverted indexAn inverted index maps terms to the documents and positions in which they occur to support efficient text retrieval.
• templateA prompt template is a reusable specification that inserts variable input into a predefined language-model prompt.
• sentiment analysisSentiment analysis is the computational determination of opinions, emotions, or evaluative attitudes expressed in text.
• projectionA projection is the mapping of a vector onto a specified line or axis to obtain its component along that direction.
• samplingSampling is a decoding method that selects the next token probabilistically from a distribution over candidate tokens.
• document frequencyDocument frequency is the number of documents in a collection in which a specified word or token occurs.
• vector spaceA vector space is a mathematical set of vectors that can be combined through operations such as addition and scalar multiplication.
• [MASK] tokenThe mask token is a special placeholder used to hide an input token during masked language model pretraining.
• labeled dataA labeled dataset is a dataset whose examples are paired with target labels or annotations for supervised learning.
• finite-state machineA finite state machine is a computational model that transitions among a finite set of states in response to input symbols.
• contextContext is semistructured information about a user or conversation session that constrains how subsequent dialog should be interpreted or generated.
• biasWord vector bias is systematic distortion in learned word associations caused by stereotypes and imbalances in the training corpus.
• optimizationAn optimization algorithm is a procedure for selecting parameter values that improve an objective function.
• linear layerA fully connected linear layer is a neural-network layer that computes an affine transformation from every input activation to its output units.
• learning rateA learning rate is a numerical step-size setting that controls how far an optimization algorithm moves parameters in response to a gradient.
• named entity recognitionNamed entity recognition identifies spans of text that refer to entities such as people, places, organizations, dates, or locations and assigns them entity categories.
• conversation designConversation design is the practice of planning and specifying how users and conversational systems interact through dialogue.
• dot-product attentionDot-product attention scores queries and keys with scalar products and uses those scores to weight values.
• scaled-dot product attentionScaled dot-product attention divides query-key dot products by the square root of the key dimension before normalization.
• stop-word removalA stop word filter is a text-processing filter that removes selected high-frequency function words from tokenized text.
• anonymizationAnonymization is the transformation of data to remove or obscure information that can identify individuals.
• ASCIIASCII is a character-encoding standard that assigns numeric codes to a limited set of Latin letters, digits, punctuation marks, and control characters.
• dataset biasDataset bias is systematic distortion in a dataset caused by how examples, labels, or populations are selected and represented.
• diacriticA diacritic is a mark added to a character to modify its pronunciation, appearance, or linguistic interpretation.
• solvable machine learning problemA solvable machine learning problem is a prediction task for which humans can answer the questions, most cases have a correct answer, and performance should not substantially exceed that of a trained human expert.
• transliterationTransliteration is the conversion of characters and spellings from one writing system into another while preserving the source language's linguistic content as closely as possible.
• upsamplingOversampling the minority class increases the representation of underrepresented classes in training data, often by duplicating or resampling examples.
• poolingA pooling layer is a neural-network layer that aggregates neighboring activations to reduce their representation while retaining salient information.
• logistic regressionLogistic regression is a statistical classification model that estimates the probability of a class from a weighted combination of input features.
• relative frequencyRelative frequency is the proportion of a document's token positions occupied by a specified term.
• relation extractionNeural relation extraction is the use of neural models to identify semantic relations between entities in text.
• Euclidean distanceEuclidean distance is the straight-line distance between two points or vectors computed from the square root of the sum of squared coordinate differences.
• structured extractionInformation extraction is the automatic identification and structuring of facts, entities, and relations from unstructured text.
• sentence boundary detectionA sentence boundary detector identifies and separates individual sentences in a text.
• sexSex assigned at birth is the classification recorded for a person at birth based primarily on observed physical and anatomical traits.
• f-scoreF1 score is the harmonic mean of precision and recall for a classification or retrieval evaluation.
• hallucinationHallucination is the production of fluent or plausible language that is unsupported, incorrect, or disconnected from reality.
• multi-step reasoningReasoning is the process of deriving conclusions or solving problems by applying relationships among facts, premises, or representations.
• question answeringA question-answering system produces answers to questions expressed in natural language.
• distanceA distance metric is a function that quantifies dissimilarity between objects while satisfying the mathematical properties of a metric.
• Hugging FaceHugging Face Hub is an online repository and collaboration platform for sharing machine-learning models, datasets, and applications.
• LlamaLlama 2 is a family of autoregressive large language models released by Meta for text generation and understanding.
• Llama2-7BLlama 2 7B is the seven-billion-parameter variant of the Llama 2 language-model family.
• false positiveA false positive classification error occurs when a system assigns a positive or applicable label to an instance when that label is incorrect.
• rule-based conversation managerA rule-based conversation manager controls dialogue with explicit rules that determine permitted states, actions, or responses.
• batchingBatch learning is a training technique that computes parameter updates from a selected group of training examples at a time.
• semantic analysisSemantic analysis is the computational interpretation of the meanings of words, phrases, or statements.
• inverse document frequencyInverse document frequency is a term-weighting measure defined from how many documents in a corpus contain a term, assigning lower weight to terms occurring in many documents.
• correlationA correlation coefficient is a numerical value quantifying the relationship between paired values in two sequences.
• embedding layernn.Embedding is a PyTorch module that maps integer token indices to trainable dense embedding vectors.
• skip-gramSkip-gram is a word-embedding training approach that predicts surrounding or skipped words from a context word.
• red-teamingA red team is an authorized person or group that tests a system by attempting adversarial actions against it.
• semantic-routerA semantic router is a routing component that predicts the most appropriate model or processing destination for an input based on its meaning.
• sigmoidA logistic function is an S-shaped mathematical function that maps an input to a bounded continuous value and provides a soft threshold.
• FastTextfastText is a word-representation model that learns from character n-grams and composes representations for words from their subwords.
• interceptA bias is a constant input signal, conventionally equal to one, that allows a neuron to shift its activation independently of the other inputs.
• step functionA step function maps a value to one of discrete outputs according to whether it crosses a specified threshold.
• locality-sensitive hashingLocality-sensitive hashing is a hashing technique designed to map similar high-dimensional objects to the same or nearby hash buckets with relatively high probability.
• masked language modelingMasked language modeling is a pretraining technique that masks tokens in an input sequence and trains a model to predict the original tokens from surrounding context.
• edgeAn edge is a graph relationship connecting two nodes or vertices.
• stochastic gradient descentStochastic gradient descent is an iterative optimization technique that updates model parameters using randomized or sampled training examples rather than the entire dataset at once.
• pattern-based approachA conversational processing system whose behavior is determined primarily by matching predefined text patterns.
• singular value decompositionSingular value decomposition is a matrix factorization that expresses a matrix as the product of orthogonal singular-vector matrices and a diagonal singular-value matrix.
• false negativeA false negative classification error occurs when a system fails to identify an instance that belongs to the target class.
• CountVectorizerCountVectorizer is a scikit-learn transformer that converts a collection of documents into a matrix of token occurrence counts.
• regular grammarA regular grammar is a formal grammar that generates exactly the regular languages through restricted production rules.
• tokenizationTokenization is the process of dividing a character sequence into discrete lexical units such as words or punctuation.
• paddingPadding is the addition of extra positions around an input sequence so boundary regions can be processed and output dimensions controlled.
• forward passForward propagation is the computation of a neural network's output by passing inputs through its units in the forward direction.
• character trigram searchA character trigram index organizes overlapping three-character sequences, including word-boundary markers, to support approximate text matching.
• wikitextWikiText2 is a corpus of Wikipedia article text distributed as a benchmark dataset for language modeling.
• HaystackHaystack is an open-source framework for building natural-language search, question-answering, and retrieval-augmented generation pipelines.
• GRUA gated recurrent unit (GRU) is a recurrent neural-network architecture that uses reset and update gates to control information flow and hidden-state updates.
• spaCyspaCy is an open-source software library for advanced natural-language processing, including tokenization and syntactic analysis.
• edit distanceA string distance equal to the minimum number of single-character insertions, deletions, and substitutions needed to transform one string into another.
• confidenceLabel confidence is a classifier's estimated probability that an input belongs to a particular label.
• graphA graph is a data structure consisting of entities called nodes or vertices connected by relationships called edges.
• grammarA grammar is a system of rules that specifies how valid expressions in a language are formed and interpreted.
• vectorizationA vectorizer is a function or component that transforms text into numerical vectors.
• Zipf's lawZipf's law is an empirical distribution law stating that the frequency of an item is approximately inversely proportional to its rank in a frequency-ordered list.
• cosine distanceThe cosine distance metric measures the angular dissimilarity between vectors and is commonly used to estimate similarity in their encoded meanings.
• softmaxSoftmax is a normalized exponential function that converts a vector of scores into nonnegative values summing to one.
• natural language understandingNatural language understanding is the subfield of natural language processing concerned with computationally determining the meaning of human language.
• max poolingMax pooling is a downsampling operation that retains the largest activation within each local region of a feature map.
• linear discriminant analysisLinear discriminant analysis is a classification technique that models classes with shared feature covariance and learns linear decision boundaries.
• gatingA logic gate in a gated recurrent network is a learned signal-control mechanism that regulates how information passes into or out of the recurrent state.
• ambiguityLinguistic ambiguity is the property of a word or phrase having multiple plausible meanings or interpretations.
• unsupervised learningUnsupervised learning is a machine-learning approach that derives patterns or representations directly from data without human-provided target labels.
• beam searchBeam search generates sequences by retaining a limited set of high-scoring partial paths and expanding them over multiple decoding steps.
• greedy algorithmGreedy search generates a sequence by selecting the highest-probability next token at each step without planning over alternative continuations.
• logitA logit is an unconstrained real-valued score obtained by applying the inverse of the logistic or softmax probability transformation.
• top-p filteringNucleus sampling restricts decoding to the smallest set of tokens whose cumulative probability reaches a specified threshold before sampling.
• information retrievalFull-text search retrieves documents by searching their complete textual contents rather than only structured fields.
• binary vectorA bit vector is a vector whose elements are binary values representing discrete features or states.
• instrumentationModel instrumentation is the addition of observations or logging that exposes a model's internal computations during execution.
• FAISSDocumentStoreFAISSDocumentStore is a document-store component that indexes and retrieves documents using a FAISS similarity-search index.
• semantic distanceSemantic distance is a distance between language representations that indicates how dissimilar their meanings or contextual associations are.
• local averagingA moving average is an aggregate that summarizes recent values over a changing or weighted window of observations.
• global optimumA global minimum is a parameter setting with the lowest objective value over the entire domain under consideration.
• part-of-speech taggingPart-of-speech tagging assigns grammatical word categories such as noun, verb, or adjective to tokens in text.
• sentence boundaryA sentence boundary is the position in text that marks the end of one sentence and the beginning of another.
• corpusA corpus is a structured collection of documents or language samples used for linguistic analysis or model development.
• data-driven chatbot designA chatbot-design approach that selects or generates responses from statistical patterns learned from conversational data.
• recurrenceA recurrence loop is a computation pattern that repeatedly feeds a step's state or output into the next step of sequence processing.
• Elo scoreAn Elo ranking is a rating system that estimates the relative strength of competitors from the outcomes of pairwise contests.
• LMSYS benchmark datasetThe LMSYS benchmark dataset is a human-generated test dataset for evaluating the general intelligence and performance of large language models and chatbots.
• ShareGPTShareGPT is a dataset of user-provided conversations with ChatGPT used for training and evaluating conversational language models.
• VicunaVicuna is an open-source large language model fine-tuned from Llama 2 for conversational and virtual-assistant tasks.
• deep learning frameworkA deep learning framework is software that provides abstractions and computational support for constructing, training, and running neural-network models.
• birth certificate datasetA birth certificate dataset is a collection of birth-record entries containing names, recorded sex, and associated counts or demographic fields.
• out-of-distributionAn out-of-distribution example is an input whose relevant data-generating conditions differ from those represented in a model's training distribution.
• bidirectional language modelA bidirectional language model represents or predicts text using contextual information from both preceding and following tokens.
• causal language modelA causal language model predicts each position using only information from earlier or currently available positions in the sequence.
• truncated singular value decompositionTruncatedSVD is a scikit-learn estimator that computes a truncated singular value decomposition, typically using randomized methods for sparse input.
• error surfaceAn error surface is a multidimensional function relating model parameter values to the resulting prediction error.
• attention headA self-attention head is an independent attention mechanism that computes relationships among sequence positions within a transformer layer.
• generative approachA conversational generation technique that produces responses dynamically instead of selecting them from a predefined set.
• rule-based filterA deterministic filtering mechanism that accepts, rejects, or routes inputs or outputs according to explicit rules.
• RasaRasa is an open-source framework for developing conversational AI applications with configurable natural-language processing and dialogue management.
• text-to-speech conversionText-to-speech is the synthesis of spoken audio from written text by software.
• wake word detectionWake word detection is the automatic identification of a designated spoken phrase that activates a voice assistant.
• word error rateWord error rate is the proportion of word-level transcription errors relative to the reference transcription.
• similarity metricSimilarity is a numerical or qualitative measure of how closely two patterns or signals resemble each other.
• confusion matrixA confusion matrix is a table counting predictions by their actual and predicted classes.
• toxicity scoreA toxicity score is a scalar value representing the estimated degree to which a comment belongs to the toxic class.
• message logA message log is a stored record of messages and conversation interactions exchanged during user sessions.
• reinforcement learning with human feedbackReinforcement learning with human feedback is a model-training technique that uses human evaluations to optimize a system's behavior.
• character classA regular-expression construct that matches one character from a specified set or range.
• raw string literalA Python string literal in which backslashes are preserved rather than normally interpreted as escape sequences.
• backpropagation through timeBackpropagation through time is a training technique that applies backpropagation across the successive time steps of a recurrent neural network.
• shared weightWeight sharing is the reuse of the same learned parameters across multiple inputs, positions, or computation steps.
• timestepA time step is one ordered position in a sequential computation, corresponding here to a token position in text.
• hyperparameter tuningHyperparameter tuning is the systematic process of selecting hyperparameter values to improve a model or processing pipeline.
• dependency graphA dependency tree is a tree representation of grammatical relationships in which words depend on other words within a sentence.
• user experienceUser experience is the overall quality of a person's interaction with a product or interface, including usability, satisfaction, and effectiveness.
• slot fillingSlot filling is the extraction and assignment of predefined conversational parameters from a user's utterance.
• common senseCommonsense knowledge is general background knowledge about everyday entities, events, relationships, and expectations.
• tf-idf-to-topic transformationA TF-IDF-to-topic transformation is a linear mapping that converts a document's TF-IDF vector into a lower-dimensional topic vector.
• topicA topic is a latent semantic theme represented as a dimension in a reduced space of language data.
• trigram modelA character trigram is a sequence of three consecutive characters or word-boundary characters extracted from text.
• user engagementAn engagement metric measures the frequency, duration, or intensity of user interaction with a product or service.
• symbolic reasoningSymbolic reasoning is an artificial-intelligence technique that derives conclusions by manipulating explicit symbols, facts, and logical rules.
• Continuous Bag of WordsContinuous bag-of-words is a word-embedding training approach that predicts a target word from nearby context words in a sliding window.
• adoption rateFeature adoption rate is the proportion or number of users who use a particular product feature.
• churn funnelA churn funnel is a stepwise representation of a user journey that shows where users disengage or drop out.
• retentionA retention metric measures the proportion of users who return to use a product or service after an earlier interaction.
• HEART frameworkThe HEART framework is a user-experience measurement framework organized around happiness, engagement, adoption, retention, and task success.
• task success metricA task success metric measures how effectively users accomplish the intended goals of an application or conversational system.
• gensimGensim is an open-source Python library for unsupervised natural-language processing and semantic representation learning, including Word2Vec models.
• coreference chainA coreference chain is a set of text mentions that refer to the same underlying entity or discourse referent.
• vector indexVector search indexing is the process of organizing numerical vector representations so that semantically similar items can be retrieved efficiently.
• activationA hidden layer activation is the numerical output produced by a neural-network unit or layer for a particular input state.
• hidden layerA hidden layer is an intermediate neural-network layer whose activations are not directly exposed as the model's input or final output.
• nodeA node is an entity or vertex in a graph that can be connected to other nodes by edges.
• hybrid searchA hybrid search pipeline combines vector-similarity retrieval with full-text retrieval to search by both semantic and lexical signals.
• dependency relationA dependency relation is a labeled grammatical link between a head word and a dependent word in a dependency parse.
• dependentA dependent is a word whose grammatical role is defined in relation to a governing head word in a dependency parse.
• headA head is the governing word in a dependency relationship to which another word is attached.
• root nodeThe root is the single word in a dependency tree that does not depend on another word in that tree.
• semantic similaritySemantic similarity is the degree to which two language items have similar meanings or contextual associations.
• CorefereeCoreferee is a spaCy pipeline extension that performs transformer-based coreference resolution.
• statistical sentence segmenterA sentence segmenter that predicts sentence boundaries from statistical patterns learned from annotated text.
• Dirichlet distributionA Dirichlet distribution is a probability distribution over categorical probability vectors whose components are nonnegative and sum to one.
• data collatorA data collator is a component that dynamically assembles samples into batches and may apply batch-level preprocessing such as padding.
• DataCollatorForLanguageModelingDataCollatorForLanguageModeling is a Transformers data-collation class that prepares batches for language-model training, including padding and token-label construction.
• TrainerTrainer is a high-level Transformers API class that orchestrates model training and evaluation from configured arguments and data components.
• training argumentsTrainingArguments is a Transformers configuration object containing parameters for a training run.
• bag-of-n-gramsBag-of-n-grams is a text representation that records occurrences of contiguous token sequences of a specified length while disregarding their order in the document.
• forward indexA forward index maps each document to the words or features it contains.
• word boundary foldingWord boundary folding normalizes punctuation and whitespace at word boundaries to one uniform character representation.
• indexingIndexing is the construction of auxiliary structures that organize document content for efficient retrieval.
• locally optimal solutionA local minimum is a point where an objective function has a value no greater than that of nearby points, without necessarily being the global minimum.
• HNSWHierarchical Navigable Small World is a graph-based approximate nearest-neighbor algorithm that organizes navigable proximity graphs into progressively sparser layers.
• chat loopA conversation loop is a cycle in dialogue transitions that revisits a previous conversational state or sequence.
• directed acyclic graphA directed acyclic graph is a graph whose edges have directions and that contains no directed cycles.
• state transitionA state transition is a change from one conversational state to another in response to an input or condition.
• dropoutA dropout layer is a neural-network layer that randomly disables units during training to reduce reliance on particular internal representations.
• carry stateA cell state is the persistent recurrent representation maintained by an LSTM across sequence positions.
• cognitionCognition is the set of mental processes involved in acquiring, representing, manipulating, and using knowledge.
• formal symbol manipulationSymbol manipulation is the transformation and recombination of symbolic representations according to explicit rules.
• cross entropy lossLog loss is a probabilistic prediction metric based on the negative logarithm of the probability assigned to the observed target.
• momentumMomentum is an optimization hyperparameter that incorporates prior update direction to accelerate or damp parameter movement during gradient-based optimization.
• torch.nn.ModuleA PyTorch module is a reusable neural-network component represented by the torch.nn.Module base class and its registered parameters and submodules.
• principal componentA principal component is an orthonormal direction in transformed data space along which variance is maximized subject to the preceding components.
• working memoryShort-term memory is a temporary store for information that remains available over a limited span of processing.
• parsingA parser is a software component that analyzes text to identify its structural parts and the relationships among them.
• force-directed layoutForce-directed layout is a graph-visualization technique that positions connected nodes using simulated attractive and repulsive forces.
• similarity matrixA similarity matrix is a square matrix whose entries quantify pairwise similarity between objects.
• undirected graphAn undirected graph is a graph whose relationships do not have a direction.
• gated updateAn update gate is a gated recurrent mechanism that controls how much of the previous hidden state is retained versus replaced by new information.
• reset gateA reset gate is a gated recurrent mechanism that controls which portions of the previous hidden state are discarded as irrelevant to the current output.
• stackabilityStackability is the property of a neural architecture whose layers can be composed repeatedly while preserving compatible input and output interfaces.
• safetyAI safety is the field concerned with preventing artificial intelligence systems from causing harmful outcomes and maintaining reliable human control, including over long-term risks.
• negative log-likelihoodNegative log likelihood is the negative of the logarithm of a model-assigned likelihood, commonly minimized as a probabilistic prediction loss.
• NLLLossNLLLoss is a PyTorch loss function that computes negative log likelihood loss for model outputs and target classes.
• document-term matrixA term-document matrix is a matrix whose rows represent terms and whose columns represent documents, with entries describing term-document associations.
• extractive summarizationExtractive summarization creates a summary by selecting and arranging passages or sentences from the source document.
• unlabeled dataUnlabeled text is textual data that lacks human-provided category, entity, or other annotation labels.
• AI control problemThe AI control problem is the challenge of keeping an artificial intelligence system aligned with human instructions and preventing harmful unintended behavior.
• bit vector language modelA bit vector language model represents language with binary dimensions indicating whether specified words or features are present.
• entityAn entity is a recognized semantic item in an utterance, such as a location, time expression, person, drug, or crop type.
• decodeDecoding is the transformation of a machine representation back into a human-interpretable or otherwise target-form output.
• en_core_web_trfen_core_web_trf is spaCy's English transformer language model pipeline.
• foundational modelA pretrained model is a model whose parameters were previously learned from a separate training corpus or task.
• Conv1DConv1d is PyTorch's one-dimensional convolution layer for applying learned or specified kernels along a sequence dimension.
• part-of-speech vectorA part-of-speech vector is a numerical representation whose coordinates indicate grammatical categories assigned to a word.
• string matchString matching is the process of finding exact or approximate occurrences or correspondences between character sequences.
• vector arithmeticVector algebra is the system of addition, subtraction, scalar operations, and other mathematical operations on vectors.
• scale invarianceScale invariance is the property of recognizing a pattern despite changes in its temporal or spatial extent.
• translation invarianceTranslation invariance is the property of recognizing a pattern despite changes in its position within an input.
• regular expression tokenizerA regular expression tokenizer is a tokenizer that divides text into tokens by applying regular-expression rules.
• composite indexA composite index combines multiple indexing algorithms or stages into a single retrieval structure.
• quantizationQuantization is the transformation of numerical values into lower-precision discrete representations, often to reduce memory use and accelerate computation.
• OpenSearchOpenSearch is an open-source search and analytics platform with full-text, vector, and approximate-nearest-neighbor search capabilities.
• product quantizationProduct quantization is a vector-compression technique that splits a vector into subvectors and quantizes each subvector separately.
• actionA bot action is an executable response or operation that a conversational system performs after processing user input.
• CountVectorsFeaturizerCountVectorsFeaturizer is a Rasa pipeline component that converts text into count-based token features.
• DIETClassifierDIETClassifier is a Rasa neural model that jointly performs intent recognition and entity detection.
• storyA story is a Rasa dialogue specification that represents a possible sequence of user intents and bot actions.
• frequency analysisFrequency analysis is the examination of how often symbols, tokens, or other elements occur in a text or signal.
• rot13A Caesar cipher is a substitution cipher that shifts each letter by a fixed position in an ordered alphabet.
• spectral analysisSpectral analysis is the analysis of a signal by examining the distribution and relationships of its frequency components.
• strideA stride is the number of input positions by which a convolution kernel moves between successive applications.
• term frequency vectorA word frequency vector is a numerical vector whose coordinates record the counts or presence frequencies of vocabulary words in a document.
• graph databaseA graph database is a database system that stores entities and their relationships as graph structures for graph-oriented querying and analysis.
• naive bayes classifierNaive Bayes is a family of probabilistic classifiers based on Bayes' theorem and a conditional-independence assumption among features.
• formal grammarA finite set of rules that specifies how symbols can be combined into strings belonging to a formal language.
• conditional probabilityConditional probability is the probability of an event given that another event or condition is known to hold.
• polynomial feature interactionA polynomial feature is a model input formed by raising one or more original features to powers or multiplying them in interactions.
• mean squared errorMean squared error is the average of the squared differences between predicted and target numerical values.
• formal languageA formal language is a language defined by strict rules specifying how valid statements are composed and interpreted.
• ChineseChinese is a human language whose writing commonly uses logographic characters and does not normally delimit words with spaces.
• dense matrixA dense matrix is a matrix representation that explicitly stores every element, including zero-valued elements.
• Poisson distributionA Poisson distribution is a probability distribution for counting events in a fixed interval given a specified average rate.
• term-topic matrixA term-topic matrix is a matrix whose entries represent the probabilities or weights of terms under latent topics.
• keyword search indexA search index is a data structure that organizes terms and their associated documents to support efficient retrieval.
• clippingClipping is truncating an input sequence by removing elements beyond a specified maximum length.
• pandaspandas is a Python library for manipulating and analyzing labeled tabular and time-series data.
• data frameA DataFrame is a tabular data structure organized into labeled rows and columns that may contain heterogeneous values.
• sentiment polarity scoreA sentiment polarity score is a numerical measure of how positive or negative a text's expressed attitude is.
• rule-based systemA rule-based algorithm determines outputs by applying explicitly designed human-authored rules to input data.
• named entityA named entity is a textual reference to a specific person, place, thing, date, location, or other identifiable information-bearing entity.
• recursionRecursion is a computational process in which an operation invokes itself or reuses its output as input until a termination condition is reached.
• encoderAn encoder is the transformer component that transforms an input sequence into contextual representations through stacked attention and feedforward layers.
• document databaseA data store that organizes and retrieves records represented as documents, typically with flexible schemas.
• readerA Reader is a Haystack question-answering component that extracts an answer from the retrieved document context.
• constituency parsingA constituency relation describes how words group together into nested grammatical phrases within a sentence.
• conditional random fieldA conditional random field is a probabilistic structured-prediction model that assigns labels to related observations under conditional dependencies.
• character-based NLP pipelineA character-based NLP pipeline processes text by representing it as individual characters or character-level tokens.
• SentencePieceSentencePiece tokenization is a subword tokenization technique that divides text into character-based pieces without requiring whitespace-delimited words.
• embodied AIEmbodied AI is artificial intelligence situated in a physical body that can perceive and interact with the real world.
• grounded language modelA grounded language model is a language model whose language behavior is linked to physical experience or verified external knowledge.
• continuous improvementA cyclic processing pattern in which system outputs are fed back as inputs for further analysis.
• co-occurrenceCo-occurrence is the occurrence of linguistic units together or in a shared sequence within a text.
• sequenceA time series is a sequence of observations indexed by successive points in time.
• brute-force searchA brute-force search algorithm evaluates candidates directly, typically examining all available possibilities to find a match or optimum.
• frequencyToken frequency is the number or relative rate at which a token occurs in a text collection.
• windowizerA windowizer partitions a token sequence into overlapping local context windows.
• surpriseSurprise is an information-theoretic quantity representing how unexpected an observed symbol or event is.
• initializationInitialization is the process of assigning starting values to a neural network’s states, weights, and biases before training.
• stateful trainingStateful training carries a recurrent network’s hidden state from one batch or document to subsequent inputs.
• happy pathA happy path is a planned interaction sequence representing a successful, expected user journey without errors or exceptional conditions.
• cachingCaching is the practice of storing computed results so repeated requests can be served without recomputing them.
• Hugging Face SpaceHugging Face Spaces is a platform for hosting and sharing interactive machine-learning applications and demos.
• StreamlitStreamlit is an open-source Python framework for rapidly creating interactive web applications.
• sentence splittingSentence segmentation is the process of dividing text into individual sentence units and identifying their boundaries.
• ChatterBotChatterBot is an open-source Python chatbot project that generates replies from stored statement-response pairs.
• fuzzy regular expressionA fuzzy regular expression is a regular-expression matching method that permits a bounded number of insertion, deletion, or substitution errors.
• inference engineAn inference engine is a computational system that derives conclusions from facts and rules and can use them to support decisions.
• shallow NLPShallow NLP is natural-language processing that relies on relatively few processing stages and limited linguistic representation depth.
• utterance labeling systemAn utterance labeling system is software for assigning annotations or intent labels to user utterances.
• intentionAn intent is a category representing an objective that a user wants an NLP system or chatbot to carry out.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.