Sampling
Sampling is a procedure for selecting or generating observations, values, or outcomes from a larger population, signal, or probability distribution according to a specified method.
Optimization Algorithms: AI techniques for design, planning, and control problems
A sampling method that generates candidate points using random draws from a specified space or distribution.
- How it is used
- Generates initial candidates for population-based optimization and supplies random choices for genetic-algorithm selection.
- How deep it goes
- Compares sampling methods by point dispersion, diversity, and cost, then demonstrates Python initialization without deriving convergence guarantees.
- Examples
- Plots candidate points from six methods and initializes ticket-pricing populations with Python and pymoo.
- What you can do after
- Apply it via NumPy or pymoo to create candidates and compare their coverage before optimization.
Advanced Algorithms and Data Structures
A random sampling algorithm generates candidate solutions through random choices and selects among the sampled results without iterative optimization after initialization.
- How it is used
- Finds a low-crossing valid graph embedding by selecting the best of independently generated drawings.
- How deep it goes
- Works through candidate generation, validation, crossing counts, and best-result tracking; leaves geometry-specific intersection tests abstract.
- Examples
- Uses random vertex positions in a rectangle and edge-intersection counts, with pseudocode for validation and scoring.
- What you can do after
- Build it from scratch for graph embeddings by rejecting invalid candidates and retaining the valid drawing with the fewest crossings.
Read how Advanced Algorithms and Data Structures explains sampling →
Deep Learning with Python, Third Edition
The process of drawing values or examples from a probability distribution, here to generate new content from a learned space.
- How it is used
- Turns token predictions or learned image representations into generated text or images.
- How deep it goes
- Works through greedy, random, temperature, and top-K text generation, then implements latent sampling in a VAE.
- Examples
- Compares mini-GPT continuations from one prompt and decodes MNIST digits from a two-dimensional latent grid.
- What you can do after
- Apply it via Keras sampling functions to vary text generation, or build a VAE sampling layer for image generation.
Read how Deep Learning with Python, Third Edition explains sampling →
Grokking Statistics
Sampling is the process of selecting observations from a population to form a sample for analysis or experimentation.
- How it is used
- Supports experiments, data mining, and decisions by relating observed data to a larger population.
- How deep it goes
- Explains population definition, random selection, sample size, context, and bias without deriving statistical sampling formulas.
- Examples
- Uses a six-person dorm sleep study, self-driving taxi records, airport metrics, filament factories, and wearable-device selection.
- What you can do after
- Apply it to define a population, select a sample, question representativeness, and disclose bias and context.
Learn Generative AI with PyTorch
A prediction strategy that samples next tokens according to predicted probabilities instead of always taking the maximum.
- How it is used
- Generates text or music by drawing each next token from predicted probabilities and feeding the extended sequence back into the model.
- How deep it goes
- Works through prompt handling, softmax, random draws, stopping conditions, temperature, and top-K rather than deriving their mathematics.
- Examples
- Compares repeated text continuations under different settings and extends a MIDI-derived prompt from 250 to 1,000 musical events.
- What you can do after
- Build it from scratch as an iterative sampling function, then adjust temperature and top-K to change generated text’s variation.
Read how Learn Generative AI with PyTorch explains sampling →
Natural Language Processing in Action, Second Edition
Sampling is a decoding method that selects the next token probabilistically from a distribution over candidate tokens.
- How it is used
- Applies probabilistic decoding to make autoregressive language generation less predictable and more varied.
- How deep it goes
- Explains conditional token probabilities, ranking, temperature, and nucleus-sampling configuration, while simplifying the full branching process.
- Examples
- Uses GPT-2 inference, ranked next-token probabilities, a fishbone decision tree, and nucleus sampling for the prompt “NLP is a”.
- What you can do after
- Apply it via Hugging Face Transformers by enabling sampling and configuring top_p during GPT-2 text generation.
Read how Natural Language Processing in Action, Second Edition explains sampling →
Tiny Python Projects: Learn coding and testing with puzzles and games
- How it is used
- Uses random selection to build varied outputs, mutate text, choose workouts, and assemble word-based passwords.
- How deep it goes
- Explains choice, unique sampling, replacement, seeding, exact index selection, and string reconstruction rather than probability theory.
- Examples
- Demonstrates insults, character mutation, ransom capitalization, workout selection, and word-based password generation.
- What you can do after
- Apply it via Python’s random module to select unique items, choose one value, reproduce runs, mutate text, and generate outputs.
Read how Tiny Python Projects: Learn coding and testing with puzzles and games explains sampling →
Cloud Observability in Action
Sampling is the technique of selectively capturing or discarding telemetry records according to a decision policy.
- How it is used
- Reduces distributed-tracing and logging volume before collection, ingestion, storage, and querying become operationally expensive.
- How deep it goes
- Compares no, head-based, and tail-based strategies, including decision timing, error coverage, state, latency, and timeout limitations.
- Examples
- Uses fixed percentages, error-bearing traces, downstream timeouts, production span volumes, and heuristic log-event pruning.
- What you can do after
- Evaluate sampling strategies against trace relevance, network and storage cost, implementation complexity, and incomplete-request behavior.
Causal AI
A sampling process is a procedure that generates observed outcomes by drawing values from a probability distribution.
- How it is used
- Uses sampling to generate model outcomes and represent how observed datasets arise from population distributions.
- How deep it goes
- Defines random draws and IID conditions, then demonstrates library calls and their place in a DGP-to-data hierarchy.
- Examples
- Shows categorical draws, pgmpy DataFrames, a Pyro plate, and five observations from a simulated window-breaking process.
- What you can do after
- Apply it via pgmpy or Pyro to generate outcomes and distinguish sampled data from its assumed population distribution.
Effective Conversational AI
Sampling is a language-generation decoding strategy that selects output tokens probabilistically from a model's predicted distribution.
- How it is used
- Generates varied candidates for chatbot intent data, dialogue rewrites, and simulated user replies.
- How deep it goes
- Contrasts nondeterministic sampling with repeatable greedy decoding and adjusts temperature, without explaining token-selection internals.
- Examples
- Uses password synonyms, repeated voice-error rewrites, and generated replies to chatbot test scenarios.
- What you can do after
- Apply it via LLM decoding settings to generate multiple candidates, then select or refine the helpful ones.
Building Quantum Software with Python
Sampling is the process of generating outcomes according to a specified probability distribution.
- How it is used
- Uses repeated measurement to generate distribution samples, interpret counts, and extract problem-relevant information from quantum states.
- How deep it goes
- Explains distribution encoding through state amplitudes, repeated measurement, count interpretation, and Python simulation without formal derivations.
- Examples
- Uses a Galton board, a single-qubit circuit, Python-generated samples, uniform random values, and good-versus-bad marble populations.
- What you can do after
- Apply it via Python's random.choices to simulate measurements and use repeated counts to approximate outcome probabilities.
Read how Building Quantum Software with Python explains sampling →
Outlier Detection in Python
Sampling is the selection of a subset of observations from a larger dataset for analysis or modeling.
- How it is used
- Reduces model-training cost and sometimes prediction cost when datasets are too large for practical full-data processing.
- How deep it goes
- Explains sample-size trade-offs, synthetic-outlier testing, prediction exceptions, and normal-class coverage without prescribing a universal size.
- Examples
- Uses hundreds or thousands of training records, synthetic outliers, 256-point Isolation Forest samples, and full normal-class coverage.
- What you can do after
- Apply it via sampling-based outlier detectors while measuring reliability, preserving legitimate variation, and balancing speed against accuracy.
Graph Neural Networks in Action
Sampling is the selection of a subset of data elements for computation or analysis.
- How it is used
- Makes graph neural network training feasible when the full graph exceeds available processing or memory capacity.
- How deep it goes
- Explains subset selection, PyG implementations, approximation, risks, and sampler choice without detailing sampling algorithms internally.
- Examples
- Uses PyG NeighborSampler with GCNConv and contrasts sampled subgraphs with full-batch graph convolution.
- What you can do after
- Apply it via PyTorch Geometric loaders and samplers to train GNNs on graphs that do not fit comfortably in memory.
Read how Graph Neural Networks in Action explains sampling →
Machine Learning Platform Engineering
A sampling strategy is a rule for selecting generated tokens from a model's probability distribution.
- How it is used
- Controls LLM response variety and determines how production systems handle probabilistic outputs.
- How deep it goes
- Explains temperature, top-K, and top-P behavior with parameter examples, but stops short of implementation internals.
- Examples
- Uses temperature values from 0.0 to 1.5, top-K values from 1 to 100, and top-P values from 0.1 to 1.0.
- What you can do after
- Operate/configure sampling parameters and replace exact-output tests with semantic, factual, and guideline-based evaluation.
Read how Machine Learning Platform Engineering explains sampling →
Statistics Every Programmer Needs
Random sampling is a sampling procedure in which selections are made according to a probability mechanism rather than a deterministic choice.
- How it is used
- Uses probability-driven scenarios to analyze uncertainty, variability, risk, staffing, and future prices.
- How deep it goes
- Explains the simulation workflow from model definition through repeated trials and result analysis, but does not provide implementation code.
- Examples
- Demonstrates absenteeism counts, Poisson outcomes, call-center overstaffing, and simulated future stock prices.
- What you can do after
- Apply random sampling to construct scenarios, compare simulated and theoretical distributions, and interpret uncertainty for operational or investment decisions.
Read how Statistics Every Programmer Needs explains sampling →
Bayesian Optimization in Action
Uniform sampling draws points so that equal-size regions of the sampling domain have equal probability.
- How it is used
- Uses uniform random points as a baseline for objective optimization and draws possible function realizations from Gaussian-process predictions.
- How deep it goes
- Defines uniform sampling, illustrates clustering and wasted evaluations, and demonstrates GP sampling in Python, stopping short of sampling theory.
- Examples
- Uses bounded objective search, clustered random evaluations, and five plotted samples from a zero-mean Gaussian process.
- What you can do after
- Apply it via random search for bounded objectives and use GPyTorch predictive distributions to generate reproducible function samples.
Read how Bayesian Optimization in Action explains sampling →
Fabulous Adventures in Data Structures and Algorithms
Sampling is the process of generating a random outcome according to a probability distribution.
- How it is used
- Uses probability distributions to generate random outcomes for games, simulations, sequences, and uncertainty-aware decisions.
- How deep it goes
- Develops PMFs, PDFs, Markov transitions, and Metropolis sampling, while deferring detailed algorithmic implementations for particular distributions.
- Examples
- Uses colored marbles, dice and simulated events, beta-shaped densities, dart-area geometry, random text, and posterior distributions.
- What you can do after
- Apply sampling to generate discrete outcomes, continuous values, Markov sequences, and Metropolis-based posterior samples.
Read how Fabulous Adventures in Data Structures and Algorithms explains sampling →
How Large Language Models Work
Sampling is the computational process of selecting output tokens from a probability distribution over vocabulary items.
- How it is used
- Explains how autoregressive language models convert next-token probabilities into generated text.
- How deep it goes
- Traces probability calculation, weighted selection, repetition, randomness, and stopping without detailing sampling variants.
- Examples
- Uses “I love to eat,” weighted food-token choices, repeated prompts, and an end-of-sequence stopping example.
- What you can do after
- Evaluate why outputs vary and adjust randomness when choosing between more consistent or more creative generation.
Effective Data Analysis
Random selection is a sampling procedure in which participants or subjects are chosen from a population using chance-based selection.
- How it is used
- Supports experiments by selecting participants or subjects from a population before testing causal hypotheses.
- How deep it goes
- Defines pure, stratified, and clustered selection, but does not detail implementation or sample-size decisions.
- Examples
- Uses experimental participants, population subgroups, and randomly selected groups as concrete selection examples.
- What you can do after
- Apply pure, stratified, or clustered selection when designing an experiment.
Deep Learning with R, Third Edition
Sampling is the process of drawing values or instances from a probability distribution or representation space.
- How it is used
- Generates text by choosing each next token from a language model’s predicted probability distribution.
- How deep it goes
- Explains greedy, random, temperature-scaled, and top-k strategies with runnable examples, but does not develop formal decoding theory.
- Examples
- Uses generated text at temperatures 2, 0.8, and 0.2, alongside top-k runs with k=5 and k=20.
- What you can do after
- Apply it via R and Keras 3 to generate text with greedy, random, temperature-scaled, or top-k choices.
Read how Deep Learning with R, Third Edition explains sampling →
AI Agents in Action, Second Edition
- How it is used
- Uses sampling to select the next token during LLM text generation.
- How deep it goes
- Explains token selection and parameter trade-offs, while noting that settings influence output rather than guarantee it.
- Examples
- Contrasts coder-agent settings with creative-writer settings and depicts token selection in an LLM generation diagram.
- What you can do after
- Configure generation behavior by tuning temperature and max tokens, while treating parameter effects as non-guaranteed.
Read how AI Agents in Action, Second Edition explains sampling →
Machine Learning System Design
Sampling is the process of selecting a subset of observations from a larger dataset for analysis or model training.
- How it is used
- Improves dataset efficiency when automatically generated data is huge, imbalanced, or full of duplicates.
- How deep it goes
- Defines stratified selection and prioritization but does not develop specific sampling algorithms.
- Examples
- Uses clickstreams, feature-based groups, algorithmic clusters, and fresh-versus-old temporal groups.
- What you can do after
- Apply stratification to reduce redundant data and emphasize groups relevant to the modeling problem.
Delivering Machine Learning Projects: From design to deployment
Sampling is the process of selecting a subset of observations from a larger population or data set.
- How it is used
- Guides data-set construction during exploratory data analysis before modeling.
- How deep it goes
- Warns against random selection and uses plots to inspect distribution, without defining formal sampling methods.
- Examples
- Uses restaurant tips and bills plus temperature readings distributed across days of the year.
- What you can do after
- Evaluate whether a sample preserves distributional information needed by modeling algorithms.
Read how Delivering Machine Learning Projects: From design to deployment explains sampling →
Related concepts
- heuristickind of · A heuristic is a practical rule or approximate method for guiding judgment, search, or problem solving without guaranteeing an exact or optimal result.
- Sobol sequencealternative to · A Sobol sequence is a low-discrepancy quasi-random sequence designed to distribute points evenly across a multidimensional domain.
- top-k samplingincludes · Top-k sampling is a decoding technique that restricts each next-token selection to the k candidates with the highest probabilities, then samples from those candidates.
- probability distributionuses · A probability distribution is a mathematical specification assigning probabilities to the possible values or outcomes of a random variable, with total probability one.
- top-p filteringincludes · Top-p filtering is a token-sampling technique that selects from the smallest probability-ranked set of candidates whose cumulative probability reaches a specified threshold.
- statistical sampleproduces · A statistical sample is one or more observations selected or generated from a population or probability process for analysis.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.