Sampling generates an outcome by drawing from a probability distribution. “Generate” and “simulate” mean the same thing here. A computer typically uses a pseudo-random number generator: a seed starts a deterministic sequence that emulates random draws. The outcome can vary between runs even when the distribution stays fixed. section 2.2.2
When the distribution models a population, a draw can represent the selection of one individual. Repeating that selection produces a random sample. Data is independent and identically distributed (IID) when each point is selected the same way, independently of the others, from the same population distribution. Repeated observations alone do not establish those conditions. section 2.3.1
Generate outcomes in Python
A pgmpy DiscreteFactor with probabilities [0.45, 0.30, 0.25] for three states can produce one outcome with dist.sample(n=1), returned as a one-row pandas DataFrame. The same method can draw from a joint distribution or generate multiple rows. In Pyro, Categorical(probs=torch.tensor([0.45, 0.30, 0.25])).sample() draws a category; pyro.plate provides a way to express repeated draws in a model. section 2.2.2 section 2.3.1
Listing 2.10 Generating IID samples in Pyro
import pyro
from pyro.distributions import Bernoulli, Poisson, Gamma
def model():
z = pyro.sample("z", Gamma(7.5, 1.0))
x = pyro.sample("x", Poisson(z))
with pyro.plate("IID", 10): #1
y = pyro.sample("y", Bernoulli(x / (5+x))) #2
return y
model()
The Pyro listing places repeated Bernoulli draws inside a plate while drawing their parameters through preceding model steps. section 2.3.1
Sampling also connects a data-generating process (DGP) to observed data. A DGP entails a full joint distribution; marginalizing latent variables yields an observational joint distribution, and sampling from it yields data and an empirical distribution. The empirical proportions need not equal the underlying probabilities: five observations cannot identify a unique observational distribution, much less a unique causal process. section 2.3.2
