liveBook · Manning

Active data collection

As explained in Grokking Bayes

Active data collection is a strategy in which the data-gathering actions or observations are selected to maximize information or reduce uncertainty.

Active data collection chooses what to observe or ask next so the result reduces uncertainty as much as possible. Bayesian experimental design makes this choice by comparing expected information gain. section 13.7

The strategy proceeds by:

  • Considering what each possible observation would imply about the unknown truth. section 13.7
  • Quantifying uncertainty before and after each observation with entropy. section 13.7
  • Selecting the observation with the lowest expected remaining uncertainty. section 13.7

Finding an unknown number

For an unknown integer from 1 to 100, asking whether it exceeds 50 splits the possibilities evenly and gains one bit of information. Repeating midpoint questions produces binary search, which identifies the number in at most seven questions and generally outperforms random guessing. section 13.7

def eig_for_threshold(x, N):
    p_left = x / N            #A
    p_right = (N - x) / N     #A
    
    H_left = entropy(x)       #B
    H_right = entropy(N - x)  #B
    
    H_post = (
        p_left * H_left 
        + p_right * H_right
    )                         #C
    
    H_prior = entropy(N)      #D
    
    return H_prior - H_post   #E

A Python function computes expected information gain for a threshold by weighting the entropy of each possible outcome. section 13.7

How Grokking Bayes covers it

How deep it goes
Defines the strategy through entropy and expected information gain, then demonstrates it with an iterative binary-search calculation.
Examples
Uses an unknown integer from 1 to 100, midpoint thresholds, entropy calculations, and comparisons with random guessing.
What it assumes
Starts with possible values and a uniform prior, then introduces entropy and information gain through a concrete search problem.
What you can do after
Apply it via expected information gain to choose data-collection actions that minimize remaining uncertainty.
How it connects
Connects Bayesian experimental design with entropy, expected information gain, priors, posterior belief, and binary search.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.