A probability mass function (PMF) assigns a probability to each possible value of a discrete random variable. In extractive question answering, tokens are discrete values, so a model can estimate the probability that each token starts or ends an answer span. section 14.1.1
A question-answering model uses PMFs to turn token-level scores into an answer:
- Compute one PMF over context tokens for possible answer starts. section 14.1.1
- Compute a second PMF over context tokens for possible answer ends. section 14.1.1
- Choose the most likely start and end tokens to extract the answer span. section 14.1.1
Extracting an answer span
Given the question “What are minimalist shoes?” and a context describing them, the model assigns start and end probabilities across the tokenized context. Selecting the strongest boundaries produces an answer such as “shoes intended to closely approximate barefoot running conditions.” section 14.1.1
