An out-of-bag observation is a training case omitted from a tree’s bootstrap sample. With N draws from N observations, a particular case is omitted with probability (1 − 1/N)^N, which approaches 36.8% as N grows; about 63.2% appear at least once. section 9.2.1
For each training case, predict using only trees whose bootstrap samples omitted it. Combine those predictions by majority vote for classification or averaging for regression, then compare with the known outcome. The resulting predictions support error estimates and other performance summaries; they also provide a basis for permutation importance. Out-of-bag evaluation is internal validation, not a reason to forgo an independent test set after model selection and tuning. section 9.5 section 9.6 section 9.9
A 500-tree forest
A case is omitted from about 36.8% of 500 independently sampled trees—roughly 184, though the actual count varies. Those trees alone contribute its out-of-bag prediction. section 9.5

