Data transformation maps each input record, or each partition handled by a machine, into intermediate information needed by a later computation. In MapReduce, the map stage extracts relevant facts rather than producing the final aggregate. section 13.2.2
A useful transformation:
- Receives a dataset entry or partition. section 13.2.2
- Extracts information relevant to the final result. section 13.2.2
- Passes intermediate output to a later reduction step, which groups homogeneous entries and combines them. section 13.2.2
Recognizing coins before counting
A mapper can run coin-recognition software on its assigned data, record each detected denomination, and send unsorted occurrences onward. Reducers then count, sum, or filter entries for each denomination. section 13.2.2
$100: 1 50¢: 1 $100: 1 $1: 1 25¢: 1 ...
An intermediate occurrence list makes the transformation visible before reduction: each detected denomination appears as a separate mapped entry. section 13.2.2
