Shuffle randomly reorders examples before splitting or training. This prevents contiguous ordering—such as grouping images by object type or digits by class—from biasing the data partition. section 12.2.1 section 5.2.3
Avoiding an ordered split
If digit samples are arranged by class, taking the first 80% for training can leave classes 0–7 in training and classes 8–9 in testing. Randomly shuffling before splitting produces more representative partitions. section 5.2.3
Use shuffle with these practical rules:
- Shuffle before ordinary training/test or training/validation splits when the data order is not meaningful. section 5.2.3
- Do not shuffle forecasting data whose order represents time; otherwise, future examples can leak into training. section 5.2.3
- Remove duplicate or redundant examples before splitting so shuffled training and validation sets remain disjoint. section 5.2.3
- For streaming datasets, use .shuffle(buffer_size) to shuffle elements within a buffer. section 8.2.4
import random metadata = list(filter(lambda x: len(x["boxes"]) <= 4, metadata)) random.shuffle(metadata)
A compact COCO preparation example filters images by annotation count and then calls random.shuffle(metadata) before model training. section 12.2.1
