The IMDb dataset supplies polarized movie reviews for binary sentiment classification. Its packaged form contains 50,000 reviews: 25,000 for training and 25,000 for testing, with balanced positive and negative labels. section 4.1.1
The Keras version is preprocessed into integer sequences, so model building, training, and evaluation can begin without implementing raw-text processing. Limiting the vocabulary to the 10,000 most frequent words keeps the vector representation manageable; rarer words are discarded. section 4.1.1
Use the dataset in two practical forms:
- Load integer-encoded reviews with Keras; each review is a sequence of word indices, and labels use 0 for negative and 1 for positive. section 4.1.1
- Process raw reviews for sequence-model experiments by splitting training data into 20,000 training and 5,000 validation reviews, then tokenizing and packing them. section 15.4.3
RoBERTa-ready reviews
A preprocessing pipeline tokenizes each review, trims sequences to 512 tokens, adds start and end tokens, pads shorter sequences, and returns token IDs with a padding mask for training, validation, and testing.
