A variable-length sequence contains a changing number of ordered items across examples. Unlike an image, it cannot be resized without potentially changing its meaning. Represent each item as a vector and use a sequence model that preserves order; T denotes the number of items. section 4.0
Typical representations and handling include:
- Treat sentences as word or character sequences, hourly temperatures as time series, and videos as sequences of image vectors. section 4.0
- Use recurrent neural networks when inputs have differing lengths and order matters. section 4.0
- Use attention to process variable-length inputs, and apply masking when batching them. section 10.5
Different values of T
The sentences “the two previous sentences” contain 18 and 8 words, so their sequence lengths are T = 18 and T = 8. Sentiment detection can classify such word sequences. section 4.0
