A timestep is one discrete position in an ordered sequence and one update of a recurrent computation. At timestep t, an RNN combines the previous hidden state hₜ₋₁ with the current input xₜ to produce hₜ, a summary of everything seen so far. section 4.1.2
In an RNN implementation:
- T is the total number of timesteps, while each xₜ is a same-sized feature vector. section 4.1.2
- PyTorch commonly represents sequence batches as (B, T, D): batch size, timesteps, and features per timestep. section 4.2
Weather sequence
For daily weather, xₜ can contain the high, low, and mean temperature for day t. Processing days in natural order lets hₜ summarize that day and all preceding days; h_T can support a prediction for the entire sequence. section 4.1.2
