An RNN is sensitive to the order in which it reads a sequence. A bidirectional RNN uses two recurrent passes—one forward and one backward—and merges their representations. This can reveal patterns that a single direction misses. For temperature forecasting, reversing an LSTM’s input order made it perform worse than a commonsense baseline: direction matters, even before combining directions. section 13.3.6
Listing 13.24 Training and evaluating a bidirectional LSTM
inputs = keras.Input(shape=(sequence_length, raw_data.shape[-1])) x = layers.Bidirectional(layers.LSTM(16))(inputs) outputs = layers.Dense(1)(x) model = keras.Model(inputs, outputs) model.compile(optimizer="adam", loss="mse", metrics=["mae"]) history = model.fit( train_dataset, epochs=10, validation_data=val_dataset, )
A forecasting model wraps an LSTM in `layers.Bidirectional`, then feeds the merged result to a Dense prediction layer. The wrapper changes how the sequence is read without requiring a different model-building pattern. section 13.3.6
Choose the direction of information flow to match the prediction:
- For whole-sequence classification, information can flow both ways. A bidirectional LSTM is used to classify movie reviews. section 14.5.1
- For sequence-to-sequence translation, the source encoder can read both ways to represent the entire input. The decoder must read only earlier target tokens when predicting the next one. section 15.2.2
- For next-token generation, do not give a prediction access to later tokens. Bidirectionality can make training look successful while breaking generation. section 15.1.2
Listing 15.11 Building a sequence-to-sequence encoder
embed_dim = 256 hidden_dim = 1024 source = keras.Input(shape=(None,), dtype="int32", name="english") x = layers.Embedding(vocab_size, embed_dim, mask_zero=True)(source) rnn_layer = layers.GRU(hidden_dim) rnn_layer = layers.Bidirectional(rnn_layer, merge_mode="sum") encoder_output = rnn_layer(x)
A translation encoder wraps a GRU with `layers.Bidirectional(rnn_layer, merge_mode="sum")`. Its merged output represents the source sentence and becomes the initial state of a unidirectional decoder. section 15.2.2
When accuracy is misleading
Replacing a GRU with `Bidirectional(GRU(...))` in a character-by-character generator sends training accuracy above 99%, but generation stops working. Training exposes the whole sequence, so the backward pass lets the model use a future character to predict the current one. In a separate movie-review classifier, a bidirectional LSTM works but trains slowly with one-hot inputs of 600 × 30,000 values per review and reaches about 84% test accuracy. Bidirectionality is not a substitute for choosing suitable inputs or respecting what will be available at prediction time. section 15.1.2 section 14.5.1
