Stackability describes a neural architecture whose layers preserve the same input and output structure, allowing repeated composition to increase capacity. Transformer layers achieve this uniform interface while combining attention, byte pair encoding, and positional encoding. section 9.1.1 section 9.5
The design matters because repeated layers can scale across larger datasets, while their matrix-heavy attention computations remain parallelizable. Convolutional networks provide a contrast: hierarchical convolution layers usually change tensor shapes, making them harder to stack and scale. section 9.1.1 section 9.2
Position-aware transformer layers
Positional encoding spreads location information through the representation, so the word “sincerely” can mean something different at the beginning of an email than at its end while passing through uniformly structured layers. section 9.1.1
Stackability supports large transformer systems for text generation and language understanding. GPT is associated with generation tasks such as translation and chatbots, while BERT remains suited to many natural-language-understanding problems. section 9.5
