Expressive power is the range and complexity of patterns a model can represent. In a language model, learned parameters shape this capacity by encoding constraints on how input features interact—not by literally storing sentences, rules, or facts. section 4.2.1
Width gives a model room to represent more features at each step; depth lets it combine and transform those features across layers. Together, they affect how finely the model can distinguish possible inputs. Greater capacity can preserve distinctions that a limited model would merge, raising the ceiling on patterns it can learn. section 4.2.1
Parameter count is only an approximate measure of scale, not a guarantee of capability. In large dense models, computation can become limiting before representational capacity is fully used; selective activation instead lets specialized components contribute when relevant. section 4.2.1 section 4.4












