liveBook · Manning

Language acquisition

As explained in How Large Language Models Work

The process through which a person develops the ability to understand and use a language through exposure, interaction, and communication.

Language acquisition is the gradual development of understanding and use through interaction, conversation, and communication with people and the surrounding environment. Children build from simple words such as “mama” and “dada” toward more complex vocabulary and concepts. section 7.1

This process differs sharply from how a large language model learns. A child can become fluent after exposure to far fewer words, while an LLM trains on enormous datasets and encounters words largely according to their frequency rather than through a staged developmental sequence. Conversation and lived interaction therefore provide an important contrast when judging what LLM learning can achieve. section 7.1

Useful comparison points include:

  • Children may hear around 15,000 spoken words per month and develop fluent native-language use by about age 18. section 7.1
  • GPT-3 trained on hundreds of billions of words, illustrating the much larger data volume used by LLMs. section 7.1
  • LLMs tokenize language, including written material, rather than acquiring words through human conversation. section 7.1
Figure 7.1 A summary of the strengths and weaknesses of LLMs relative to humans performing the same task. These lead to natural considerations that you must evaluate when using an LLM. From these, we can draw broad recommendations for successful LLM use.
figure

Use the comparison figure to weigh human and LLM strengths and weaknesses before relying on an LLM for a task. section 7.1

How How Large Language Models Work covers it

How deep it goes
Explains interaction, data volume, vocabulary sequence, and tokenization without developing a full learning algorithm.
Examples
Uses children’s early words, monthly word exposure, GPT-3’s training corpus, and book tokenization as comparison examples.
What you can do after
Evaluate LLM capabilities more cautiously by distinguishing fluent output from human-like language learning.
How it connects
Connects language acquisition with conversation, interaction, tokenization, fine-tuning, and reinforcement learning from human feedback.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.