liveBook · Manning

Memory

As explained in Machines that Think

The capacity to encode, retain, and retrieve information, including information carried between successive sequence positions.

Memory allows past experience to shape present behavior, but similar outcomes can arise through different processes. Human remembering is reconstructive: recall can reactivate and subtly modify the patterns supporting a memory. What is remembered is also shaped by emotion, motivation, prior experience, and the conditions in which the memory formed. section 5.4.3

In a trained neural network, prior experience is embedded in parameters that shape how inputs become outputs. Ordinary use accesses this structure without changing it. The model can produce varied responses, but those responses do not by themselves revise its parameters; changes require a new training process. section 5.4.3

Figure 5.3 Human memory is reconstructive: experiences form memories that are reshaped each time they are recalled, so remembering can modify the structure that supports future recall. Artificial neural networks instead store prior experience in model parameters created during training. During inference these parameters remain fixed, so using the model does not alter its stored knowledge.

The figure contrasts recall that can reshape human memory with fixed model parameters during inference. section 5.4.3

Conversational continuity is distinct from changing a model’s parameters. Prompts and corrections can influence behavior within a conversation, but they do not normally alter the underlying model. Memory systems, fine-tuning, or external learning loops can be added to preserve corrections or accumulate experience. section 9.4.4

A system may recall prior exchanges and, through memory and personalization, make an interaction feel continuous. That continuity should not be confused with a shared relationship: the user may attach meaning to the exchange while the system responds without awareness, commitment, agency, or responsibility. section 8.5.4

How Machines that Think covers it

How deep it goes
Explains reconstructive recall and fixed parameters, but does not detail how added memory systems are implemented.
Examples
Contrasts recall that modifies memory with prompts and corrections that influence a language model within a conversation.
What you can do after
Evaluate apparent memory by distinguishing reconstructed experience, fixed parameters, and conversational continuity.
How it connects
Connects memory to learning, neural network parameters, inference, and personalization.

Also explained in

Compare how 26 books explain memory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.