liveBook · Manning

Vector program

As explained in Deep Learning with R, Third Edition

A vector program is a highly nonlinear learned mapping that transforms points within a latent embedding space.

A vector program is a highly nonlinear learned mapping that transforms points within a latent embedding space. Transformer models can store millions of such programs, far beyond simple linear relationships learned by word2vec. section 15.5

From vector arithmetic to generated style

Word2vec supports mappings such as plural(cat) → cats and male_to_female(king) → queen. A large Transformer can instead apply a complex program such as write_this_in_style_of_Shakespeare("…your poem…") → "…new poem…". section 15.5

The model behaves like a continuous, interpolative database: tokens retrieve locations in a semantic vector space, while movement and interpolation can produce related or intermediate results. This supports generalization but can also produce inaccurate or meaningless outputs, including hallucinations. section 15.5

How Deep Learning with R, Third Edition covers it

How deep it goes
Explains nonlinear mappings, interpolation, and generalization, but stops short of implementation details or formal derivations.
Examples
Uses pluralization, gender transformation, Shakespearean rewriting, and continuous database retrieval as examples.
What it assumes
Assumes familiarity with word2vec and semantic vector spaces while contrasting their simple transformations with Transformer programs.
What you can do after
Evaluate/understand how Transformer representations support complex transformations and why interpolation can hallucinate.
How it connects
Connects vector programs to Transformers, word2vec, vector spaces, autoregressive optimization, interpolation, and hallucination.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.