liveBook · Manning

Patch extractor

As explained in How Large Language Models Work

A patch extractor is an image-processing component that divides an image into fixed-size patches and converts those patches into vector representations.

A patch extractor turns an image into a sequence of small image tiles, then converts each tile into a vector that a transformer can process. It plays a role similar to tokenization for text. section 6.3.1

The conversion proceeds in a few practical steps:

  • Divide the image into fixed-size patches, commonly square tiles such as 16 × 16 pixels. section 6.3.1
  • Represent every pixel with its red, green, and blue values, producing a 768-value vector for each 16 × 16 RGB patch. section 6.3.1
  • Pass each initial vector through a small neural network before sending the sequence to the transformer. section 6.3.1

A tiled image

Treating an image like a sheet of tiles makes the process easier to picture: each tile is individually uninformative, but the sequence of vectors can represent the complete image for the transformer. section 6.3.1

How How Large Language Models Work covers it

How deep it goes
Explains fixed-size tiling, RGB flattening, vector sizing, and neural preprocessing, then stops before implementation details.
Examples
Uses 16 × 16 RGB patches, 768-value vectors, square image tiles, and a reverse-direction patch combiner.
What it assumes
Assumes basic familiarity with images and neural networks while defining patches, RGB vectors, and their text-token analogy.
What you can do after
Understand how images become transformer inputs and how patch-based image outputs are reconstructed.
How it connects
Connects patch extraction to tokenization, embedding, transformers, gradient descent, and patch combining.
Cover of How Large Language Models Work
liveBookmanning.com
Read How Large Language Models Work on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.