Vectorization layer
A Keras preprocessing layer that tokenizes text and converts it into numerical representations using a vocabulary and configurable encoding.
Deep Learning with Python, Third Edition
A Keras preprocessing layer that tokenizes text and converts it into encoded vectors such as integer sequences or multi-hot representations.
- How it is used
- Converts raw text into numeric inputs for sentiment classification, character-level language modeling, and English-to-Spanish translation.
- How deep it goes
- Works through vocabulary adaptation, output modes, splitting, sequence lengths, and custom standardization without examining the layer’s internals.
- Examples
- Uses IMDb review vectors, Shakespeare character sequences, and separate English and Spanish translation vocabularies.
- What you can do after
- Apply it via Keras TextVectorization to learn a training vocabulary and prepare multi-hot or integer inputs for downstream models.
Read how Deep Learning with Python, Third Edition explains Vectorization layer →
Deep Learning with R, Third Edition
TextVectorization is a Keras preprocessing layer that transforms text into numerical representations using a learned vocabulary and configurable tokenization.
- How it is used
- Turns review, language-model, and translation text into numerical inputs for Keras models.
- How deep it goes
- Works through vocabulary adaptation, encoding modes, n-grams, and custom standardization, but does not examine the layer’s internals.
- Examples
- Uses IMDb sentiment reviews, Shakespeare next-character prediction, and English-to-Spanish sentence pairs.
- What you can do after
- Apply it via Keras to learn training vocabularies and prepare multi-hot or integer text inputs for models.
Read how Deep Learning with R, Third Edition explains Vectorization layer →
Related concepts
- tokenizationused by · Tokenization is the process of dividing text or other sequential input into discrete units called tokens for computational processing.
- multi-hot encodinguses · Multi-hot encoding represents multiple selected items from a fixed vocabulary with a binary vector whose corresponding positions are set to one.
- tf.data APIused by · The tf.data API is TensorFlow’s API for constructing input pipelines that read, transform, batch, and iterate over data efficiently.
- paddingconfigures · Padding is the addition of extra space, values, or data around or within a structure to satisfy size, alignment, layout, or processing requirements.
- adaptconfigured by · adapt() is a preprocessing-layer method that learns data-dependent internal state, such as a vocabulary or normalization statistics, from supplied data.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.