How deeply does Deep Learning with JAX cover sparse representation?

Defines sparse storage and its tradeoff, but defers the experimental API and implementation details.

Sparse representation

A tensor representation that stores only non-zero values to reduce memory usage.

When a tensor contains many zeros and relatively few non-zero values, storing every element wastes memory. A sparse tensor stores only the non-zero values, which can substantially reduce memory usage. section 3.3.2

Use sparse tensors with care:

Pair sparse storage with linear algebra routines that support sparsity; ordinary dense operations may not exploit the representation efficiently. section 3.3.2
JAX exposes sparse tensors and sparse matrix operations through the experimental jax.experimental.sparse API, whose design may change. section 3.3.2

A mostly empty tensor

A tensor representing sparse data may contain many zero-valued elements and only a small number of non-zero ones. A sparse representation keeps the non-zero values instead of explicitly storing the zeros. section 3.3.2

How Deep Learning with JAX covers it

liveBook manning.com

Read Deep Learning with JAX on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Natural Language Processing in Action, Second EditionA sparse representation stores only nonzero or otherwise present values instead of explicitly storing the many default values in a larger representation.
liveBook manning.com

Graph Neural Networks in ActionSparseTensor is the torch_sparse sparse multidimensional tensor structure that stores values and indices rather than a full dense array.
liveBook manning.com

Starting Data Analytics with Generative AI and PythonA sparse matrix is a matrix data structure that stores primarily nonzero elements without allocating equivalent storage for the many zero elements.

Compare how 4 books explain sparse representation →

Related concepts

CountVectorizerused by · CountVectorizer is a scikit-learn transformer that converts text documents into a matrix of token-occurrence counts.
arraykind of · An array is an indexed, ordered collection of elements arranged in one or more dimensions, with elements accessed by position.
dense matrixcontrasts with · A dense matrix is a matrix representation that explicitly stores a value for every position, including positions containing zeros.
pandasused by · pandas is an open-source Python library that provides labeled data structures and tools for manipulating and analyzing tabular and other structured data.
dense representationalternative to · A dense representation explicitly stores values for most or all dimensions or positions, including zero or otherwise uninformative values.
scikit-learnused by · scikit-learn is an open-source Python library providing tools for classical machine learning, including algorithms, preprocessing, model selection, and evaluation.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with JAX. The section links point to the parts of the book it draws on.