liveBook · Manning

Topic

As explained in Math and Architectures of Deep Learning

A latent group of semantically related terms represented as a direction or subspace in document feature space.

A topic is a latent group of semantically related terms, such as gun and violence, whose relationship may be missed by direct word overlap. Documents can therefore be compared through shared topics rather than only shared vocabulary. section 4.6.2

Geometrically, a topic is a direction or linear subspace in document feature space: a weighted combination of vocabulary dimensions. In classical latent semantic analysis, useful topics are principal components with high variance, because they discriminate documents; low-variance topics are pruned. section 4.6.2

Gun and violence

Documents containing only gun can still resemble documents containing only violence when both terms frequently co-occur elsewhere. Topic-based similarity captures this secondary semantic evidence that cosine similarity on explicit document vectors misses. section 4.6.2

Figure 4.6 Document vectors from our toy dataset d0, ⋯ d6. Each word in the vocabulary corresponds to a separate dimension. Dots show projections of document feature vectors on the plane formed by the axes corresponding to the terms gun(s) and violence.

Inspect document-vector projections onto the gun and violence axes to visualize their geometric relationship. section 4.6.2

How Math and Architectures of Deep Learning covers it

How deep it goes
Defines topics semantically and geometrically, then ties them to principal components, variance, discrimination, and pruning.
Examples
Uses gun and violence, a toy document matrix, and projections of document vectors onto vocabulary axes.
What it assumes
Assumes document feature vectors and cosine similarity, while developing the topic concept from semantic and geometric intuition.
What you can do after
Evaluate document similarity through latent semantic structure when exact vocabulary overlap is insufficient.
How it connects
Connects topics to latent semantic analysis, cosine similarity, document vectors, principal components, and variance.
Cover of Math and Architectures of Deep Learning
liveBookmanning.com
Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

Compare how 2 books explain topic →

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.