Topic modeling infers recurring topics from a document collection by identifying words that tend to co-occur, then assigns each document scores describing its relationship to each topic. section 9.3.2
Use topic-related features to:
- Find documents that are unusual because they discuss rarely covered topics. section 9.3.2
- Identify unusual combinations between topics and other document features. section 9.3.2
- Cluster strings according to how similarly they cover the discovered topics. section 9.3.3
Interpreting discovered topics
Expense comments containing “dinner,” “restaurant,” and “client” may form one topic, while “taxi,” “cab,” “hotel,” and “plane” may form another. Topic names are inferred from co-occurring words rather than supplied in advance, so results may differ from expected categories. section 9.3.2
Listing 9.10 Using BERTopic with the 20 Newsgroups dataset
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups
data = fetch_20newsgroups(subset='all',
remove=('headers', 'footers', 'quotes'))
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(data['data'])
Run BERTopic on the 20 Newsgroups dataset to produce topic assignments and probabilities. section 9.3.2
