gensim is a Python library for working with unsupervised word-vector representations. It loads pretrained Word2Vec models, exposes similarity and analogy queries, and supports domain-specific training. section 6.3.4 section 6.3.5
Use gensim to:
- Load Google News vectors from the original binary format, optionally restricting the vocabulary to reduce memory use during development. section 6.3.4
- Inspect vectors, measure similarity, find nearest neighbors, identify an outlier, and combine positive and negative terms for analogies. section 6.3.4
- Train a domain-specific Word2Vec model from tokenized sentences when general-domain vectors miss specialized language. section 6.3.5
Domain-specific training workflow
Prepare a corpus as sentences containing tokens, set vector size, minimum word count, worker count, context window, and subsampling, then instantiate and save a Word2Vec model. Reloading the saved model enables queries such as model.most_similar('radiology'). section 6.3.5
>>> word_vectors.most_similar(positive=['cooking', 'potatoes'], topn=5)
[('cook', 0.6973530650138855),
('oven_roasting', 0.6754530668258667),
('Slow_cooker', 0.6742032170295715),
('sweet_potatoes', 0.6600279808044434),
('stir_fry_vegetables', 0.6548759341239929)]
>>> word_vectors.most_similar(positive=['germany', 'france'], topn=1)
[('europe', 0.7222039699554443)]
Explore nearest-neighbor queries that combine terms, including cooking with potatoes and Germany with France. section 6.3.4
