liveBook · Manning

Dictionary-based method

As explained in Starting Data Analytics with Generative AI and Python

A dictionary-based method analyzes text by matching its terms against a predefined dictionary or lexicon of relevant words and expressions.

Suppose you need to find customer reviews about cold food or soggy fries. A dictionary-based method searches text using a prepared lexicon of relevant terms rather than relying only on keywords you invent for the task. The dictionary is the starting point, not a guarantee that every relevant review will be found. section 5.6

Choose the lexicon before trusting the results:

  • Check whether it covers the subject and includes enough relevant terms and detail. section 5.6.2
  • Check its accuracy, publication date, and fit for current language. section 5.6.2
  • Look for cultural, linguistic, or domain bias. Compare candidate lexicons with pilot tests rather than taking the first search result. section 5.6.2

A lexicon can hold more than a list of words: entries may carry usage tags, regional meanings, or emotion intensity. Such detail can support a richer reading of sentiment and context than simple keyword detection, but its usefulness still depends on the dictionary’s quality. section 5.6.1

When review wording does not match the dictionary

A Portuguese sentiment search showed low sensitivity for positive reviews. The lexicon contained lemmas, but the reviews had not been converted to lemmas before matching. In an iterative AI-assisted Python workflow, the follow-up request was to adapt preprocessing; the proposed change used spaCy’s Portuguese model to extract lemmas from review tokens. Inspect matches against the original reviews after making such a change: matching word forms fixes one possible cause of missed reviews, not every cause. section 5.6.5

How Starting Data Analytics with Generative AI and Python covers it

How deep it goes
Explains lexicon selection and a lemmatization fix, without establishing that matching alone resolves all sentiment errors.
Examples
Uses cold-food and soggy-fries reviews, then a Portuguese sentiment workflow that misses positive reviews.
What it assumes
Builds on keyword generation and review extraction while introducing how to choose an existing lexicon.
What you can do after
Apply it via spaCy to align Portuguese review lemmas with a lexicon, then check whether matching improves.
How it connects
Connects keyword search and sentiment analysis with lexicon selection, text preprocessing, and lemmatization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Starting Data Analytics with Generative AI and Python. The section links point to the parts of the book it draws on.