liveBook · Manning

Lemma

As explained in Starting Data Analytics with Generative AI and Python

A lemma is the canonical dictionary form representing a set of inflected word forms.

A lemma is the canonical dictionary form representing related inflected words. Adjectives commonly use masculine singular forms, countable nouns singular forms, and verbs infinitives. section 5.6.5

Dictionary-based sentiment search works best when reviews and lexicons use the same representation. If a lexicon contains lemmas but preprocessing keeps surface word forms, sentiment detection can miss matches and show low sensitivity. section 5.6.5

Matching a Portuguese sentiment lexicon

Choose a lemma-based resource such as SentiLex-lem-PT02.txt when appropriate, then use spaCy’s Portuguese model to normalize each review, extract alphabetic lemmas, remove stop words, and compare the resulting list with the lexicon. Inspect the output and validate sentiment results against sample reviews, because language morphology and resource differences can affect performance. section 5.6.5 section 5.6.3

pip install spacy
python -m spacy download pt_core_news_sm

Install spaCy and download the Portuguese language model required for the lemmatization workflow. section 5.6.5

How Starting Data Analytics with Generative AI and Python covers it

How deep it goes
Defines canonical forms and demonstrates Portuguese lemmatization, but does not detail morphological internals.
Examples
Uses Portuguese SentiLex files and a spaCy preprocessing function that returns lemmas from customer reviews.
What it assumes
Assumes a dictionary-based search workflow and teaches the need to match preprocessing forms with lexicon entries.
What you can do after
Apply it via spaCy to make review tokens compatible with a lemma-based sentiment lexicon.
How it connects
Connects lemmatization with inflection, stop-word removal, lexicons, part-of-speech tags, and sentiment analysis.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Starting Data Analytics with Generative AI and Python. The section links point to the parts of the book it draws on.