liveBook · Manning

Rule-based system

As explained in Natural Language Processing in Action, Second Edition

A rule-based algorithm determines outputs by applying explicitly designed human-authored rules to input data.

A rule-based system produces outputs by applying human-designed heuristics to input data. In sentiment analysis, it maps words, stems, lemmas, or n-grams to numerical scores and combines those scores into a positivity value. section 2.6

A basic rule-based sentiment pipeline follows these steps:

  • Create a hand-authored dictionary that assigns sentiment weights to selected tokens. section 2.6
  • Tokenize the document, optionally using stems, lemmas, or n-grams as dictionary keys. section 2.6
  • Add the matching token scores to produce a value from −1 for strongly negative text to +1 for strongly positive text. section 2.6

Neutral sentiment

The statement “It was OK. There were some good and some bad things.” illustrates a mostly neutral result near zero, such as +0.1 or −0.1. section 2.6

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Explains keyword dictionaries and score summation, but defers the detailed VADER implementation.
Examples
Uses a sentiment dictionary, weighted tokens, and the neutral statement “It was OK.”
What it assumes
Assumes tokenization knowledge and introduces how token forms can become rule-lookup keys.
What you can do after
Build a sentiment scorer by composing a weighted token dictionary and summing matching scores.
How it connects
Connects sentiment analysis with tokenization, stemming, lemmatization, n-grams, and machine learning.

Also explained in

Compare how 6 books explain rule-based system →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.