Polarity is a numerical value that represents whether text expresses favorable, unfavorable, or neutral sentiment. A sentiment lexicon assigns polarity scores to words, which can then contribute to a review-level score. section 5.6.4
Choose the lexicon representation according to the analysis goal:
- Use a flexible-form lexicon when grammatical categories and part-of-speech information matter. section 5.6.3
- Use a lemma lexicon when matching base forms and estimating overall text sentiment is more important. section 5.6.3
- Compare both representations when the better fit for the dataset or task is uncertain. section 5.6.3
Reading a SentiLex polarity entry
An entry such as “à-vontade.PoS=N;TG=HUM:N0;POL:N0=1;ANOT=MAN” contains a polarity field whose value can be extracted as the word’s score. A Python loader can parse the text file line by line, isolate the word and POL:N0 value, and build a dictionary for scoring tokens. section 5.6.4
def load_sentilex(filepath):
sentilex_dict = {}
with open(filepath, 'r', encoding='utf-8') as file:
for line in file:
parts = line.strip().split(';')
word = parts[0].split('.')[0] # Get the word before '.PoS='
pol_entry = [part for part in parts if
part.startswith('POL:N0=')]
if pol_entry:
# Assuming there's exactly one 'POL:N0=' entry per line
polarity = int(pol_entry[0].split('=')[1])
sentilex_dict[word] = polarity
return sentilex_dict
# Example usage
filepath = 'SentiLex-lem-PT02.txt'
sentilex_dict = load_sentilex(filepath)
# Now sentilex_dict is ready to be used as in the previous example.
Inspect a file loader that extracts SentiLex words and POL:N0 values into a scoring dictionary. section 5.6.4
