Extractive summarization builds a shorter text by selecting important words, phrases, or sentences directly from the source. It preserves the source wording rather than inventing new sentences, making factual accuracy and traceability central concerns. section 6.3.2 section 6.3.4
Use the technique according to the text and the level of detail required:
- Summarize short news articles or blog posts by identifying and compiling their most informative sentences. section 6.3.4
- Apply sentence scoring with position, sentence length, keyword presence, or term frequency to rank candidate sentences. section 6.3.4
- Prefer a hybrid with abstractive summarization for scientific or technical documents when selected sentences need synthesis into a coherent account. section 6.3.4
- Extract keywords or topics instead of forcing a summary from very short customer reviews; combine them with sentiment analysis when useful. section 6.3.4
Frequency-based review summary
A practical workflow loads customer reviews, removes empty records, tokenizes Portuguese text, removes punctuation and stopwords, counts word frequencies, and returns key words from a review. For example, “I received it well before the stipulated deadline” becomes “received well before.” This is concise and source-grounded, but it can omit relationships and produce awkward fragments. section 6.3.4 section 6.3.4
Listing 6.6 Summarizing the longest review—a frequency-based approach
import pandas as pd
import nltk
import string
from collections import Counter
# Download the required NLTK resources
nltk.download('punkt')
nltk.download('stopwords')
# Sample data (manually adapted to remove empty records)
df = pd.read_csv('olist_order_reviews_dataset.csv')
df = df.dropna(subset = ['review_comment_message'])
# Function to tokenize and remove stopwords
def preprocess(text):
stopwords = nltk.corpus.stopwords.words('portuguese')
tokens = nltk.word_tokenize(text.lower())
tokens = [token for token in tokens if token not in string.punctuation
and token not in stopwords]
return tokens
# Function to create word frequency distribution
def word_frequency(tokens):
frequency = Counter(tokens)
return frequency
# Function to summarize short reviews
def summarize_reviews(text, num_keywords=3):
tokens = preprocess(text)
frequency = word_frequency(tokens)
important_words = [word for word, count in
frequency.most_common(num_keywords)]
summary = ' '.join(important_words)
return summary
# Apply the function to the DataFrame
df['summary'] = df['review_comment_message'].apply(summarize_reviews)
# Display the results (manually adapted to print the summary of the longest
message)
print("Longest review:", df.loc[1316]["review_comment_message"])
print("Summary:", df.loc[1316]["summary"])
Inspect the Python workflow for loading reviews, preprocessing Portuguese text, calculating word frequencies, and producing a short extractive result. section 6.3.4
Table 6.5 Summary efforts on a sample of five customer reviews
|
Translated review
|
Three-word summary by frequency-based methods
|
Three-word summary by ChatGPT 4
|
ChatGPT’s haiku
|
|---|---|---|---|
|
I received it well before the stipulated deadline.
|
received
well
before
|
Early delivery
|
Arrived well before
Stipulated deadline nears
Satisfaction found
|
|
Congratulations Lannister stores, I loved shopping online, safe and practical. Congratulations to everyone and happy Easter.
|
congratulations
lannister
stores
|
Lannister stores praised
|
Lannister stores cheer
Safe and practical shopping
Happy Easter joy
|
|
Efficient device. On the website, the device brand is printed as 3desinfector, and upon arrival, it has another name... Update with the correct brand since it’s the same device.
|
efficient
brand
device
|
Efficient, mislabeled device
|
Efficient device
Misnamed but still performs well
Update with true brand
|
|
But a little, lagging... for the price, it’s good.
|
a bit
lagging
...
|
Lagging, decent value
|
Slight lag appears,
Yet value stands its ground,
Acceptable deal.
|
|
Reliable seller, product ok, and delivery before the deadline
|
reliable
seller
product
|
Reliable seller, prompt
|
Trustworthy vendor,
Product meets expectations,
Early delivery.
|
Compare frequency-based three-word outputs with ChatGPT-generated summaries and haiku for five customer reviews. section 6.3.4
