liveBook · Manning

Summarization

As explained in Starting Data Analytics with Generative AI and Python

Text summarization is the generation of a shorter text that preserves the important information and key ideas from a source text.

What do customers complain about most often? First identify the negative reviews; then condense them into something a decision-maker can read quickly. Text summarization turns longer text into a shorter, coherent account that retains essential information. It can also make reports, support messages, and research easier to review. section 6.3 section 6.3.1

Choose an output that fits the source:

  • An extractive summary selects important words or sentences from the original. Rule-based scoring can use sentence position, length, or frequent terms. section 6.3.4
  • An abstractive summary conveys the main ideas in new wording. Generative AI can produce one, but may drift toward rewriting even when asked to extract. section 6.3.2
  • For a short customer review, keywords or topics—possibly paired with sentiment analysis—may be more useful than a miniature prose summary. section 6.3.4

Start with a review you can check

Select a negative review using a review score, then prompt ChatGPT to summarize it. One example uses a Portuguese complaint about a product not arriving and asks for a Portuguese summary plus English translations of both the review and summary. Compare those outputs with the original: a shorter answer is useful only if it preserves the complaint. section 6.3.3

Listing 6.6 Summarizing the longest review—a frequency-based approach
import pandas as pd
import nltk
import string
from collections import Counter
# Download the required NLTK resources
nltk.download('punkt')
nltk.download('stopwords')
# Sample data (manually adapted to remove empty records)
df = pd.read_csv('olist_order_reviews_dataset.csv')
df = df.dropna(subset = ['review_comment_message'])
# Function to tokenize and remove stopwords
def preprocess(text):
    stopwords = nltk.corpus.stopwords.words('portuguese')
    tokens = nltk.word_tokenize(text.lower())
    tokens = [token for token in tokens if token not in string.punctuation 
    and token not in stopwords]
    return tokens
# Function to create word frequency distribution
def word_frequency(tokens):
    frequency = Counter(tokens)
    return frequency
# Function to summarize short reviews
def summarize_reviews(text, num_keywords=3):
    tokens = preprocess(text)
    frequency = word_frequency(tokens)
    important_words = [word for word, count in 
    frequency.most_common(num_keywords)]
    summary = ' '.join(important_words)
    return summary
# Apply the function to the DataFrame
df['summary'] = df['review_comment_message'].apply(summarize_reviews)
# Display the results (manually adapted to print the summary of the longest 
message)
print("Longest review:", df.loc[1316]["review_comment_message"])
print("Summary:", df.loc[1316]["summary"])

The frequency-based Python workflow offers a simpler comparison. It reads a review CSV with pandas, removes records without review text, tokenizes Portuguese with NLTK, removes stopwords and punctuation, and counts remaining words. Inspect what the selected words capture—and what context they leave behind. section 6.3.4

Table 6.5 Summary efforts on a sample of five customer reviews
Translated review
Three-word summary by frequency-based methods
Three-word summary by ChatGPT 4
ChatGPT’s haiku
I received it well before the stipulated deadline.
received
well
before
Early delivery
Arrived well before
Stipulated deadline nears
Satisfaction found
Congratulations Lannister stores, I loved shopping online, safe and practical. Congratulations to everyone and happy Easter.
congratulations
lannister
stores
Lannister stores praised
Lannister stores cheer
Safe and practical shopping
Happy Easter joy
Efficient device. On the website, the device brand is printed as 3desinfector, and upon arrival, it has another name... Update with the correct brand since it’s the same device.
efficient
brand
device
Efficient, mislabeled device
Efficient device
Misnamed but still performs well
Update with true brand
But a little, lagging... for the price, it’s good.
a bit
lagging
...
Lagging, decent value
Slight lag appears,
Yet value stands its ground,
Acceptable deal.
Reliable seller, product ok, and delivery before the deadline
reliable
seller
product
Reliable seller, prompt
Trustworthy vendor,
Product meets expectations,
Early delivery.

Five translated reviews place three-word frequency outputs beside ChatGPT summaries and haiku. Use the original review as the check, rather than treating the most fluent output as automatically the most faithful. section 6.3.4

Match the method to the job. Longer narratives may benefit from abstractive overviews; technical documents may call for a mix of extracted details and synthesis. At scale, also weigh generative AI’s speed and cost against the control lost when a company depends on third-party models or APIs. section 6.3.4 section 6.3.2 section 6.4

How Starting Data Analytics with Generative AI and Python covers it

How deep it goes
Defines summarization, contrasts extractive and abstractive choices, and demonstrates prompts and frequency-based preprocessing without developing model internals.
Examples
Compares a Portuguese delivery complaint, frequency-based Python output, and ChatGPT responses across five customer reviews.
What it assumes
Builds on identifying negative reviews through sentiment analysis or review scores while defining summarization in plain language.
What you can do after
Apply it via ChatGPT or Python NLTK, then check whether the output retains the original review’s important information.
How it connects
Links summarization to sentiment analysis, keyword extraction, translation, rule-based methods, and generative AI.

Also explained in

Compare how 12 books explain summarization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Starting Data Analytics with Generative AI and Python. The section links point to the parts of the book it draws on.