Domain shift occurs when a model’s development data differs from the data it is applied to. A finance-focused sentiment classifier may not fit general customer reviews; sarcasm, regional language variation, and model biases can further affect results. section 6.2.5
Listing 6.2 Extracting positive reviews with the help of a suboptimal model
import pandas as pd
from transformers import pipeline
# Assuming df is your DataFrame and it has a column named
'review_comment_message'
# Load the sentiment analysis pipeline with the FinBERT-PT-BR model
classifier = pipeline("sentiment-analysis",
model="lucas-leme/FinBERT-PT-BR")
def get_sentiment(review):
try:
result = classifier(review)[0]
return result['label'], result['score']
except Exception as e:
print(f"Error processing review: {e}")
return None, None
# Apply the sentiment analysis to each review
df['sentiment'], df['score'] =
zip(*df['review_comment_message'].map(get_sentiment))
# Filter the DataFrame to only include positive reviews
positive_reviews_df = df[df['sentiment'] == 'LABEL_1']
# Assuming 'LABEL_1' is positive; adjust label as necessary based on model
output
# Now positive_reviews_df contains only the positive reviews
See how a sentiment pipeline applies FinBERT-PT-BR to Portuguese reviews, then filters records by predicted label. section 6.2.5
