Suppose an e-commerce platform predicts customers’ review scores from delivery, product, and payment details. Classification accuracy answers a narrow question: what proportion of predicted scores exactly match the recorded scores? It does not say why customers gave those scores. section 3.4.3
To check it in Python:
- Prepare the predictor columns and recorded review scores, then split the records into training and test sets. section 3.4.3
- Fit an ordinal logistic regression model on the training set and predict scores for the test set. section 3.4.3
- Use scikit-learn’s accuracy_score to compare predictions with recorded scores; inspect a confusion matrix to see which scores are confused. section 3.4.3
Is 43% good enough?
One run reports accuracy of 0.43035: about 43 in every 100 test predictions match the recorded review score. The confusion matrix provides an essential reality check. Its diagonal contains 14,616 correct predictions out of 33,963 records, but 19,506 records belong to its largest actual-score group. Predicting that group every time would match about 57.4% of these records. Here, 43% alone is not a persuasive result. section 3.4.3 section 4.7
Be skeptical when an AI assistant turns this score into confident business advice. A response describes 43.03% accuracy as evidence that the variables explain some variation in reviews; accuracy alone does not establish that conclusion. Check the recorded outcomes and the pattern of errors before acting on claims about delivery, prices, or product listings. section 4.7 section 3.4.3
import pandas as pd
import mord
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, accuracy_score
# Prepare the data (assuming you have already merged necessary
DataFrames into analysis_df)
# Convert categorical variables to numerical format if required
# (e.g., using pd.get_dummies() for product categories)
# Split the data into training and testing sets
X = analysis_df.drop("review_score", axis=1)
y = analysis_df["review_score"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,
random_state=42)
# Fit the ordinal logistic regression model
model = mord.LogisticAT(alpha=0)
model.fit(X_train, y_train)
# Predict the review scores on the test data
y_pred = model.predict(X_test)
# Calculate the accuracy of the predictions
print("Accuracy:", accuracy_score(y_test, y_pred))
# Display the confusion matrix
print("Confusion Matrix:\n", confusion_matrix(y_test, y_pred))
# Analyze the model's coefficients
print("Coefficients:\n", model.coef_)
The Python workflow shows where accuracy_score fits: after the data split, model fitting, and test-set prediction. section 3.4.3
Accuracy: 0.43035067573535907 Confusion Matrix: [[ 385 70 401 1637 1302] [ 38 13 71 563 513] [ 55 16 109 1342 1365] [ 49 18 100 2909 3501] [ 54 23 208 8021 11200]] Coefficients: [-4.92618413e-02 1.63964377e-03 3.08760598e-03 1.38901936e-02 1.93889038e-05 4.21370114e-01 -8.75650964e-06 3.59787432e-01 2.93638614e-02 -1.72296023e-03 2.82130024e-02 9.50451240e-07 1.32936583e-01 2.23209948e-01 1.31639907e-02 -5.95202359e-03]
Compare the reported accuracy with the confusion matrix rather than relying on the single score. section 3.4.3
