Precision asks: of everything a system returned or labeled positive, how much was actually relevant or correct? For a classifier, divide correct positive predictions by all positive predictions. A high overall accuracy can hide poor precision when the class of interest is uncommon. section 4.2.1
Toxic-comment detection
A classifier labeled 1,038 test comments toxic, but only 125 were toxic. Its precision was 125 ÷ 1,038, or about 12%: most alerts were false alarms. It also found only 125 of the 326 toxic comments, giving it 38.3% recall. The two measures answer different questions. section 4.2.1
>>> from sklearn.metrics import confusion_matrix
>>> confusion_matrix(y_test, lda.predict(X_test))
array([[1261, 913],
[ 201, 125]], dtype=int64)
The confusion matrix shows the counts behind those rates. Of the comments predicted toxic, 913 were nontoxic and 125 were toxic; another 201 toxic comments were missed. section 4.2.1
The right balance depends on what a prediction triggers. Low precision can flood community moderators with alerts or wrongly hide acceptable comments. Labeling every comment toxic would catch every toxic comment but make precision poor. Labeling every comment nontoxic could achieve 87% accuracy in this dataset while finding none of the toxic comments. section 4.2.1
Search has a similar tradeoff. Treating Age and age alike can retrieve more relevant documents, but also return results a user did not want. Supporting both normalized searches and exact-capitalization queries calls for separate indexes. section 2.3.2
