liveBook · Manning

Classifier

As explained in Deep Learning with Python, Third Edition

A supervised learning task that assigns text inputs to one of several classes.

A text classifier predicts a label from text: whether a movie review is positive or negative, whether an email is spam, or which topic a newswire covers. The first question is whether each input has one possible label or can have several. The second is whether labeled examples are available to train the model. section 4.1 section 4.2 section 6.1.1

From newswire to topic

The Reuters dataset contains short newswires assigned to one of 46 topics. In Keras, each newswire is supplied as a sequence of word indices, and its label is an integer from 0 to 45. There are 8,982 training examples and 2,246 test examples. A classifier trained on these labeled inputs produces a score for each possible topic. section 4.2.1 section 4.2.3

Match the output layer and loss to the labeling problem:

  • Two mutually exclusive classes: use one sigmoid output and binary_crossentropy. The output is a probability between 0 and 1, as in positive-versus-negative review classification. section 4.1.7
  • Exactly one of many classes: use one softmax output per class. For 46 topics, the outputs form a probability distribution summing to 1. Use categorical_crossentropy for one-hot targets or sparse_categorical_crossentropy for integer targets. section 4.2.3 section 20.1.6
  • Several possible labels per input: use one sigmoid output per class with binary_crossentropy and k-hot targets. This is a different problem from choosing one newswire topic. section 4.2 section 20.1.6
Listing 4.15 Model definition
model = keras.Sequential(
    [
        layers.Dense(64, activation="relu"),
        layers.Dense(64, activation="relu"),
        layers.Dense(46, activation="softmax"),
    ]
)

A Keras topic classifier uses two 64-unit Dense layers followed by 46 softmax outputs. The wider intermediate layers help avoid discarding information needed to distinguish many classes. section 4.2.3

Prepare text as numeric input before feeding it to a neural network. Binary vectors of words are one option; text can also be treated as a sequence when word order matters. Whatever the representation, monitor performance outside the training set: improvement on training data can accompany worsening results on unseen examples. For many classes, also avoid intermediate layers so small that they become information bottlenecks. section 4.1.7 section 14.3 section 4.4

How Deep Learning with Python, Third Edition covers it

How deep it goes
Works through labeled datasets, numeric inputs, output layers, losses, and evaluation; defers more complicated models such as Transformers.
Examples
Uses IMDb movie-review sentiment and Reuters newswire topic prediction as concrete text-labeling tasks.
What it assumes
Assumes basic neural-network knowledge while introducing text-to-numeric preprocessing and simple text classifiers.
What you can do after
Build it from scratch in Keras for binary or single-label multiclass text prediction, choosing an appropriate output and loss.
How it connects
Links text preprocessing and bag-of-words or sequence representations to binary, single-label multiclass, and multilabel classification.
Cover of Deep Learning with Python, Third Edition
liveBookmanning.com
Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 31 books explain classifier →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.