concept IMDb dataset in category deep learning

appears as: IMDb dataset
Deep Learning with JavaScript: Neural networks in TensorFlow.js

This is an excerpt from Manning's book Deep Learning with JavaScript: Neural networks in TensorFlow.js.

  • It addresses the size problem with one-hot encodings. embeddingDims is usually much smaller than vocabularySize. For example, in the 1D convnet we are about to use on the IMDb dataset, vocabularySize is 10,000, and embeddingDims is 128. So, with a 500-word review from the IMDb dataset, representing the example requires 500 * 128 = 64k float numbers, instead of 500 * 10,000 = 5M numbers, as in one-hot encoding—a much more economical vectorization.
  • The advantage of using pretrained word embeddings such as GloVe is two-fold. First, it reduces the amount of computation during training because the embedding layer doesn’t need to be trained further and hence can simply be frozen. Second, pretrained embeddings such as GloVe are trained from billions of words and hence are much higher-quality than what would be possible by training on a small dataset, such as the IMDb dataset here. In these senses, the role played by pretrained word embeddings in natural language processing problems is similar to the role of pretrained deep convnet bases (such as MobileNet, which we saw in chapter 5) in computer vision.