10 Natural language processing with TensorFlow: Language modeling

 

This chapter covers

  • Implementing an NLP data pipeline with TensorFlow
  • Implementing a GRU-based language model
  • Using a perplexity metric for evaluating language models
  • Defining an inference model to generate new text from the trained model
  • Implementing beam search to uplift the quality of generated text

In the last chapter, we discussed an important NLP task called sentiment analysis. In that chapter, you used a data set of video game reviews and trained a model to predict whether a review carried a negative or positive sentiment by analyzing the text. You learned about various preprocessing steps that you can perform to improve the quality of the text, such as removing stop words and lemmatizing (i.e., converting words to a base form; e.g., plural to singular). You used a special type of model known as long short-term memory (LSTM). LSTM models can process sequences such as sentences and learn the relationships and dependencies in them to produce an outcome. LSTM models do this by maintaining a state (or memory) containing information about the past, as it processes a sequence one element at a time. The LSTM model can use the memory of past inputs it has seen along with the current input to produce an output at any given time.

10.1 Processing the data

 
 

10.1.1 What is language modeling?

 
 
 

10.1.2 Downloading and playing with data

 
 
 
 

10.1.3 Too large vocabulary? N-grams to the rescue

 
 
 

10.1.4 Tokenizing text

 
 

10.1.5 Defining a tf.data pipeline

 
 
 

10.2 GRUs in Wonderland: Generating text with deep learning

 
 

10.3 Measuring the quality of the generated text

 
 

10.4 Training and evaluating the language model

 
 
 

10.5 Generating new text from the language model: Greedy decoding

 
 

10.6 Beam search: Enhancing the predictive power of sequential models

 
 
 
 

Summary

 
 

Answers to exercises

 
 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest