Natural language understanding (NLU) is the part of natural language processing that determines and analyzes the meaning of human language. It converts raw text into numerical representations—usually vectors—that computers can process and compare. section 1.1.1
A vectorizer, also called an encoder in scikit-learn, performs this conversion. Count vectors and term-frequency vectors support keyword and full-text search, while embeddings represent word meaning more directly and enable semantic operations. section 1.1.1 section 1.1.1
Common NLU tasks include:
- Semantic search and paraphrase recognition. section 1.1.1
- Intent classification, sentiment analysis, and topic modeling. section 1.1.1
- Authorship attribution, reading comprehension, and analogy problem solving. section 1.1.1
NLU in practical systems
Search engines use meaning-aware representations to return relevant pages, while chatbots place NLU at the input side of the dialogue pipeline so replies can follow the user's message. NLU also supports toxic-message detection, autocomplete, spelling correction, and customer-service automation. section 1.1.1 section 1.3 section 12.2
Figure 1.1 Encoding natural language
Trace the NLU pipeline from raw language input to a numerical representation of meaning. section 1.1.1
