liveBook · Manning

Natural language

As explained in Natural Language Processing in Action, Second Edition

A natural language is a human language that develops and is used for communication rather than being defined solely by formal syntactic rules.

A natural language, such as English, lets people communicate without following the strict, unambiguous rules required by programming languages. Software that processes natural language can let people use their own words instead of learning commands designed for a computer. That freedom creates the central challenge for natural language processing (NLP): the same words may need context to be understood. section 1.1 section 1.2 section 1.2.3

Both natural and programming languages have tokens—roughly, words or other units—and grammars for combining them. But a natural language can have hundreds of thousands of possible tokens, while Python has 33 reserved keywords. Its meaning also cannot be recovered from syntax alone. “Good morning” might look like a statement about the time of day, but it commonly serves as a greeting. Understanding it draws on knowledge that a human listener is assumed to have. section 1.1 section 1.2.3

This is why a complete tree of spelling and grammar rules would be brittle: unexpected punctuation or spelling could defeat it, and rules alone would not capture every meaning. Statistical relationships between words offer another approach. Regular expressions remain useful for narrower jobs, such as recognizing a phrase, but they do not define a natural language in full. section 1.2.3 section 1.4.2 section 1.4.1

A greeting is easier to spot than to understand

A combination lock accepts one correctly ordered sequence and gives one response. A chatbot can similarly look for a greeting such as “Hello, Rosa” and trigger a reply. Python string processing can handle that narrow task without machine learning. It does not follow that the chatbot understands everything a greeting conveys, or that the same strict pattern will handle unrestricted conversation. section 1.4.1 section 1.4.2 section 1.2.3

Words also help people shape and share thoughts, not merely issue instructions. NLP works with those words, but processing them should not be mistaken for possessing a human listener’s commonsense knowledge. A practical path is to start with a specific language and task—English and greeting recognition, for example—then choose tools suited to the range of expressions the system must handle. section 1.2.1 section 1.2.3 section 1.2 section 1.4.1

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Explains tokens, grammar, ambiguity, and contextual meaning, but defers knowledge-base techniques for interpreting implicit knowledge.
Examples
Uses “good morning,” a combination lock, and chatbot greeting recognition to distinguish contextual meaning from strict pattern matching.
What it assumes
Builds on familiar words and programming languages while defining tokens and grammar in plain terms.
What you can do after
Evaluate/understand when simple Python string processing can recognize a phrase and when broader language understanding needs more than rigid rules.
How it connects
Connects natural language to programming languages, tokenization, grammar, regular expressions, statistical NLP, and commonsense knowledge.

Also explained in

Compare how 4 books explain natural language →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.