concept non - linear function approximation in category reinforcement learning

This is an excerpt from Manning's book Grokking Deep Reinforcement Learning.
For the rest of the book, we are moving on to discuss the subtleties that arise when using non-linear function approximation with reinforcement learning. Everything that you have learned so far still applies. The only difference is that instead of using vectors and matrices for holding value functions and policies, now we move into the world of supervised learning and function approximation. Remember, in DRL, agents learn from feedback that is simultaneously sequential (as opposed to one-shot), evaluative (as opposed to supervised), and sampled (as opposed to exhaustive). So far, we haven’t touched the “sampled” part; agents have always been able to visit all states or state-action pairs, but starting with the next chapter, we concentrate on problems that cannot be exhaustively sampled.
It’s essential to understand why we use function approximation for reinforcement learning in the first place. It is common to get lost in words and pick solutions due to the hype. You know, if you hear “deep learning,” you get more excited than if you hear non-linear function approximation, yet they are the same. That’s human nature. It happens to me; it happens to many, I’m sure. But our goal is to remove the cruft and simplify our thinking.