Computing till to date has been dominated by the von Neumann architecture. Here the processor and the program are separate. The program sits in memory from where it gets fetched and executed by the processor. The advantage is that completely different programs, solving totally unrelated problems can be loaded in the memory and the same processor can execute them. But, neural networks have a fundamentally different architecture. Here there are no separate processors and programs. There is a single entity called, well, the neural network1 . In this chapter we will study this paradigm in detail.
In section 1.7 we have seen an overview of neural networks2 . There we indicated that most intelligent tasks performed by humans can be expressed in terms of mathematical functions. While that gives us hope of developing automated solutions, we are hobbled by two serious difficulties.
- In addition to being arbitrarily complicated, the functions underlying different problems are completely different. Hardly any common pattern exists.
- For most problems, we do not know its underlying function.
The first issue makes it challenging to come up with a mechanized repeatable solution for performing generic intelligent tasks. If we have to start from scratch and estimate the underlying function every time we need to solve a problem, there is little hope of imparting human like intelligence to a machine.