Thoughts on John von Neumann's "The Computer and the Brain"

I like to read old computer texts to get a different perspective on the technological context we find ourselves in today. Given my current obsession with how the brain works (or doesn't, rather) it struck me that I had never read John von Neumann's "The Computer and The Brain".

In 1955, von Neumann was invited by Yale university to give the Sillimann Lectures in the spring of 1956. For his theme he chose, "The Computer and the Brain", a subject he'd been interested in for a number of years. Sadly, von Neumann became too sick to give the lecture series - in fact, he never finished the manuscript before his death in February 1957. Being only 82 pages long, it took me less than an hour to read. Yet for all its brevity it was extremely thought provoking...

• why are we emulating neurons in software when their may be a hardware equivalent? This is not a unique thought - Carver Mead did some work in the 1990s which has lead to the Neurogrid project a Stanford. (Mead is the guy credited with coining the term "Moore's Law"). The Neurogrid chip emulates ion-flow across a neuron's membrane using electron-flow through a transistor (its the same physical forces after all!). Seemingly, they can emulate neurons with as few as 8 transistors. The chip therefore is populated with these hardware neurons instead of the usual logic gate configurations of a standard CPU. Seemingly, they get a hugely more efficient emulation of neuronal activity.
• much emphasis is put on the neuron, leaving the brain as a collection of neurons rather than considering it as a whole complex system. It is a continuously running system which has a number of physical and chemical processes being applied to the neurons. Von Neumann uses the analogy of neurons being like AND/OR gates for nerve impulses from 'upstream' neurons (with a bit of non-linear sigmoid functions thrown in). This is a view I've never been comfortable with, but in my experience it is still the prevalent belief amongst technologists.What about the effects of neurotransmitters? What about the physical effects of increased blood flow when areas of the brain are active? (surely the physical movement of neurons will effect the excitation patterns?). We seem to think of the neuron as a node which has some consistent, simple behaviour but when in reality it is a complex item in its own right and is a member of a complex system where it is behaving in a stigmergic manner. (NB: stigmergy = where individual behaviour is effected by and affects the environment, producing larger scale emergent behaviour amongst independent agents)
• ignoring the power issue (brain ~=20W, modern single CPU ~=80W) how close are we to emulating major neuronal activity with conventional hardware? In 1989, TJ Sejnowski wrote a paper revisiting "The Computer and the Brain" which predicted that by 2010 we should have computer that could emulate the "minimal processing capabilities of the brain", which he put at 10¹⁵ operations per second. Let's be generous and just call these petaFlops (I know they're not floating point ops, but this is only a blog...) . Currently, the Top500 list has 6 supercomputers peaking over 1 petaFlop (in fact the first petaFlop machine was around 2006) - these of course are using 10s of thousands of cores and drawing megaWatts of power!! There is work going on with these, and there was the controversial "big as a cat's brain" paper last year. But what about more realistic hardware, say something I could run on my desktop? An NVidia Tesla M2070 can produce about 1 teraFlop (and its SIMD so suitable for neuronal simulation) so surely it could be used to simulate the neuronal activity of a simpler organism such as an ant (~10,000 neurons) ? (During this same investigation I came across Steve Furber's (the ARM chip guy) involvement in the SpiNNaker Project - using arrays of ARMs to simulate neuronal activity. Looks interesting , too)