Neurocomputing

anything? I am responsible for my own actions. Nothing that I do can take away the fact that someone else acting in a certain way, and nothing that they can do can absolve me of my own responsibility. Original Sin and salvation by Christ are both deeply offensive ideas to me and to all extropians who value individual responsibility.

In ending this discussion, I want to remind you that you are all Popes. You are all you own highest authority. You are the source of your action. You choose

your values - whether you do so actively or by default. You choose what to believe, how strongly to believe, and what you will take as disconfirming evidence. No one has authority over you - you are your own authority, your own value-chooser, your own thinker. Join me, join Lucifer, and join Extropy in fighting God and his entropic forces with our minds, our wills, and our courage. God’s army is strong, but they are backed by ignorance, fear, and cowardice. Reality is fundamentally on our side. Forward into the light!

1010101010101010101010101010

Neurocomputing:

Why Computers Don’t Think Like Humans, and How They Might

by Simon D. Levy

How do you feel about computers? I think they suck. I’m not talking about the idea of having computers do the things that people can’t or won’t do, such as storing huge amounts of data or performing millions of calculations in a second. I’m talking about the actual machines, from the pocket calculators that are practically free these days, to the multi-million dollar supercomputers. They suck.

‘But,’ you object, ‘Computers are incredible! Today microchips the size of your fingernail can perform

calculations that a few decades ago would have taken a warehouse full of circuits.’ Sorry, I’m not convinced. If something is the size of a warehouse and sucks, and you shrink it to the size of a fingernail, it still sucks, at least in my book.

What I’m getting at is that the fundamental abilities of computers have not changed since the first relay-based machines were built in the 1940’s. The basic architecture of computers — a storage area, a processor, and an input/output device — has seen very few qualitative improvements. Using a

EXTROPY

#4 - Summer Issue, 1989

12

computer is easier now than it ever has been, but the number of people who can actually program a computer represents a very, very small percentage of the population. Most people I know who have tried to learn the skill end up frustrated and angry. Why is this?

A lot of answers have been given for this question, but I think that the essential reason why computers are so difficult is that they are (1) denotative, (2) deterministic, and (3) serial. They are denotative because they can only use a name to point to a single object. For example, think of all the images that the word ‘man’ conjures up in your mind when you hear it. In a very real way, there is no definable limit to the amount of information that you associate with this (and perhaps all other) words in your language. In a computer language, on the other hand, every name you assign to an object (a number, a list of numbers, a list of properties, a device to be controlled, et al.) can refer only to that object, and no other. Of course, if you are writing a program to fire a missile, you don’t want your computer to be making any guesses as to what you are telling it to do. On the other hand, imagine trying to hold a conversation with a person for whom every word refers to a single, rigidly defined real-world object or class of objects. You tell him to answer the telephone. He happens to have grown up on the other side of the country, and for him, ‘the telephone’ is a certain device located in his home, thousands of miles away. He can’t associate the telephone in your house with the general notion of what a telephone is (a class of objects with the same function), because his language is purely denotative. You get the idea.

Computers are deterministic in that, given the same input and programming, they will always produce the same output. Again, this feature is desirable at a certain level of organization, because you want to know what your machine in going to do based on what you tell it to do. In my estimation, though, determinism is the main reason that learning to use or program a computer can be such a nerve-wracking experience. One little error in a program, a single misspelling, can result in a horrifying heap of garbage with no resemblance to what you wanted the machine to do. The analogy here does not even need to be made with something as complex as a human being; imagine, for instance, a car that would explode if you gave it regular gasoline instead of unleaded.

Most computers are serial devices: They deal with things one step at a time. For example, in multiplying a large list of numbers by a single number, the average computer will loop through the entire list, multiplying each number in the list by the first number, in succession. This strategy fails to take advantage of the distributive property of multiplication over addition, one of the fundamental properties of mathematics. As far as the computer is concerned, the single number may as well be a different number each time it is multiplied with a number in the list. This problem, named the von Neuman Bottleneck (after one of the founders of computer science), may not seem to bear much relation to the problem of getting computers to think like people, but many researchers have come to believe that it is the serial nature of present-day computers that causes them to be stupidly denotative and

EXTROPY

#4 - Summer Issue, 1989

13

stupidly deterministic, so unlike human beings.

These researchers, working in a field known as parallel distributed processing, neural networks, neurocomputing, and a few other buzzwords that escape me at the moment, believe that the answer to the problem lies in the way that computers represent and store information. As complicated and mysterious as this field has come to seem, its fundamental assumption can be summed up as follows: rather than representing an idea or behavior locally, in a single memory address or addresses, represent the idea or behavior globally, as a pattern of activity in a network of simple units. Instead of representing the word ‘man’ or ‘telephone’ as a single object or list of properties at a specific location in memory, a neural computer will respond to the word ‘man’ with a pattern of electrical activity over a net of symbolic units. These units, like the neurons that make up the animal brain, are called sub-symbolic because they themselves do not represent anything interesting; rather, the way they are connected determines what they are ‘thinking’ about.

There is a great deal of justification, both empirical and pragmatic, for building computers this way. Empirically, we know that the processors in present-day computers work at orders of magnitude faster than human neurons. The inability of these computers to perform even the simplest human (and animal) tasks points to the conclusion that the architecture (setup) of the processing units, not their speed, is what counts. The parallelism of the neural connections in the brain is well known. Each neuron can connect to a large number of other neurons. The total

number of these connections in the human brain has been calculated as ten to some ridiculous power. Clearly, biology has chosen connections over speed in setting up the brain.

Pragmatically, neural computers are capable of overcoming many of the problems associated with denotation and determinism. Because it represents a concept or pattern in a distributed way, a neural network is less likely to be upset by variation in individual tokens of that concept or pattern. For example, say we present our neural network with a picture of a man (I won’t get into just how to do this; that’s another article, book, and career in itself). Anyway, the man in the picture happens to have a beard. We tell our network ‘This is a man.’ After we ‘train’ it in this manner, the network has stored the representations of a man as a pattern of connections over its individual neurons. Some of these connections represent the idea of ‘having a beard.’ To test our network, we present it with another picture, this time a picture of a man without a beard. Because so much of the ‘non-beard’ part of the image of a man is activated in the network when it sees the second picture, the network is able to recognize the second picture as that of a man. Instead of isolating beardedness in a certain location in the computer and telling the computer that beardedness is optional for manhood, we have allowed the computer to make its own judgements about what is or isn’t important for recognizing a man. In this way, the network can be said to overcome some of the problems of determinism mentioned earlier. Because the programmer has not actively isolated any feature as crucial to the man/not-man decision, it is unlikely that any particular variation in the

EXTROPY

#4 - Summer Issue, 1989

14

pictures will have a catastrophic effect of the decision. A single ‘mistake’ won’t result in garbage.

Of course, we want the network to pick out the essential features of what a man looks like, and we can do this by training it with many pictures. After a while, the network will learn on its own what these features are, just as a child does not have to be told ‘This is a man; note the optional beard, usual two legs, two arms, deep voice, …’

The distribution of computation in a neural net has another useful property worth mentioning, that of robustness, or graceful degradation. Destroying a connection or connections won’t result in complete chaos, unless the damage is extensive. Here, again, biology provides a parallel. Our brain cells are killed off by various agents (alcohol is the only one I can think of right now, ha ha), but we generally suffer no long-term memory loss or retardation from losing them. The loss of these abilities that comes from aging, severe as it may be, is not sudden; it is usually difficult to pinpoint the time at which a certain faculty was lost completely. Once more, the likeliest explanation for these observations is that the function of the brain is not isolated in a particular cell or set of cells, but in the zillions of connections between the cells. Breaking a few of these connections won’t cause too much damage.

It is true that certain areas of the brain seem to be specialized for certain functions — the left brain/right brain distinction is the most commonly mentioned distinction of this sort. My response to this is that (1) the isolation of cognitive areas in the brain is so ill-

understood that anyone raising this objection to neural networks is standing on shaky ground, and (2) even if someone someday proves that a single area is responsible for a certain function, we will simply create a specialized neural net for this ‘sub-brain,’ and not worry about it any further.

Finally, it is worth mentioning that the neural networks issue has some profound implications for the mind/brain problem, the problem of the ‘soul,’ and for epistemological problems in general. If the brain really is a neural network, then the mind is simply (incredibly!) the connections between the neurons. To me, this formulation has the exact flavor of the abstract/concrete distinction that is at the core of the mind/brain problem, the distinction that philosophers have been agonizing over for a long, long time. If you’re looking for a reason why this article has been published in Extropy, that’s it.

I hope that I’ve conveyed some of the basic ideas of neurocomputing, and some of the excitement of the people who work in this field. I won’t tell you that what I’ve said is a vast oversimplification, because that kind of caveat is a lot of crap — either you get an idea across or you don’t; the rest, as the Hebrew sage Hillel said, is just detail. The detail is fascinating, though, and I plan to continue talking about neural networks in Extropy’s next issue. I’d like to review Marvin Minsky’s classic Perceptions, the book that was supposed to have put an end to neurocomputing around twenty years ago. Stay tuned.

EXTROPY

#4 - Summer Issue, 1989

15