Digital Soul

Why does the idea of AI stir up such emotion? Some people get nervous about AI when their egos won't allow them to recognize any kind of intelligence except the human kind. Their idea of human dignity depends on a natural superiority over all other creatures (and even over some other humans). Recent movies like AI and Bicentennial Man reinforce the idea that, although we might someday create machines that act human in many respects, they will always lack (and secretly long for) that intangible quality that would make them truly human.

Others might be called carbon chauvinists. They scoff at AI because they believe that only living things made from flesh and blood can exhibit intelligence. When they look at the "smart" machines of today, they seize upon their lapses and say, "See, I told you that a mere machine can never be as smart as a living creature."

Such emotions are hard to set aside, especially since they form the basis for many of our moral and ethical beliefs. Yet they also form a barrier to rational scientific inquiry. To investigate objectively whether AI is really possible, we must put aside the insecurities that lie at the root of such feelings and keep an open and curious mind. Does this mean that we have to check our dignity at the door if we want to ask questions like those posed in the preface? If we derive our dignity from a natural superiority over others, then the answer is probably yes. But this is the kind of dignity that we lost when Copernicus displaced Earth from the center of the universe, and when Darwin suggested that humans evolved from lower life-forms. Intellectual honesty is incompatible with a need to be at the center of the universe. It also requires us to ask questions without fear of where the answers may lead us.

The questions we want to ask challenge religious traditions, as well as New Age thinking. Both assert that the human mind—soul, spirit, what have you—lies beyond the limits of scientific examination. If you ask why it can't be studied scientifically, they say It just can't! There are just some things that you can't analyze! and Subjective experience lies beyond the reach of scientific inquiry! Besides begging the question, such answers are all saying essentially the same thing: There are certain questions that shouldn't be asked. Whenever someone doesn't want you poking into something, it's generally because the answers would threaten cherished beliefs or entrenched power structures. Which is fine. That's an honest reason. But there's a big difference between saying something can't be studied and something shouldn't be studied.

We will proceed in the belief that ignorance doesn't solve any problem, that mysticism explains nothing, and that there is nothing that can't be studied. We won't find all the answers, but this will not stop us from raising the questions.

How long will it take our science and technology to catch up with those fantasies? It may happen gradually or suddenly; in twenty years or two hundred years. It may be happening already. Intelligent machines may assume a form more or less as we imagine them now, or something entirely unexpected. In any case, we have already set off down that road, and there seem to be no exits. We are headed inexorably toward a future with intelligent machines doing more and more things that we now think of as uniquely human. Is it just a matter of time before there are machines that outperform humans in every important way?

Technology always perturbs our values and moral codes—the set of rules that guide the way we interact with each other. Fire, the wheel, the printing press, atomic energy, and genetic engineering—each made some of those rules obsolete. It is just as foolish to think that our moral and ethical codes are written in stone, as it is to imagine that knowledge of any of these technologies could somehow be suppressed or ignored. Citing our abuses of atomic energy, some say that we shouldn't even be thinking about intelligent machines, because we aren't morally and ethically ready to deal with the consequences of such an inquiry. And they would be right. We never are. But this is no reason to suppress scientific knowledge. Besides, even if we wanted to stop the development of intelligent artifacts, we couldn't. It's one of those ideas "whose time has come." But does this mean that we are powerless? That we have to passively watch as we become slaves to our technology? Of course not. It just means that once again we will be forced to rethink and overhaul outdated values and moral codes. This time, however, the changes will be so massive, they will upset the very way we think about ourselves.

One widely used tool for understanding intelligence is called introspection. At first, it seems to offer insights into mental processes by examining how they "feel" to us. This tool is flawed because the observing instrument itself is a part of what we are trying to explain. Such an instrument would most likely fool itself. What we are "conscious" or "aware" of at any given moment tells us nothing at all about how consciousness works. The feeling of consciousness, though "subjectively obvious," defies detection and analysis by conventional "objective" means, except that we know that it is strongly correlated with electrical activity in a certain part of the brain stem. The same can be said for feelings that we subjectively experience as anger, love, happiness, grief, intuition, or creativity. These labels could well be comfortable metaphors for complex genetic programs that create illusions, like the Wizard of Oz, for their own procreative purposes.

Another tool, used by brain researchers, maps the parts of the brain that are electrically or metabolically active when we feel certain emotions. Such maps, in their present state, are useful in diagnosing brain damage, but they are not the same as explaining how emotions and consciousness work, any more than an array of flashing lights on a computer panel tells how information is being processed inside.

How shall we proceed, then, in the absence of a suitable, objective vocabulary? We will muddle through by using the language of unambiguous definitions and subjective experience, but with the implicit understanding that when we use words like anger or happiness , we will agree that we mean, not a thing, but "the mental state that we subjectively experience as anger or happiness."

My "explanations" of these experiences will consist mostly of plausible mechanical analogies. These analogies may or may not model how the brain actually produces these experiences, but I do not regard this as important. What is important is that a plausible mechanical model can be constructed.

Once a logical structure for our own thinking is spelled out in detail, creating a physical realization of it by reverse engineering would seem just a problem in technology. But the logical structure of our own thinking patterns is as opaque to us as ever! Early optimism about understanding human thought from an informationprocessing point of view has been replaced with more restrained language. We are beginning to understand how deeply intertwined our thought patterns are with the accidents of our biological and social evolution and our environmental experiences. We may find it impractical, even undesirable, to replicate in detail all the human mental functions that nature took eons to develop. To be sure, many of nature's steps along the evolutionary trail are responsible for behaviors that cause us a great deal of trouble.

The study of intelligent behavior will therefore likely diverge along two different paths: One will explore the origins and structure of human thinking patterns from the point of view of the environment in which they evolved and the genetic purpose they served. This path has been called evolutionary psychology. The other path will pursue the evolution of intelligence less and less like our own, with utterly different awareness, priorities, and goals. As James Hogan put it in his book, Mind Matters, "Evolution produced flying things that have feathers and chirp, but engineering a flying machine gets you a 747."³

The potential for widespread social and economic disruption precipitated by failures of critical computer systems can hardly be overstated. When a NORAD (North American Air Defense Command) computer mistook the rising moon for a flight of attacking Soviet missiles, nuclear war could have been the result! The failure of software designers to foresee the arrival of the year 2000 prompted a panic to avert disruption of critical systems when the new millennium arrived. To avoid having to understand and fix the offending programs, many users simply replaced them.

Such catastrophes are the natural result of underestimating the chances that our machines can fail. One "obvious" solution is more care and discipline in engineering, testing, and maintaining programs, but that is not enough. It will also be essential to build into complex programs more intelligent self-monitoring and self-correcting capabilities. We will see that such capabilities are the ingredients of a rudimentary kind of consciousness. Indeed, our own consciousness can probably be traced to the survival value of such capabilities.

Perhaps we have very little control, on the average, over the course of evolution of intelligent machines—no more control than fourteenth-century Europeans had over the progress of the Black Death. If the victims of the bubonic plague had understood sanitation and the ways that pathogens are transmitted, the effects of the plague would have been minimized. But they didn't understand—any more than we understand today about how rapidly and inevitably intelligence evolves and takes on new and unpredictable forms. Creating machines that we cannot completely control, and then putting them in charge of critical aspects of our lives, poses risks whose consequences we may not have the luxury of contemplating afterward.

Our place in that world will be largely determined by the choices we make, both as individuals and as a species. To survive our technological adolescence and to preserve even a facade of human dignity, we may have to lose some of our self-destructive evolutionary baggage. Before we can learn to live with intelligent machines, we may first have to learn how to live with each other. If we continue dealing with each other and with twenty-first-century technology, equipped only with rigid moral, ethical, and religious beliefs that haven't changed significantly in most of the world since the Middle Ages, then the machines will surely triumph. Instead of asking what will we do with intelligent machines, we may well be asking What will they do with us? If we're lucky, they may keep us as pets!

Is there anything we can do to keep this creepy scenario from playing out? Calls for global awareness of the problem, and for global agreement to regulate scientific inquiry into certain fields, do not paint an optimistic picture. Only in the last century have we developed technologies that could wipe out civilization, so we have not had much practice dealing with global-scale crises. Now we are talking about turning the scientific paradigm on its head! Yet, as odd as it may sound, there are still steps that we as individuals can take that may have some collective effect.

How is it that "mere machines" can perform tasks so complex that we compare them with human brains? What goes on inside their silicon circuitry that seems so much like thinking? Are we ourselves merely organic machines executing programs coded in our genes and shaped by our environment? Is human thought just a fancy kind of computing? If so, will machines eventually get so smart that they will be able to do anything a person can do? If not, where is the gap that no machine will ever be able to bridge? Are there any limits to what machines whose capacities are not confined by organic vessels might be able to do?

So mechanisms have been around for centuries to help people remember , calculate, and make decisions. If you think about it, you will recognize that most computers in use today still serve these three basic functions, just in more elaborate combinations.

Today, we continue to develop machines to replicate more advanced human abilities—sensors that interact with their surroundings, speech, vision, language, and locomotion—even the ability to learn from their experiences to better achieve their goals. For example, many kinds of robotic pets now commercially available interact with their world, make seemingly autonomous decisions about their next actions, and communicate with people in ways that seem to express emotions like joy, anger, and anxiety.

Other learning machines emulate the part of human behavior that amasses knowledge and keeps track of the relations between things. These expert systems often know more about their specialty than do their human counterparts. Expert systems already compile and extend the experience of the world's specialists by dispensing medical diagnoses, legal analyses, and psychological advice, and by forecasting the weather, locating natural resources, and designing vehicles, structures—and even new computers!

Consequently, our efforts to supplement and enhance human intelligence have already paid off with machines that actually exceed human thinking capacity in many specialized areas: calculating, decision making, remembering, learning, creating, making plans, pursuing goals, and—who knows?—perhaps even enjoying it! A reasonable extrapolation of present capabilities would predict the connection and cooperation of computing systems that will increase the breadth of machine intelligence as well. Joined by networks like the Internet, machines might even combine and integrate specialized capabilities and eventually evolve their own communal and social rules akin to ethics and morality.