Conscious Machines - Minsky
-
Upload
hernando-herrero -
Category
Documents
-
view
217 -
download
0
Transcript of Conscious Machines - Minsky
-
7/29/2019 Conscious Machines - Minsky
1/11
http://kuoi.com/~kamikaze/doc/minsky.html
Conscious Machines
Marvin Minsky
M.I.T.
Published in "Machinery of Consciousness", Proceedings, National Research
Council of Canada, 75th Anniversary Symposium on Science in Society, June1991. I don't have the final publication date.
Many people today insist that no machine could really think. "Yes," they say,
"machines can do many clever things. But all of that is based on tricks, just
programs written by people to make those machines obey preconceived rules.The results are useful enough -- but nowhere in those cold machines is there
any feeling, meaning, or consciousness . Those computers simply have nosense that anything is happening."
They used to say the same about automata vis-a-vis animals. "Yes, thoserobots are ingenious, but they lack the essential spark of life." Biology then,
and psychology now: each was seen to need some essence not mechanical.
The world of science still is filled with mysteries. We're still not sure of how
the Sun produces all its heat. We do not know precisely where our early
ancestors evolved. We can't yet say to what extent observing violence leads to
crime. But questions like those do not evoke assertions of futility. We can try
harder to detect more neutrinos, find more fossils, or perform throrough
surveys. However, in certain areas of thought, more people take a differentstance about the nature of our ignorance. They proceed to work hard, but not
toward finding answers, but toward trying to show that there are none. ThusRoger Penrose's book [1] tries to show, in chapter after chapter, that human
thought cannot be based on any known scientific principle.
I have already written a book [2] that discusses various attempts to show that
men are not machines, but mainly works to demonstrate how the contrary
might well be so. You might object that no one has time to read all such
books, so why can't I just summarize? And that's what this essay is all about:that certain things are too complex to summarize! This includes the
mechanisms of highly evolved organisms and, especially, the workings oftheir nervous systems. It also includes the highly evolved systems that we call
cultures and societies. And especially, it includes what we call consciousness.
In particular, consider the problem of describing the brain in detail -- in view
of the fact that it is the product of tens of thousands of different genes. We cancertainly see the attractiveness of proposing to get around all that stuff, simply
http://kuoi.com/~kamikaze/doc/minsky.htmlhttp://kuoi.com/~kamikaze/doc/minsky.html -
7/29/2019 Conscious Machines - Minsky
2/11
by postulating some novel "basic" principle by which our minds are animated
by some vital force or essence we call Mind, or Consciousness, or Soul.
That tendency is not confined to religion and philosophy. The same approach
pervades our everyday psychology. We speak of making decisions by
exercising 'freedom of will'; or by finding what something 'means', or of
discovering truths by means of 'intuition'. But none of those terms explains
very much; each only serves to name another set of mysteries.
The situation is different in Physics. Consider the whirlpools that form whenwater flows down drains. When a scientist says that this can be explained by
the Conservation of Momentum, that's very different from attributing it tosome convenient Whirlpool God -- because precisely the very same
mathematical rule can be used to explain and predict a vast range of other
phenomena, with a precision and lack of exception found in no other realm ofideas. That principle apparently applies to everything in our universe and,
because of its singularly good performance, we regard this sort of
"fundamental" or "unified" principle as an ideal prototype of how to accountfor mysterious phenomena. But one can carry that quest too far by only
seeking new basic principles instead of attacking the real detail. This is what Isee in Penrose's quest for a new basic principle of physics that will account for
consciousness.
The trouble is that this approach does not work well for systems whose
behavior has evolved through the accretion of many different mechanisms,over the course of countless years. For example, in physiology, the excretion
of excess potassium in the urine occurs because our ancestors evolvedelaborates system of receptors and transport mechanisms, along with intricate
machinery for controlling them. This is understood so well today, that no onefeels that there's any need to postulate a separate, special principle for the
Conservation of Potassium. Progress in this area is no longer news for biology
because we have seen two hundred years of great success accrued from
working out details. Since Harvey, Darwin and Pasteur, the idea of a Vital
Force has nearly vanished from biology. Why is it still so much a part ofpresent-day psychology?
I'll argue that vitalism still persists because we're only starting to find a way to
understand the brain. (I see this as the irony of Penrose's book, because thepath toward understanding lies in that flood of new ideas that began to grow
around the time, half a century ago, along with the emergence of computers in
the 1950s -- include the work of Turing in 1936, McCulloch and Pitts in 1943,
and the hundreds of thinkers who joined them afterward. Yet Penrose takes
the other side, and argues that the abilities of human mathematicians to
discover new mathematical truths cannot be explained on the basis ofanything a machine could do. He argues in [1], p110, that this kind of thinking
-
7/29/2019 Conscious Machines - Minsky
3/11
must be based on "insights that cannot be systematized -- and, indeed, must lie
outside any algorithmic action!" He bases this on the assumption that anythinking machine we build for attempting to discovering knowledge about
mathematics must itself be based on some absolutely consistent logical
foundation -- that is, one that cannot possibly produce any type of logicalcontradiction or inconsistency. This is the same assumption used in Godel's
celebrated 'incompleteness theorem'. Penrose's application of this idea to
psychology is due, as Penrose notes, to J.R.Lucas, in Philosophy, 36, pp120-4, 1961.
Consistency
It seems to me that all of this stands upon a single and simple mistake. It
overlooks the possibility, as my colleague Drew McDermott once remarked,of including systems "that are mistaken about mathematics to some degree, orsystems that can change their minds." By inadvertently ruling such machines
out, you've simply begged the question whether human mathematicians can be
kinds of machines -- because people do indeed change their minds, and can
indeed be mistaken about some parts of mathematics. An entire generation of
logical philosophers has thus wrongly tried to force their theories of mind to
fit the rigid frames of formal logic. In doing that, they cut themselves off fromthe powerful new discoveries of computer science. Yes, it is true that we can
describe the operation of a computer's hardware in terms of simple logical
expressions. But no, we cannot use the same expressions to describe themeanings of that computer's output -- because that would require us to
formalize those descriptions inside the same logical system. And this, I claim,
is something we cannot do without violating that assumption of consistency.
If you are not a logician, then you might wonder what's all the fuss about.
"What could possibly be wrong with logical consistency. Who wants thosecontradictions, anyway?" The trouble with this is that the problem is worse
than it looks: paradoxes start to turn up as soon as you permit your machine touse ordinary common-sense reasoning. For example, troubles appear as soon
as you try to speak about your own sentences, as in "this sentence is false" or
"this statement has no proof" or in "this barber shaves all persons who don't
shave themselves." The trouble is that when you permit "self reference" you
can quickly produce absurdities. Now you might say, "Well then, why don't
we redesign the system so that it cannot refer to itself?" The answer is that thelogicians have never found a way to do this without either getting into worse
problems, or else producing a system too constrained to be useful.
Then what do ordinary people do? So far as we know they scarcely use any
logic at all. The studies made by the great child psychologist Jean Piagetsuggest that the abilities required for to manipulating formal expressions are
-
7/29/2019 Conscious Machines - Minsky
4/11
not reliably available to children until their second decade, if ever. And even
as a mathematician, I cannot recognize the psychology Penrose describes.When doing mathematics, my mind is filled with many things non-logical. I
imagine examples based on gears and levers, I imagine conversations that
might reveal to me what Andrew Gleason or Dana Scott might do in the samesituation, or I imagine explaining my solution to a student and discovering
something wrong with it There's little sign of consistency in any of that
experience. Nor is that famous 'intuition' really a privileged route to the truth,because although the answer seems to come with a feeling of certainly, later
it's likely to turn out to be wrong.
Perhaps the most important aspect of how humans work are the ways in which
we ask ourselves (not necessarily by using words) what problems have we
seen before that most closely resemble the present case, and how did we
manage to deal with them. For those were where we made our mistakes andthen sometimes managed to learn from them. And notice that in doing so, we
somehow must employ some capabilities for retrieving and then manipulatingsome descriptions of some of our earlier mental activities. Now, notice how
self-referent this is. Often when you work on a problem you consider doingsome certain thing -- but before you actually carry that out, you often inquire
about yourself, about whether you actually be able to carry it through. Solving
problems isn't merely applying rules of inference to axioms. It involves
making heuristic assessments about which aspects of the problem areessential, and which of one's own abilities might be adequate to dealing with
them. Then, whatever happens next arouses various feelings and memoriesabout of situations that seem similar and methods that might be appropriate. Is
this done by some kind of non-physical magic or it is accomplished, as I
maintain, by the huge and complex collection of knowledge-base
representations and pattern-matching processes that we all regard as 'common
sense'?
Now it happens that when we do such things, we often find that we talk to
ourselves about what we're doing. And when we thus "refer to ourselves" we
sometimes speak of being conscious or aware. I think it no coincidence thatPenrose feels that this, too is something present-day science cannot explain.
Indeed, he Could this result from just that fear of inconsistency and self-reference? Indeed, Penrose sometimes speaks of a "reflection [principle" with
something resembling awe: "The type of 'seeing' that is involved in a
reflection principle requires a mathematical insight that is not the result of the
purely algorithmic operations that could be coded into some formalmathematical system (p110)." In my opinion this is just a mistake! He appears
to assume that when this is applied to humans, the word "consistent" can befreely inserted between 'some' and 'mathematical' -- as though people possess
some marvelous gift whereby they can tell which assertions are true. But inview of the many mistakes we all make, I see no compelling evidence that
-
7/29/2019 Conscious Machines - Minsky
5/11
anyone has any direct such access to truth. All we can depend upon (including
the power of formal proof) is based on our experience. I think. And in anycase there really is no problem at all in programming a computer to perform
that sort of reflective operation. Indeed John McCarthy has pointed out that
forming a Godel sentence from a proof predicate expression (which is thebasis of the Lucas-Penrose argument) requires no more than a one line LISP
program. So in my view Penrose and many other philosophers have put the
problem upside down: the difficulty is not with making algorithms that can doreflection -- which is easy for machines, but with consistency -- which is hard
for people. In summary, there is no basis for assuming that humans are
consistent -- not is there any basic obstacle to making machines use
inconsistent forms of reasoning.
Consciousness.Even the most technically, sophisticated people maintain that whatever
consciousness might be, it has a quality that categorically places it outside the
realm of science, namely, a subjective character that is makes it utterly private
and unobservable. Why do so many people feel that consciousness cannot be
explained in terms of anything science can presently do?
Instead of arguing about that issue, let's try to understand the source of thatskeptical attitude. I have found that many people maintain that even if a
machine were programmed to behave in a manner indistinguishable from aperson, it still could not have any subjective experience. Now isn't that a
strange belief -- considering that unless you were a machine yourself, how
could you possibly know such a thing? As for 'subjectivity,' consider
that talking about consciousness is a common, objective form of behavior.
Therefore, any machine that suitably simulated a human brain would have to
produce that behavior. Then, wouldn't it be curious for our artificial entity tofalsely claim to have consciousness? For if it had no such experience, then
how could it possibly know what to say? Of course a classic question inphilosophy is asking for proof that our friends have minds; perhaps they are
merely unfeeling machines. But then one must ask how they'd know how to
lie.
In any case, we have much the same problem with ourselves; try asking a
friend to describe what having consciousness is like. Good luck! Most likely
you hear only the usual patter about knowing oneself and being aware, ofsensing one's place in the universe, and so on. Why is explaining
consciousness so dreadfully hard? I'll argue that this is something of anillusion, because consciousness is actually easier to describe than most other
aspects of mind; indeed, our problem is a far more general one, because ourculture has not developed suitable tools for discussing and describing thinking
-
7/29/2019 Conscious Machines - Minsky
6/11
in general. This leads to what I see as a kind of irony; it is widely agreed that
there are "deep philosophical questions" about subjectivity, consciousness,meaning, etc. But people have even less to say about questions they'd consider
more simple:
How do you know how to move your arm?
How do you choose which words to say?
How do you recognize what you see?
How do you locate your memories?
Why does Seeing feel different from Hearing?
Why does Red look so different from Green?
Why are emotions so hard to describe?
What does "meaning" mean?
How does reasoning work?
How do we make generalizations?
How do we get (make) new ideas?
How does Commonsense reasoning work?
Why do we like pleasure more than pain?
What are pain and pleasure, anyway?
We never discuss these in everyday life, or bring them up in our children's
schools. An alien observer might even conclude that those Earth-people seemto have a strong taboo against thinking about thinking. It seems to me that this
is because our traditional views of psychology were so mechanistically
primitive that we simply had no useful ways to even begin to discuss such
things. This is why I find such irony in the arguments of those who reject the
new mechanistic concepts of psychology -- the new ideas about computational
processes that promise at last to supply us with adequate descriptions of these
complex processes.
The science of Psychology, as we know it today, is scarcely one hundredyears old. Why did humanity wait so long before the emergence of thinkers
like Freud, Piaget, and Tinbergen? I think the answer lies in the fact that the
brain is not merely a kind of machine, but one that is far more complex than
anything ever imagined before. The pivotal notion provided by those three
pioneers was that the mind has many parts. A person doesn't simply See by
"looking out" through the eyes. Instead, vision involves many differentprocesses, cooperating, competing, being promoted and inhibited by other
processes, being managed and regulated by yet others. You can not simply'recognize' a telephone, because that is scarcely at all a matter of vision;
instead, you have to "re-cognize" it -- that is, the input has to somehow
activate some memory representations of a device with a certain kind of
structure (handset and dial, say) coupled with a certain functional disposition
(to hold to the mouth and ear for communication purposes). This is nothing
like the sorts of unitary concepts found both in commonsense and philosophy,e.g., of a platonic ideal of a telephone, or some sort of model inside the head.
In recent years we've learned much more about the complexity of the brain. It
now appears that perhaps fully half of our entire genetic endowment is
involved in constructing our nervous systems. This would suggest that the
brain is nothing like a single large-scale neural net; instead, it would have
-
7/29/2019 Conscious Machines - Minsky
7/11
even more parts than the skeletomuscular system -- which can be seen to have
hundreds of functional parts. If you examine the index of a book onneuroanatomy, you will find the names of several hundred different organs of
the brain. A good fraction of those are already known to have psychologically
distinct functions. To pursue the analogy a little further, note that the skeletalanatomies of animals have been known for millennia, but only in rather recent
years have scientists understand the mechanics of locomotion and its various
gaits; that had to wait until scientists learned more about the mechanics offorces and materials. Similarly, mechanistic theories of psychology may have
to wait even longer for adequate conceptual tools because the 'mechanics' of
heuristic computation could turn out to be more complex than those of
physics. Before these new ideas emerged, with the era of complex
information-processing computer models, such models were not considered
convincing -- perhaps because there were no feasible experiments. I don't
mean to say that there was no progress at all before computers, only that therewas precious little. Freud himself was one of the first to conceive of "neural-net-like" machines -- only no one would listen to him except Fliess. Later
came the astounding insights of Post, Godel, and Turing, followed by those of
Rashevky's group, McCulloch and Pitts, and Grey Walter's simple yet
somewhat life-like mini-robots. But significant progress began only in the
1950s when more serious models could be conceived, tested, and discarded in
days or weeks instead of years. Soon the researchers in Artificial Intelligencediscovered a wide variety of ways to make machine do pattern recognition,
learning, problem solving, theorem proving, game-playing, induction andgeneralization, and language manipulation, to mention only a few. To be sure,
no one of those programs seemed much like a mind, because each one was so
specialized. But now we're beginning to understand that there may be no need
to seek either any single magical "unified theory" or and single and hitherto
unknown "fundamental principle"-- because thinking may instead be the
product of many different mechanisms, competing as much as cooperating,and generally unperceived and unsuspected in the ordinary course of our
everyday thought.
What has all this to do with consciousness? Well, consider what happened inbiology. Before the 19th century there seemed to be no alternative to concept
of "vitality" -- that is, the existence of some sort of life-force. There simply
seemed no other way to explain all the things that animals do. But then, as
scientists did their work, they gradually came to see no need for a "unified
theory" of life. Each living thing performed many functions, but is slowlybecame clear that each of them had a reasonably separate explanation! For the
most part each separate function was served by a different and specializedorgan! Thus the lungs oxygenate the blood, while the heart pumps it to other
organs. The nucleus reproduces all the organs' structural information, while
the ribosomes translates those codes into proteins which then self-configurethemselves. For some time that subsequent appeared to entail a mystery. It
-
7/29/2019 Conscious Machines - Minsky
8/11
seemed natural to assume that those configurations were based on a uniform
energy-minimizing mechanism -- but simulations did not bear this out. Thisappears to not be so; instead, each protein has had to evolve this property on
its own. (A random string of peptides cannot usually manage it.) Conclusion:
There is no central principle, no basic secret of life. Instead, what we have arehuge organizations, painfully evolved, that manage to do what must be done
by hook or crook, by whatever has been found to work.
Why not assume the same for the mind? (I could have said the brain, instead --but in my view minds are simply what brains do.) Why else would our brains
contain so many hundreds of organs? Of course there are many old argumentsagainst localization of brain-functions because it seemed that often a mind still
works when some of its brain has been lost. One answer to that is to argue that
many functions are accomplished in multiple ways, not only to provide
resistance to some injuries, but perhaps more important, because no particularway is likely to be always reliable. To be sure, there still seem to be some
mental phenomena that have not yet been shown to "organ-ized". So there isstill some room for theories about mechanisms that are not so localized. But
now, I maintain, it is time for "insulationism" to take its place along with, andin complementary opposition to, connectionism.
Then what might be the functions and the organs of what we call
consciousness? To discuss this, we'll have to agree on what we're talking
about -- so I'll use the word consciousness to mean the organization of
different ways we have for knowing what is happening inside your mind, yourbody, and in the world outside. Here is my thesis; some people may find it tooradical:
We humans do not possess much consciousness. That is, we have very littlenatural ability to sense what happens within and outside ourselves.
In short, much of what is commonly attributed to consciousness is mythical --and this may in part be what has led people to think that the problem of
consciousness is so very hard My view is quite the opposite: that some
machines are already potentially more conscious than are people, and that
further enhancements would be relatively easy to make. However, this does
not imply that those machines would thereby, automatically, become much
more intelligent. This is because it is one thing to have access to data, butanother thing to know how to make good use of it. Knowing how your
pancreas works does not make you better at digesting your food. So consider
now, to what extents are you aware? How much do you know about how you
walk? It is interesting to tell someone about the basic form of biped
locomotion: you move in such a way as to start falling, and then you extend
your leg to stop that fall: most people are surprised at this, and seem to havewhich muscles are involved; indeed, but few people even know which
-
7/29/2019 Conscious Machines - Minsky
9/11
muscles they possess. In short, we are not much aware of what our bodies do.
We're even less aware of what goes on inside our brains.
Similarly we can ask the extents to which we're aware of the words we speak.
At first one thinks, "yes, I certainly can remember that I just pronounced "the
words we speak." But to what extent are we aware of the process that
produced those particular words? Why, barely at all! We have to employ
linguists for lifetimes of research even to discover the simplest aspects of the
language production process.
Finally, I can ask you questions like, "Can you tell me what you are thinking
about." The answers to such questions are hard to interpret. The listener mightlist the names of some subjects or concerns that were recently in mind, and
sometimes can describe a bit of the trains of thought that led to them. These
kinds of answers clearly feed upon memories of recent brain-activities. Butevery such answer seems incomplete, as though the act of probing into any
one of those memories interferes with subsequently reaching any other ones.
In any case, I cannot think of any aspect of consciousness that could operatewithout making use of short-term memories, and this suggests that the term
'consciousness' is usually used in connection with whatever processes brainsuse for accessing memories of their recent states.
This raises the question the extent to which such memories might really existinside our brains. Clearly there is a problem: if the same neural network has
been used recently for only a single purpose, then it may still containsubstantial information about what it recently did. But if it was used for
several things, then most of those traces will have been overwritten -- unlesssome special hardware has been evolved for maintaining such records. For a
modern computer, there is much less of a problem with this because we canwrite programs to store such records inside the machine's 'general purpose
memory". Of course, there will be ultimate limits on the size of such records,
but not on the nature of their contents. For example, most LISP language
systems allow the user to specify that all the activations of an arbitrary set of
program-components will have traces stored recursively. If you specifyenough of this before you run your program, then subsequently you'll be ableto find out everything it did -- and even to simulate running it backwards.
However, as we've already said, having such access does not by itself enablethe machine to make a good interpretation of those records. Certainly a certain
degree of consciousness -- in the sense of access to such records -- is
necessary for a person (or machine) to be intelligent. But even a large degree
of such 'consciousness' would not by itself yield intelligence.
So this finally leads us to some really important questions about what are the
uses of consciousness. It seems entirely clear to me that consciousness hasusefulness. It can't be what some philosophers claim: some sort of useless
-
7/29/2019 Conscious Machines - Minsky
10/11
metaphysical accessory. On the contrary, there are important ways to exploit
short term memories. For example, one has to keep out of loops -- that is,repeating an unsuccessful action many times -- which requires knowing what
already has been done. Also, after one has successfully solved a difficult
problem, one wants to "assign credit" to those actions that actually helped.This may involve a good deal of analysis -- in effect, thinking about what
you've recently done -- which clearly requires good records. Furthermore,
such evaluations must be done on various scales; did you waste the last fewmoments, and why; or did you waste an entire year? Why do we use the term
consciousness only about the shorter term memories?) On each such scale,
you'd better have an adequate array of memories. Otherwise you cannot
intelligently revise your plans, adjust your strategies, take stock of your
resources, and in many other ways maintain some control over your future. On
how many such time-scales do we work, and how many different mechanisms
are involved with each? Because we're living in the early times of psychology,no one can yet answer such questions. Clearly it is time to begin to seekconstructive ways to study them. To do this we should prepare ourselves for
coping with complexity, because it seems unlikely that so many different
functions can emerge from a single, completely new principle.
Then what is the alternative. We'll simply have to face the facts that our
many-hundred-organ-ed brain is not a useless luxury. By the time of your
birth the brain contains hundreds of specialized agencies, and by the time thatyou're an adult, most of those systems have probably grown through dozens of
stages of development. Now at various times in those first few yours, some ofthose systems create the most supremely useful of all fictions, namely, that the
unwritten novel that constitutes your life is centered on a principle protagonist
-- that you conceive of as your consciousness, like an actual person inside
your head! Some sections of [2] describe in more detail why this illusion is so
useful in life; indeed, in effect, it makes itself true. But the point of all this is
to emphasize that none of those old simplistic concepts from the past -- thosespirits, souls, and essences -- can help us with that modern task, of
understanding how all those different resources, are constructed, operated and
managed. Surely they work to a large extent as a partially cooperative parallelsystem -- but also, surely, those are largely controlled (much as Dennett
suggests in [3]) by one or several sequentially controlled systems, which in
turn are assembled from smaller parts. The first sentence in my book [2],
attributed to Einstein, is "Everything should be made as simple as possible,
but not simpler." The first step to take toward doing that is to exorcise thoseSpirits from Psychology.
[1] The Emperor's New Mind, Roger Penrose
[2] The Society of Mind, Marvin Minsky
-
7/29/2019 Conscious Machines - Minsky
11/11