Articles

88 AI MAGAZINE

An IntroductionBenjamin Kuipers

At AAAI-15 (25–29 January 2015) in Austin, Texas, we met tocelebrate the impact of the Shakey project, which took placefrom 1966 to 1972 at the Stanford Research Institute (nowSRI International) in Menlo Park.

We researchers in artificial intelligence during this time inhistory have the privilege of working on some of the mostfundamental and exciting scientific and engineering prob-lems of all time: What is a mind? How can a physical objecthave a mind?

Some of the work going on today will appear in future text-books, even centuries from now. We gain insights into ourown struggles in the field today by learning about the his-torical struggles of great scientists of the past about whom weread in today’s textbooks. The textbooks tempt us to thinkthat they moved surely and confidently from questions toanswers. In reality, they were frequently as confused then aswe are now, by the mysterious phenomena they were tryingto understand. When we read their history, we know the

Copyright © 2017, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Shakey: From Conception to History

Benjamin Kuipers, Edward A. Feigenbaum, Peter E. Hart, Nils J. Nilsson

■ Shakey the Robot, conceived 50 yearsago, was a seminal contribution to AI.Shakey perceived its world, plannedhow to achieve a goal, and acted to car-ry out that plan. This was revolution-ary. At the 29th AAAI Conference onArtificial Intelligence, attendees gath-ered to celebrate Shakey and to gaininsights into how the AI revolutionmoves ahead. The celebration includeda panel that was chaired by BenjaminKuipers and featured AI pioneers EdFeigenbaum, Peter Hart, and Nils Nils-son. This article includes written ver-sions of the contributions of those pan-elists. —ed.

Articles

SPRING 2017 89

answers they were seeking, and we can learn from theblind alleys they spent time in, and the insights thatled them to the right paths.

Artificial intelligence marks its birth at the 1956Dartmouth Conference. There have been manyimportant milestones along the way. The importantmilestone we will celebrate today is the Shakey proj-ect, which created a physical robot that could per-ceive its environment and the objects within it.Shakey could make a plan to achieve a goal state. Andit could carry out that plan with physical actions inthe continuous world. The Shakey project laid afoundation for decades of subsequent research. Weare here to celebrate and understand that project.

The centerpiece of the Shakey celebration was apanel presentation at AAAI-15, designed to give theaudience an understanding and appreciation of theprocess of the research in the Shakey project, and ofthe long-term impact of that work on the larger fieldof AI. The goal was to have three speakers address (1)the state of the art in AI before the Shakey project (EdFeigenbaum); (2) the progress of the Shakey projectitself (Peter Hart); and (3) the impact of the Shakeyproject on the future of AI (Nils Nilsson).

Celebrating Shakey and Its BuildersEdward A. Feigenbaum

The history of science is a source of knowledge of thecomplex search for solutions to difficult problems.Not only is this history endlessly intriguing and awe-inspiring; but also it should be of particular interestto AI scientists because this kind of complex problemsolving and discovery is at the heart of many of ourtheories of mental activity.

Life is lived in the moment. Everything else ismemory and stories. The word history itself containsthe word story. This talk is constructed as several sto-ries of the Shakey project situated in its time, andamong other landmark AI projects.

I have been lucky enough to have lived andworked through the entire 60 years of AI, from early1956, months before the famous “founding” Dart-mouth Conference, to today’s AAAI-2015. My storiesare drawn from those 60 years of memories, helped,but only a little, by the best memory assistant ever,the web.

My role today is to set the historical context inwhich the Shakey project was born, lived a remark-able but short life, and was terminated. Shakeyresearch set the stage for decades of important exper-imental work in AI and robotics, and in other AIapplications that will be mentioned later by Nils Nils-son.

I phoned several well-known robotics scientists toask about the grandchildren of Shakey. All of themsaid the robots they developed were grandchildren ofShakey.

As shown in an original Shakey video, we remem-

ber Shakey as slowly and laboriously computingmodels of its environment; planning; moving andnavigating its way around obstacles toward a goal onthe far side of one large room.

Fast forward to some recent news about grandchil-dren of Shakey, from Manuela Veloso at Carnege Mel-lon University (CMU):

I am very pleased to tell you that today, on November18, 2014, the CoBot robots (3 of them) have jointlyautonomously navigated for 1,000 km in our multi-floor SCS buildings at Carnegie Mellon University!

A great-grandchild of Shakey, Stanford’s self-dri-ving car Stanley, the car that drove itself across theMojave Desert, is in the Smithsonian National Airand Space Museum in Washington, DC. Other carslike Stanley, built at Google, have driven more than700,000 miles, navigating the San Francisco Bay Areaand other roads, according to the San Jose MercuryNews of November 12, 2014. (Consider this: Shakey’straversal, integrated over the whole life of the exper-iment, probably never made it to one kilometer).

Shakey’s grandchildren on Mars are still having aproductive long life — 11 years into a planned 90-dayvisit, semiautonomously assisting planetary scien-tists.

I would now like to tell you personal stories thattogether made the importance of Shakey researchvivid to me.

First StoryIn 1993, a major Japanese corporation asked me todo an evaluation of the quality of a robotics projectthat its research lab been working on for severalyears. After signing a nondisclosure agreement, I wasshown a robot that was “humanoid,” but very big(scary, actually). Tethered to a power source, itsmotion was fluid, a marvel of modern electro-mechanical engineering.

Though heavy, it could walk reliably withoutfalling, and it could even climb a flight of stairs. Butthis creature had no Mind. It did no symbolic pro-cessing, no problem solving. It did not have goal-directed behavior.

There was more than enough space inside for a PC-sized computer and there was plenty of power. Whatthis project lacked were scientists and engineerstrained in AI, or even trained in software systems.There were no young Nils Nilssons, no young PeterHarts, no young Bert Raphaels or Richard Fikes —and of course no visionary like Charles Rosen to inte-grate AI with electromechanical engineering.

And this was 1993, twenty years after the end of theShakey project! It can be perilous to ignore scientifichistory.

Second Story The Computer History Museum in Mountain View,California, is the world’s premier museum for the his-tory of computers and information technology and is

Articles

90 AI MAGAZINE

recognized for its interpretation of that history. InJanuary 2011, the museum opened its permanentexhibition, called Revolution. Here is a quote fromthe museum’s press release:

Ten years in the making, Revolution is the product ofthe Museum’s professional staff collaborating withdesigners, content producers and more than 200experts, pioneers and historians around the world.

Revolution showcases 20 different areas of comput-ers, computer science, semiconductors, and commu-nications, from early history to futuristic visions.Among those 20 areas is one called AI and Robotics.

For each area, the museum staff has chosen onehistorical artifact to be the icon exhibit for the area.For AI and Robotics, the icon is Shakey the Robot,beautifully exhibited.

The museum could have chosen any one of adozen or more landmark AI artifacts. It could havechosen AI’s first heuristic problem-solving program(the Logic Theorist of Newell, Shaw, and Simon); or aspeech-understanding program from Reddy; or oneof the early expert systems from our Stanford group;or Deep Blue, the AI system that beat the world’schess champion. It could have … but in the end themuseum chose Shakey.

So let’s put the first story and the second storytogether to make a:

Third StorySRI’s Shakey work was a decade or two ahead of itstime in demonstrating the power of integrating AIwith robotics. Remarkably, even today, when robot-ics is being taught to high school students, and com-puting and sensors cost almost nothing, most robotsin labs and companies do not have the AI capabilitiesthat Shakey had in the 1970s.

Historians of the field have given Shakey deservedrecognition, but the field of AI had not. It took awhile for an AAAI national program committee torecognize this and make room for this celebration. Iwant to thank the AAAI-15 program committee, andhope that this will be a model for bringing forth oth-er important parts of AI’s history.

Fourth StoryThe Shakey Project was done from 1966 to 1972.What was AI and computer technology like beforeand during that period?

There is a generation of younger researchers thathave no idea how few were the powerful ideas of thefirst decade of AI (1956 to 1966) to build upon fornew AI systems. Nor can that younger generationenvision the lack of power of the computers that wehad upon which to build these systems.

But there was no lack of enthusiasm, and excite-ment; no lack of interaction, because almost everyonein the field knew almost everyone; and we all readeach other’s papers, tech reports, and books. That’swhat it’s like, when a field is small and emerging.

The AI science had a workable set of ideas abouthow to use heuristic search to solve problems. Butproving things about heuristic search had to waituntil later (the Shakey group’s A*). Some powerfulsuccessful experiments had been done: the LogicTheorist; Gelernter’s Geometry Theorem Provingprogram; Slagle’s calculus problem solving programsare examples. These were all on the “cognitive” sideof AI work. On this side, much discussion and ener-gy was focused on generality in problem solving:Newell and Simon with means-ends analysis;McCarthy and other “logicists” with theorem prov-ing.

On the “perceptual” side of AI work, a similar sto-ry can be told about research on vision. There wereseveral basic workable techniques involving line find-ing, curve finding, and putting elements togetherinto logical descriptions of objects. Generality of thetechniques was also an issue, as it still is today.

What did we have with which to do this work? Ourprogramming languages were great! List processingwas invented at CMU and then made more powerfuland beautiful in LISP at the Massachusetts Institute ofTechnology (MIT). But there was almost no interac-tion between people and computers. Time-sharedinteraction did not become available to mostresearchers in this first decade.

Try to imagine this about computer processingpower and memory: I did my thesis work on an IBM650 computer in the late 1950s: maximum 2500operations per second; memory was 20,000 digits(what we would now call bytes). Not only your pro-gram, but your language interpreter had to fit intothis memory. There was no virtual memory.

In 1959, the IBM’s large multimillion-dollar tran-sistorized computer was introduced. It ran at 100KFLOPS, and had about 150K bytes of main memory.The largest DEC computer that would have beenavailable in 1966 for the Shakey group to buy was thePDP-6, which operated at 250,000 additions per sec-ond with a memory of about 150K bytes.

Compare these numbers with, say, today’s AppleMacPro at four gigaops/sec with memory of 16 giga-bytes; or even today’s smartphones at about 1gigaop/sec but with memories going up to 128 giga-bytes.

Fifth StoryAll projects end, even the great ones. The DARPAfunding pendulum for support of AI swung awayfrom robotics and toward both knowledge-based sys-tems and the national speech understanding project.As funding shifted, SRI continued to do world-classwork in both of these other themes of the 1970s.

Final StoryThe Shakey project, as cutting edge work in comput-er science, inspired young people to do great things.In an email to Eric Horvitz, former president of AAAI,

Articles

SPRING 2017 91

Figure 1. Charles A. Rosen and the “Automaton.”

let me quote from one of these people, a junior inhigh school at the time. He and a high-school friendtraveled to visit the Shakey project in 1971, unan-nounced, but were welcomed by the Shakey team.

I was inspired by the Shakey video from SRI. I actual-ly went down and visited when I was a junior in highschool and they showed me the lab.

Shakey was pretty cool — vision, modeling, planning. Itdecided to move things around so it could go up a ramp.

Paul — do you remember how we got this video?

The “Paul” is … Paul Allen; and the author of thequote is Bill Gates.

Making ShakeyPeter E. Hart

The proposal that launched the Shakey project wassubmitted by the Artificial Intelligence Center ofStanford Research Institute (now SRI International)in January, 1965. SRI proposed to develop “intelli-gence automata” for “reconnaissance applications.”But the research motivation — and this was the inspi-ration of Charles A. Rosen, the driving force behindthe proposal — was to develop an experimental testbed for integrating all the subfields of artificial intel-ligence as then understood. SRI wanted to integratein one system representation and reasoning, plan-ning, machine learning, computer vision, naturallanguage understanding, even speech understand-ing, for the first time.

Readers interested in technical details of Shakey’sdevelopment will find an excellent summary1 in anSRI report. A 25-minute video, made by the Shakeyteam at the time, is available.2

The design of the “automaton,” as it was initiallycalled (perhaps out of a justifiable concern that“robot” sounded like science fiction, which it wasbefore Shakey), was governed by two ground rules:First, in order to keep it mechanically as simple aspossible, no arm was installed. And second, to avoidissues of miniaturization, the design evolved as anelectronics rack on wheels with a sensor assemblymounted on top.

The project team was well aware of Shakey’s limit-ed mechanical and sensory capabilities, and designeda correspondingly simple experimental environmentconsisting of half a dozen rooms populated withlarge, geometric blocks. The blocks were painted sothat edges were visible to the low-resolution TV cam-era, while still being sufficiently reflective for ourhomemade laser rangefinder to work. We also useddark baseboards, again for visibility, and exploitedthem to update the position error that accumulatedin the dead reckoning process that relied on Shakey’sstepping motors.

Our first computer was an SDS 940, an early com-mercial time-shared mainframe (whose main memo-ry was smaller than the L2 cache of most laptops). In

1970 we upgraded to a more powerful DEC PDP-10.Shakey talked to the PDP-10 through a communica-tions processor, and the system was one of the hand-ful of nodes that constituted the birth of theARPANET. Around this time we embarked on a com-plete rewrite of much of Shakey’s software, whilemaking only minor upgrades to the robot hardware.In the next section we describe this version 2 ofShakey.

Articles

92 AI MAGAZINE

Figure 2. Shakey with Components Labeled.

CameraControl

Unit

BumpDetector

CasterWheel

DriveMotor

On-BoardLogic

RangeFinder

TelevisionCamera

DriveWheel

Antenna forRadio Link

Articles

SPRING 2017 93

Figure 3. Triple Exposure of Shakey Moving Among Boxes.

Shakey’s Control SoftwareThere were two big ideas behind version 2. The firstidea was to represent Shakey’s world by statements inthe first-order predicate calculus, augmenting a formof grid model that was a key component of the firstversion (figure 4).

The second idea was to structure Shakey’s controlsoftware as a series of layers, the first time this designwas used to control a robot (figure 5). In the follow-ing we briefly describe each layer, beginning with thelow-level actions.Low-Level ActionsLow-level actions like ROLL and PAN talked directlyto Shakey’s hardware (figure 6). Also in this layer areactions like PANTO, which rotates the “head” to aspecified orientation.Intermediate-Level Actions: Markov tablesAbove this level are intermediate-level actions likeGOTHRUDOOR. These actions are put in their ownlayer because all of them are represented as Markovtables (figure 7).

One interprets a Markov table by scanning downthe left column until the first true condition isreached, executing the corresponding action, andthen looping back to the top. Accordingly, Markovtables have an inherent perseverance: they keep try-ing to do something useful. (This account is slightlysimplified, but the looping behavior is fundamentaland, as we’ll see, an important feature of thesetables.)

If these intermediate-level actions were the end ofthe software story, Shakey would be very limited inwhat it could achieve. It would only be able toachieve goals that require just a single prepro-grammed action. To do more, Shakey has to be ableto compose a sequence of actions into a plan. That’sthe job of STRIPS, the Stanford Research InstituteProblem Solver, which constitutes the next highersoftware level.STRIPS, the Stanford Research Institute Problem SolverSTRIPS came about by combining two big ideas of theday. The first was the planning strategy called means-ends analysis, as exemplified by the General ProblemSolver program of Newell and Simon.

The second big idea was theorem proving in thepredicate calculus and its application to questionanswering systems, as exemplified by the work ofCordell Green. Richard Fikes and Nils Nilsson com-bined these ideas to create STRIPS (Fikes and Nilsson1971), which applied means-ends analysis to predi-cate calculus representations (figure 8).PLANEX, the Plan Execution ExecutiveShortly after designing STRIPS, the SRI team found away to generalize a STRIPS plan by replacing constantsin the plan with variables. They also invented a datastructure called a triangle table that represents theinternal dependencies of a generalized plan. These

constructs formed the basis of PLANEX, the Plan Exe-cution Executive that is the top layer of Shakey’s con-trol software (Fikes, Hart, and Nilsson 1972).

Using this software machinery, PLANEX couldmonitor the real-world execution of a plan. It coulddetect if something had gone wrong, and couldreplan from that point, reusing portions of the exist-ing plan wherever possible. It could even be “oppor-tunistic”: If by chance Shakey was closer to achievingits goal than anticipated, it could capitalize on itsgood fortune.

This error detection and recovery ability was acritically important part of Shakey’s control soft-ware. A chasm separates planning for a physicalrobot, that has to execute plans in the real worldwhere things often go wrong, and an “abstract”planner that merely needs to print out a symbolicplan once it is computed. The Plan Execution Exec-utive, together with those persevering Markovtables, was the solution to the problem of achievingrobust, real-world plan execution.

Computer VisionThe initial project plan did not call for intensiveresearch in computer vision. Rather, the plan was tointegrate existing computer vision techniques intothe experimental test bed. But, as it turned out, verylittle technology was available, so a focused effort incomputer vision was started.

One important result of this work was the inven-tion of what could be called the modern form of theHough transform for finding lines in images (Dudaand Hart 1972). This result came about by combiningtwo concepts that on the surface appear unrelated.

The first idea is contained in a patent by PaulHough, in which he described a transform frompoints in an image plane to straight lines in a trans-

Articles

94 AI MAGAZINE

Figure 4. Predicate Calculus Model Fragment with Plan View of World and Grid Model

ROBOT

d1

d2

f3

o1

f1

r1

r3

r430

25

30

15

10

5

5 10 15 200

f2

r2

at(robot, 6.2, 10)doorstatus(d1, open)name(ol, box)inroom(ol,r2)joinsrooms(d1, r1, rs)

R A

B

C

DJ

H

GI

FK

E

Figure 5. Layered Software Architecture.

STRIPS (Symbolic Planning System)

Intermediate Level Actions

Low Level Actions

Planex (Executive)

Articles

SPRING 2017 95

Figure 6. Low-Level Actions.

Intermediate-Level Actions

ROLLTO TURNTO PANTO FOCUSTO IRISTO

OVRID ROLL TURN PAN TILT TVMODE SHOOT RANGE FOCUS IRIS

Bottom Level: Machine Language and Hardware

TILTO

TA-8973-8

Figure 7. Markov Table for GOTHRUDOOR.

infrontof(door) /\eq(s,OPEN)

near(door) /\eq(s,OPEN)

near(door) /\eq(s,UNKNOWN)

eq(s,CLOSED)

T

bumblethru(room1,door,room2)

align(room1,door,room2)

doorpic(door)

return [fail]

navto(nearpt(room1,door))

ActionCondition

form space. Intersecting lines in the latter correspondto collinear points in the form. But a problem of infi-nite slopes arises that makes this transform compu-tationally unwieldy (figure 9).

The second idea comes from an obscure branch of19th century mathematics called integral geometry.Mathematicians had theoretical reasons for using anangle-radius parameterization of a line, rather than

the more familiar slope intercept used by Hough.Peter Hart noticed that by replacing Hough’s lineartransform with a sinusoidal one, not only is the prob-lem of infinite slopes avoided, but the new transformis invariant to choice of coordinate accesses. Hart andRichard Duda also extended this method to detectanalytic curves in images, and this transform hasbeen used ever since.

Navigation and the A* AlgorithmShakey had to find its way around, so several short-est-path algorithms were developed. One, called A*by its creators, Peter Hart, Nils Nilsson, and BertramRaphael, had two very desirable properties. It can berigorously proved that (a) it always finds the shortestpath, and (b) that it does so while considering thesmallest possible number of alternatives. In non-mathematical shorthand, we can say that it alwaysworks and it’s computationally efficient.

One would think that such a strong result wouldbe eagerly accepted by any reputable publication, butit’s perhaps a sign of those times that just the oppo-site was the case. The A* manuscript was rejected bythe most prestigious journals of the day. Looking atthose old reviews, we can speculate that review edi-tors sent the manuscript to mathematicians, becauseof all those intimidating-looking theorems. Butmathematicians were unimpressed because theproofs were limited to graphs with only a finite num-ber of nodes. It seemed to the authors at the time thatmathematicians saw no difference between a graph

with ten nodes and one with ten trillion nodes, butto computer scientists that difference matters.

The paper (Hart, Nilsson, and Raphael 1968) wasfinally accepted by the IEEE Transactions on SystemsScience and Cybernetics, where it eventually gotnoticed and continues to be referenced more than 45years after it was published.

The World Back ThenThe foregoing gives a glimpse of some (though by nomeans all!) of the work done by the Shakey projectteam. To place this work in a broader societal con-text we can take a brief look at the intellectual andcultural climate of the time.

In 1970, Life,3 a popular magazine of the day, rana big story about Shakey (figure 10). The author, jour-nalist Brad Darrach, seemed to hyperventilate a bit,with a subtitle, “the fascinating and fearsome realityof a machine with a mind of its own.” But whilesome like Darrach worried that machines might takeover the world, there were deep skeptics. HubertDreyfus was one, who argued on deep philosophicalgrounds that AI is in principle impossible. And some-

Articles

96 AI MAGAZINE

Figure 8. STRIPS.

STRIPS: The Stanford ResearchInstitute Problem Solver

“GPS, A Program That SimulatesHuman Thought,” A. Newell and H.A. Simon, in Lernende Automaten, H. Billing, editor, Muchen: R. Oldenbourg, 1961.

1961

“Theorem-Proving byResolution as a Basis for Question-Answering Systems,” Cordell Green,in Machine Intelligence 4, BernardMeltzer and Donald Michie, editors,Edinburgh University Press,Edinburgh, Scotland, 1969.

“STRIPS, A New Approach to theApplication of Theorem Proving toProblem Solving,” R. E. Fikes andN. J. Nilsson, in Proceedings of the Second International Joint Conference on Artificial Intelligence (IJCAI), 1971.

1969

1971

Articles

SPRING 2017 97

Figure 9. The Hough Transform.

ρ

θ

The “Hough” Transform

Communications of the ACM, January 1972

Hough Patent

1962

Figure 10. Shakey Story in Life Magazine.Photo © by Ralph Crane, The LIFE Picture Collection.

where between Brad Darrach and Hubert Dreyfuswere labor union leaders who worried that robotsmight some day take manufacturing jobs.

Charlie Rosen was undaunted by the critics, notingthat there will always be “naysayers,” as he calledthem, whenever something new is done. The bestresponse is to push onward.

Shakey’s VisitorsThe Shakey team was generous with its time and wel-comed virtually any visitor who was interested in thework. We can get another perspective on the worldback then by viewing it through their eyes. Here aresome examples:

A school group visited, and a teacher asked what our“real jobs” were. “This robot is your hobby, isn’t it?”

A general visited and asked “Can you mount a 36-inchblade on that?”

Arthur C. Clark visited just after the movie 2001appeared, but was more interested in talking about theNew York Times review of the movie than about thefuture of robots.

A young high school student drove all the way fromSeattle to Menlo Park, California, to see Shakey.Decades later Bill Gates recalled being impressed.

A US government auditor visited and asked whetherSRI had indeed taken delivery of billions of “packets ofbits.” This question was followed by others regardingthe state of those packets, including whether therewas any tarnish or corrosion on any of those bits.

The End of the Shakey ProjectThe Shakey project ended in 1972, not for lack ofexciting ideas to pursue, but because the funding cli-mate had changed and the research program becameunsupportable. What had been achieved, as viewedfrom the perspective of 1972?

While there are likely as many views as there wereproject team members, it seems safe to make a fewbroad generalizations:

There was an appreciation that many of the individualresults — STRIPS, PLANEX, A*, and the new form ofthe Hough transform are good examples — were solidtechnical contributions.

Overall, Shakey was a significant achievement, beingboth the first mobile, intelligent robot, and also beingthe first system that integrated AI software with phys-ical hardware.

But Shakey’s overall capabilities, both mechanical andsoftware, didn’t reach the level of the initial aspirations.This would hardly be surprising, given those lofty earlygoals. Indeed, it would take decades before some werereached, while others remain as research challenges.

Today’s perspective is very different from the viewin 1972. Shakey has had impacts on both currentresearch and on the everyday lives of all of us thatcould not have been recognized or anticipated at thetime. Those impacts are the subject of the remainingsections of this article.

Shakey’s LegacyNils J. Nilsson

“Shakey the Robot” was the first system that inte-grated artificial intelligence programs (most of whichwere newly developed during the project) with phys-ical hardware. In this part of the panel discussion,

Nils Nilsson used the chart (figure 11) to list somemajor achievements of the project. In greatly elabo-rated and extended form, descendants of some of thissoftware are still in use today.

Agent Control ArchitecturesAlthough hierarchies and layers had previously beenused in software systems, Shakey was the first robotto be controlled by a layered architecture. As illus-trated earlier, there were four main layers, namely,the PLANEX (executive), STRIPS (symbolic planningsystem), the intermediate-level actions, and the low-level actions. Layered control architectures have beenused in several subsequent robot systems, amongthem the DS1’s “Remote Agent” (RAX), which con-trolled a space craft (Bernard et al. 1999), and theMonterey Bay Research Institute’s (MBARI)autonomous underwater vehicle (McGann et al.2008). Of course the control architectures of thesemodern systems, although layered as Shakey’s were,are much more complex.

Robust Action ExecutionAs described earlier, Shakey used two main tech-niques to guarantee robust action execution. Onewas Markov tables, which scanned a list of conditionsto find the first one that was satisfied by the currentsituation and then invoked the corresponding inter-mediate level action. The second was a structure we

called a “triangle table,” which stored preconditionsand actions assembled by the STRIPS planning sys-tem. These techniques evoked actions that were bothreactive to the current situation and opportunistic inunforeseen situations.

One follow-on to those techniques used by Shakeyis the concept of teleo-reactive (T-R) programs devel-oped by Nilsson and his students during the 1990s(Nilsson 1994) (figure 12).

That action associated with the first currently sat-isfied condition in the list (or tree) is the one that isexecuted. But execution continues only so long asthat condition remains the first one currently satis-fied. As soon as it is no longer satisfied, the list isscanned again to find the one now first satisfied, andso on. In the T-R formalism, the actions could them-selves be T-R programs. Dozens of papers have beenwritten about T-R programs, one book (Clark andRobinson 2016) is soon to appear, and many robotshave been controlled by them.4

Another control technique uses structures called“hierarchical state machines.” Hierarchical statemachines are similar to T-R programs except thatactions are represented by nodes and conditions bylinks. Several robots use them, including the PR2robots developed by Willow Garage and the SaviOnerobot developed by Savioke. There is a Pythonlibrary, called SMACH,5 that can be used to buildhierarchical state machines.

Articles

98 AI MAGAZINE

Figure 11. Shakey’s Achievements.

Agent Control Architectures

Robust Action Execution

Adaptive Cell Decomposition

Planning — STRIPS Rules

Heuristic Search — A*

Computer Vision

Articles

SPRING 2017 99

Figure 12. Teleo-Reactive Programs.

K1 => a1

K2 => a2

…

Km => am

Ga1

K1a2

K2

Km

am

Figure 13. Shakey’s Adaptive Grid Model.

R A

B

C

DJ

H

GI

FK

E

Adaptive Cell DecompositionShakey used a grid model, such as the one shown infigure 13,6 to map the obstacles in its environment.If a cell is not completely empty or completely full, itis divided into smaller cells and so on until one ofthese conditions is met for all cells. We believeShakey’s was the first use of an adaptive grid model.Adaptive cell decomposition is still used in robotnavigation and in computer-aided design and manu-facturing.

STRIPS RulesSTRIPS was the system Shakey used for generatingplans to accomplish goals. Figure 14 shows a STRIPSrule for modeling the action of moving a toy blockfrom C to B. The preconditions must be satisfied beforethe action can be applied, and the terms on the deletelist can no longer be guaranteed to be satisfied afterthe action is applied, so they are deleted fromShakey’s post-action model of the world. The termson the add list are added to the post-action model.

STRIPS rules (or their derivatives) are used in mostmodern planners. (The STRIPS paper gets over 5000citations on Google Scholar, and “STRIPS-style plan-ning” gets over 3410 results on Google.) It’s the rulesthat are used, not the STRIPS program itself. STRIPSrules were a practical solution to the “frame prob-lem” — inherent in the use of the “situation calcu-lus,” proposed by McCarthy and Hayes (1969) forgenerating plans.

Hierarchical task networks (HTNs) are much usedand powerful planning systems that use STRIPSrules.7 These systems assemble plan steps into net-works of actions, some of which can be executed inparallel and others that must be executed serially. Weshow an example in figure 15.

SIPE-2,8 O-Plan (Currie and Tate 1991), andSHOP29 are examples of implemented HTNs. Amongother important applications of HTNs, SIPE-2 hasbeen used for production planning at an AustralianBrewery.

Some video games make use of STRIPS rules andHTNs for planning the actions of nonolayer charac-ters (NPCs).10

Heuristic Search and A*A* is a heuristic search algorithm developed duringthe Shakey project for efficiently searching a graph ofnavigation waypoints. It uses an evaluation functionto rank the nodes reached during search and contin-ues the search below the best-ranked node. The eval-uation function for a node, n, is the sum of the costof the links traversed already on the way to n plus anestimate of the cost from n to a goal node.

There are lots and lots of descendants and variantsof A*. Here is a list of just some of them: D*, Field D*,Theta*, Real-Time A*, Iterative Deepening A*, Life-Long Planning A*, Simplified Memory Bounded A*,and Generalized Adaptive A*. Richard Korf at UCLA

and researchers at Carnegie-Mellon University haveplayed major roles in the development of many ofthese.

The Mars rover, Curiosity, uses Field D*, a deriva-tive of A* written by CMU’s Tony Stentz and his stu-dent, Dave Ferguson (now with Google). It is capableof planning paths around obstacles in unknown, par-

tially known, and changing environments in an effi-cient, optimal, and complete manner.

Most route-finding algorithms in maps use vari-ants and elaborations of A*. Elaborations include theuse of hierarchies, saved routes (which don’t need tobe recomputed), and much more.

In other uses of A*, linguists Dan Klein and ChrisManning write, “The use of A* search can dramati-cally reduce the time required to find a best parse …”(Klein and Manning 2003). And Steven Woodcock, acomputer games consultant, wrote that “A* is far andaway the most used … and most useful … algorithmfor [nonplayer-character] path finding in gamestoday …. developers have noted that they make moreuse of A* than any other tool for pathfinding.”11

(Actually, we are gratified that the major applicationsof A* are on problems a bit more serious than videogames!)

Computer VisionAs mentioned earlier, we had hoped that we coulduse then-existing computer vision routines toprocess images from Shakey’s camera. But the state ofcomputer vision was quite primitive at that time, sowe did have to develop some routines of our own.One, which influenced subsequent vision systems,was a system for segmenting images into “like-appearing regions” (Brice and Fennema 1970). Anexample of the regions found for some of the objectsin Shakey’s environment is illustrated in figure 16.Segmentation is still a major technique used in com-puter vision today.12

Another result of work on computer vision duringthe Shakey project was the development of the“modern form” of the Hough transform for findinglines and curves in images. As mentioned earlier,Richard Duda and Peter Hart modified the original

Articles

100 AI MAGAZINE

Figure 14. Application of a STRIPS Rule.

Preconditions: On(A,C), Clear(A), Clear(B)Delete List: (On(A,C), Clear(B)Add List: On(A,B), Clear (C)

On(A,C)Clear(B)Clear(A)On(C,Table)On(B,Table)Clear(Table)

move (A,C,B)

Clear(A)On(C,Table)On(B,Table)Clear(Table)On(A,B)Clear(C)

A

C B

A

C B

Articles

SPRING 2017 101

version of the Hough transform to include circles andanalytic curves and to use a rho-theta parameteriza-tion (Duda and Hart 1972). Their paper gets 4500 hitson Google Scholar.)

The modern form of the Hough transform is usedin automobiles to detect lane markings to warn thedriver about drifting out of his or her lane.13

ConclusionsOne reason for the success of the Shakey project andfor its extensive legacy is that we were the first groupto think that developing a robot that could perceive itsenvironment and make and execute plans was a feasi-ble idea. At the time, there was little existing softwarefor us to use, so we had to invent what we needed. It

turned out that the new inventions were ones that hadbroad applicability once people heard about them.

Another reason for our success is that we had avery talented team of AI researchers and softwaredevelopers, along with people who could make theconnections between software and hardware (figure17). Some team members had a reunion at SRI Inter-national in November 2014.

There are still many problems in AI where talentedresearchers could be first. An idea mentioned by Nils-son during the panel is to develop an “action hierar-chy” analogous to the deep learning hierarchies thatare being used for vision and speech recognition.14

Some of these are said to be rough models of the per-ceptual part of the neocortex. But the cortex also coor-dinates and plans actions, as illustrated in the diagram

Figure 15. A Hierarchical Task Network.

S

CLEARB

END

S J

J

PUTONAB

PUTONBC

CLEARC

PUTONC

TABLE

Figure 16. Region Finding as Used by Shakey’s Vision System.Reprinted with permission from Claude Brice and Claude Fennema, Scene Analysis Using Regions. Artificial Intelligence 1970 (3-4).

A B C

(figure 18).15 How about developing a“deep action” system, with cross con-nections to the perceptual hierarchyand using (perhaps) hierarchical rein-forcement learning to learn theactions?16 One could then try to useboth hierarchies to control a robot.

Notes1. www.ai.sri.com/pubs/files/629.pdf.2. ai.stanford.edu/~nilsson/Shakey.mp4.3. LIFE, November 20, 1970.4. For more information, see the T-R websiteteleoreactiveprograms.net.

5. See wiki.ros.org/smach.

6. The figure is from Nils J. Nilsson, “AMobile Automaton: An Application of Arti-ficial Intelligence Techniques,” Proceedingsof the International Joint Conference on Arti-ficial Intelligence, 7–9 May, 1969. Washing-ton, DC. Los Altos, CA: William KaufmannInc.

7. See en.wikipedia.org/wiki/hierarchical_task_network for more information.

8. See www.ai.sri.com/~sipe.

9. See www.cs.umd.edu/projects/shop.

10. See aigamedev.com/open/review/plan-ning-in-games.

Articles

102 AI MAGAZINE

Figure 17. Some of the People Who Worked on ShakeyFront Row, left to right: Richard Fikes, Helen Chan Wolf. Rear row, left to right: Charles A. Rosen,

Bertram Raphael, Richard O. Duda, Milt Adams, Jerry Gleason, Alfred E. (Ted) Brain, Peter E. Hart, and Jim Baer.

11. Email from Steven Woodcock sent toNilsson on 6/14/2003.

12. See, for example, en.wikipedia.org/wiki/Image_segmentation.

13. For a video of the Hough Transform inaction, see www.youtube.com/watch?v=DPApsnpPjuU/.

14. See, for example, www.cs.toronto.edu/~hinton/.

15.  Diagram from willcov.com/bio-con-sciousness/sidebars/Perception—Action%20Cycle.htm.

16. The following paper seems relevant tothis problem: Nicholas K. Jong and Peter

Articles

SPRING 2017 103

A New Approach to the Application of The-orem Proving to Problem Solving. ArtificialIntelligence 2(3–4): 189–208.Fikes, R. E.; Hart, P. E.; and Nilsson, N. J. 1972.Learning and Executing Generalized RobotPlans. Artificial Intelligence 3(4): 251–288.Hart, P. E.; Nilsson, N. J.; and Raphael, B.1968. A Formal Basis for the Heuristic Deter-mination of Minimum Cost Paths inGraphs, IEEE Transactions on Systems Scienceand Cybernetics SSC-4(2): 100–107.Klein, D., and Manning, C. D. 2003. “A*Parsing: Fast Exact Viterbi Parse Selection.In Proceedings of the Human Language Tech-nology Conference and the North AmericanAssociation for Computational Linguistics(HLT-NAACL), 119–126. Stroudsberg, PA:Association for Computational Linguistics.McCarthy, J., and Hayes, P. 1969. SomePhilosophical Problems from the Standpointof Artificial Intelligence. In Machine Intelli-gence 4, ed. B. Meltzer and D. Michie. Edin-burgh, Scotland: Edinburgh University Press.McGann, C.; Py, F.; Rajan, K.; Thomas, H.;Henthorn, R.; McEwen, R. 2008. A Deliber-ative Architecture for AUV Control. In Pro-ceedings of the 2008 IEEE International Con-ference on Robotics and Automation.Piscataway, N.J.: Institute for Electrical andElectronics Engineers.

Nilsson, N. J. 1994. Teleo-Reactive Programsfor Agent Control. Journal of Artificial Intel-ligence Research 1: 139–158.

Benjamin Kuipers is a professor of com-puter science and engineering at the Uni-versity of Michigan. He is a Fellow of AAAI.

Edward A. Feigenbaum is a professor ofcomputer science emeritus at Stanford Uni-versity, a joint winner of the 1994 ACM Tur-ing Award, a Fellow of AAAI, and an AAAIformer president.

Peter E. Hart was chair and president ofRicoh Innovations and an alumnus of SRIInternational. He is a Fellow of AAAI..

Nils Nilsson is the Kumagai Professor ofEngineering (Emeritus) in the Departmentof Computer Science at Stanford University,a Fellow of AAAI, and an AAAI former pres-ident.

Stone, Hierarchical Model-Based Reinforce-ment Learning: Rmax + MAXQ, in Proceed-ings of the Twenty-Fifth International Confer-ence on Machine Learning, July 2008.

ReferencesBernard, D.; Dorais, G.; Gamble, E.; Kanefsky,B.; Kurien, J.; Man, G. K.; Millar, W.; Muscet-tola, N.; Nayak, P.; Rajan, K.; Rouquette, N.;Smith, B.; Taylor, W.; Tung, Y. W. 1999. Space-craft Autonomy Flight Experience: The DS1Remote Agent Experiment. In Proceedings ofthe American Institute of Aeronautics and Astro-nautics, 28–30. Reston, VA: American Insti-tute of Aeronautics and Astronautics.Brice, C. R., and Fennema, C. L. 1970.Scene Analysis Using Regions. ArtificialIntelligence 1(3): 205–226.Clark, K., and Robinson, P. 2016. Program-ming Robotic Agents: A Multi-Tasking Teleo-Reactive Approach. Berlin: Springer. Currie, K., and Tate, A. 1991. O-Plan: TheOpen Planning Architecture. Artificial Intel-ligence 52(1): 49–86.Duda, R. O., and Hart, P. E. 1972. Use of theHough Transform to Detect Lines andCurves in Pictures. Communications of theACM 12(1): 11–15. Fikes, R. E., and Nilsson, N. J. 1971. STRIPS:

Figure 18. Cortex Motor and Perceptual Hierarchies.Adapted from Joaquín Fuster, The Prefrontal Cortex, 360 (New York: Raven Press.)

Prefrontal

Premotor

Primarymotor

Environment

Secondary(upper)

associative

Primary(lower)

associative

Primarysensory