Occasionally, mature industriesare turned upside down by in-novations. The years of re-search on robotics and multia-gent systems are coming togeth-er to provide just such adisruption to the material-han-dling industry. While au-tonomous guided vehicles (AGVs)have been used to move materialwithin warehouses since the 1950s,they have been used primarily to trans-port very large, very heavy objects like rolls ofuncut paper or engine blocks. The confluence of inex-pensive wireless communications, computational pow-er, and robotic components are making autonomousvehicles cheaper, smaller, and more capable.

In recent years, we have seen an increase in the useof autonomous vehicles in the field. Examples includeteleoperated military devices like iRobots Packbot andthe pilotless Predator aircraft, both of which have seenservice in Iraq and Afghanistan. The Mars rovers, Spiritand Opportunity, exemplify the use of autonomous ro-bots in scientific exploration. Closer to home, theAerosonde autonomous aircraft has been used toplumb weather systems and recently flew in tropicalstorms that are unsafe for piloted aircraft. Commer-cially, autonomous vehicles are just hitting the market.ActivMedias PatrolBot is a mobile monitoring systemfor buildings, and Aethons Tug maneuvers supply cartsaround hospitals. Robots have even penetrated thehome in an attempt to relieve homeowners of theirmost tiresome chores. iRobot sells the Roomba au-tonomous vacuum and the Scooba floor washer, andFriendly Robotics, among others, markets robotic lawnmowers.

Many more research projects are under way to buildrobots for search and rescue, mine exploration, land

Articles

SPRING 2008 9Copyright 2008, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Coordinating Hundreds of Cooperative, Autonomous

Vehicles in Warehouses

Peter R. Wurman, Raffaello DAndrea, and Mick Mountz

The Kiva warehouse-management systemcreates a new paradigm for pick-pack-and-ship warehouses that significantly improvesworker productivity. The Kiva system usesmovable storage shelves that can be lifted bysmall, autonomous robots. By bringing theproduct to the worker, productivity is in-creased by a factor of two or more, while si-multaneously improving accountability andflexibility. A Kiva installation for a large dis-tribution center may require 500 or more ve-hicles. As such, the Kiva system representsthe first commercially available, large-scaleautonomous robot system. The first perma-nent installation of a Kiva system was de-ployed in the summer of 2006.

AI Magazine Volume 29 Number 1 (2008) ( AAAI)

mine removal, and a wide variety of other intrigu-ing tasks.

Recent advances in software engineering havelaid the foundation for building large, complexsystems of autonomous vehicles. In particular, themultiagent programming paradigm has beenshown to be an effective way to build and controlcomplex systems (Jennings and Bussmann 2003).A great deal of research has focused on au-tonomous agents and multiagent systems, withthe expectation that, in the future, environmentswill be populated with hundreds or thousands ofautonomous agents. In this article, we will use theterm multiagent system (MAS) to refer to the gener-al class of systems in which autonomous agentscarry out actions and communicate with each oth-er through messages and the term multivehicle sys-tem (MVS) to refer to multiagent systems with au-tonomous, robotic vehicles. Although systemswith as many as 100 robots have been demon-strated, like the experimental CentiBot project(Konolige et al. 2004), the applicationsdisasterrecovery or terrorist eventsthankfully are notdaily occurrences. Real, mundane applicationswith more than a few vehicles have been lacking.

Recently, Kiva Systems announced availabilityof an automated material-handling system target-ed at pick-pack-and-ship warehouses. The key in-novation in the Kiva system is the application ofinexpensive robots capable of lifting and carryingthree-foot-square shelving units, called inventorypods. The robots, called drive units, transport the in-ventory pods from storage locations to stationswhere workers can pick items off the shelves andput them into shipping cartons. Throughout theday, the picker stays in her station while a contin-uous stream of robots presents pick-faces. By mov-ing the inventory to the worker, rather than theother way around, we typically see worker produc-tivity at least double. These results have beenborne out in pilot projects and at a permanent in-stallation that went online in the summer of 2006.

Unlike the commercial robotic applicationsmentioned previously, which consist of a handfulof robots per deployment, a typical installation ofa Kiva system in a large warehouse will involvehundreds of robots.

Field conditions for MVSs can be characterizedin a variety of dimensions. One is the degree towhich the environment is known or unknown.The extreme example of an unknown environ-ment is planetary exploration.1 Search-and-rescuescenarios and land-mine detection pose such chal-lenging environments that robots to perform thesetasks are still a long way from being cost effective.In contrast, the Kiva drive units operate in a con-trolled, known environment, greatly simplifyingthe design problem and making the solution prac-tical. The business case for installing a Kiva system

usually projects a one- to three-year return on in-vestment.

Another distinguishing attribute of multiagentsystems is the extent to which agents are coopera-tivein the sense that they must coordinate activ-ities to achieve a system goalor are self-interestedand have independent, often conflicting, objec-tives. Although the overall system is cooperative,the Kiva robots are essentially independent. No ro-bot depends upon any other robot to accomplishits task, although the system requires them all tosucceed to complete a customer order. Even in thiscooperative setting, the multiagent paradigm helpsbreak the system up into manageable componentswith clear interfaces, knowledge, and responsibili-ties (Lesser 1999).

Another common research topic is the resource-based nature of most multiagent systems. In a pure-ly computational setting, the resources may bememory and CPU time. In physical systems, the re-sources may include space and elements of the en-vironment that are needed to accomplish tasks. Inthe noncooperative setting, the resource allocationis typically mediated, and therefore a great deal ofthe research is focused on economic metaphors(Boutilier, Shoham, and Wellman 1997; Wellmanand Wurman 1998). With the exception of militaryscenariodriven research (Butenko, Murphey, andPardos 2003), many fewer papers address resourceallocation when agents are cooperative, and amongmost of them, the common theme is task negotia-tion (Jennings 1996, Malone et al. 1988, Rosen-schein and Zlotkin 1994). Although task negotia-tion is an approach that can be applied to the Kivasystem, the issues addressed in the literature captureonly a fraction of the complexity of the allocationproblem. One goal of this article is to present a fer-tile and interesting nonmilitary cooperative re-source allocation problem of practical importance.

The Distribution CenterWarehouses and distribution centers (DCs) play acritical role in the flow of goods from manufactur-ers to consumers. They serve as giant routing cen-ters in which pallets of products from differentmanufacturers are split, and the items are redirect-ed into outgoing containers. Figure 1 illustrates atypical DC in which incoming pallets of productsare first stored in the reserve inventory location. Asneeded, cases of product are moved into the for-ward location where they are opened and individ-ual items accumulated into shipping cartons. Thecartons may be plastic totes that are sent to a re-tailer for in-store restocking or boxes sent directlyto the consumer.

Although the tasks at the DC are necessary tothe flow of products, it is generally considered acost center in that the work done at the facility

Articles

10 AI MAGAZINE

adds no inherent value to the product. Thus, thereis constant pressure to reduce operational costsand improve accuracy. However, surprisingly littlehas changed in the industry in the last 20 years.New automation systems have been introduced,such as carousels and high-speed sorters, but themajority of these have proven to be inflexible andcostly. Thus, many warehouses still use manualprocesses or conveyors to move orders to the picklocations.

Consider, for example, what happens if you or-der two books from a highly sophisticated onlinebook retailer. A large Internet bookstore may carrya million different book titles. Although it doesntstock all of these products in its warehousessomeare shipped directly from the publisherthe com-pany likely stocks hundreds of thousands of differ-ent products. The majority of the picking done inthis type of warehouse involves what is calledopen-case picking; the retailer receives a case ofbooks and ships them out as individuals. Only oc-casionally does someone order an entire case ofbooks, and often that order is filled from a differ-ent part of the warehouse.

Naturally, when you order your books, it wouldbe inefficient for the retailer to send a picker outinto the warehouse to fetch the books for your or-der, and only your order. Instead, the retailer mayqueue up several hours worth of orders and com-pute the best way to batch work in the warehouse.Often, multiline orders are split across differentpickers because the elements of the order are stored

in distant locations. Each picker in a zone of thewarehouse fills a tote with books that are part ofseveral different customers orders. The filled toteis conveyed to a mechanical sorting machine. Atypical sorter is composed of flat trays attached toa revolving track. An induction worker empties thetote and puts the items, barcode up, one-at-a-time,on empty trays as they circulate around the ma-chine. The sorter has a scanner to identify theproduct on the tray, which allows it to dump theproduct off the tray at the appropriate momentand into a chute that serves the pack worker. Tokeep her busy, the pack worker is working on sev-eral orders at a time and thus, on receiving prod-ucts dumped off the tray, must re-sort them intothe individual orders. When the pack worker hasreceived all of the products you ordered, she putsthem together into a box, inserts the packing slip,attaches the shipping label, and sends the com-pleted box to the shipping dock.

Thus, the process of filling your order for twobooks involved four people: two pickers in differ-ent zones of the warehouse each picking batchesof products, someone to split the batches back in-to individual items on the sorter, and a fourth per-son to collect the products kicked off the sorter in-to customer orders. Moreover, it involved miles ofexpensive conveyor and sorting equipment(Gilmour 2003), which takes months to install andcannot be easily moved once set up.

Myriad automation solutions exist in the indus-try, including the aforementioned sorters and in-

Articles

SPRING 2008 11

MFG

DC

Retailer

Consumer

Reserve Forward Shipping

Pallets Cases Cartons

Figure 1. An Abstraction of Product Flow from Manufacturer to Consumer through a Distribution Center.

telligent conveyors. Automated storage and re-trieval systems (AS/RSs) can achieve very highthroughput for highly specialized and uniformproducts, like CDs. But among the automation so-lutions that are designed for a broad range of prod-uct types and sizes, the actual benefits have rarelylived up to the promises. For example, carouselsvertical shelves attached to a rotating trackap-peared promising but have proven to be bottle-necks in high throughput situations. The very bestof automation technologies claim to be able toachieve peak rates of 200 to 400 lines per hour perpicker, but most highly automated warehouses ex-perience somewhat less than that. In fact, thehigh-profile online grocer Webvan burned through

several hundred million dollars in funding to opena few, highly automated warehouses (withcarousels) but was unable to reduce costs to thepoint where it could make money on the orders.

Traditional automation approaches have severaldrawbacks. First, they are costly. Price tags for au-tomated material-handling systems typically runin the tens of millions of dollars. Second, they in-volve long design cycles. Most large DC projectstakes 12 to 24 months to bring online, in part dueto the difficulty of installing and tuning the au-tomation. Third, they are inflexible. Once in-stalled, conveyors, sorters, carousels, and other sys-tems are difficult and costly to move. They alsotend to be inflexible to changes in the inventorymix. Fourth, they are not expandable. Few au-tomation systems can be incrementally expanded,which forces companies to buy enough capacity upfront to handle several years worth of anticipatedgrowth. This results in excessive capital expendi-tures for automation systems that run under ca-pacity for several years. Fifth, they rely on batchprocessing. Like the bookseller example above, at-tempts to improve the efficiency of the pickingtask lead to aggregating orders. Once the productsare picked in batches, the warehouse employs ex-pensive automation (sorters) to undo the aggrega-tion and break the batches back up into individualorders. Sixth, they require fixed locations. Most ofthe automation systems rely on products beingstored in fixed locations, and most warehouse-management software assumes the pickable prod-ucts are stored in a single location. Like in a retailstore, a particular width of shelving must be desig-nated to each product, which makes it difficult toadjust the shelf space to accommodate changes inthe stocking level. Further, having fixed locationsmeans that the replenishment worker must movethe incoming product to that specific location inorder to restock the product. Finally, traditional au-tomation approaches require manual reslotting.Because of the batch processing and fixed loca-tions, warehouse managers are constantly evaluat-ing inventory locations to keep workloads bal-anced and the most popular products in the choic-est picking locations. Reslotting requires a lot ofmanual movement of product from one storage lo-cation to another.

The Kiva solution improves on all of these fac-tors, and in some cases, eliminates the problemcompletely.

The Kiva SolutionFigure 2 shows the physical elements of the Kivasolution: an inventory pod and a drive unit. Thedrive units are small enough to fit under the in-ventory pod and are outfitted with a mechanical

Articles

12 AI MAGAZINE

Figure 2. A Kiva Drive Unit and Storage Pod.

lifting mechanism that allows them to lift pods offthe ground. The pods consist of a stack of trays,each of which is subdivided into bins. A variety oftray sizes and bin sizes create the mixture of stor-age locations for the profile of products the ware-house stores.

Typically, a Kiva installation is arranged on agrid with storage zones in the middle and invento-ry stations spread around the perimeter. Figure 3shows an exemplary layout, with storage cells ingreen, drive units in orange, and station queues onthe left in purple and pink. The drive units are usedto move the inventory pods with the correct binsfrom their storage locations to the inventory sta-tions where a pick worker removes the desiredproducts from the desired bin. Note that the podhas four faces, and the drive unit may need to ro-tate the pod in order to present the correct face.When a picker is done with a pod, the drive unitstores it in an empty storage location.

Each station is equipped with a desktop com-puter that controls pick lights, barcode scanners,and laser pointers that are used to identify the pickand put locations. Because every product isscanned in and out of the system, overall pickingerrors go down, which potentially eliminates theneed for postpicking quality control. In general,every station is capable of being either a pickingstation or a replenishment station. In practice, pickstations will be located near outbound conveyors,

and replenishment stations will be located nearpallet drop-off points.

The power of the Kiva solution comes from thefact that it allows every worker to have random ac-cess to any inventory in the warehouse. Moreover,inventory can be retrieved in parallel. When thepicker is filling several boxes at the same time, theparallel, random access ensures that she is notwaiting on pods to arrive. In fact, by keeping asmall queue of work at the station, the Kiva systemdelivers a new pod face every six seconds, whichsets a baseline picking rate of 600 lines per hour.Peak rates can exceed 600 lines per hour2 when theoperator can pick more than one item off a pod.3

For a large warehouse, the savings in personnelcan be significant. Consider, for example, what aKiva implementation of the book warehousewould involve. A busy bookseller may ship100,000 boxes a day. With existing automation,this level of output would employ perhaps 75workers picking, sorting, and packing orders overtwo 8-hour shifts. Now consider a Kiva solution. Ata conservative 500 lines per worker per hour, and2 lines per outgoing box, the days work would re-quire 400 hours of picking. Maintaining the 16hours of picking a day, the warehouse would need25 Kiva inventory stations to generate the 200,000outgoing boxes. It would take about 200 driveunits to serve the 25 pickers at the stations, deliv-ering and storing inventory pods.4 However, the

Articles

SPRING 2008 13

Figure 3. A Small Region of a Kiva Layout.

The dark gray cells represent pod storage locations, the gray ovals the robots (with pods not pictured), and the lighter gray regions on theleft represent the queues around the inventory stations.

warehouse can be run with 50 fewer employees, at$10 per hour for 16 hours a day, which means thebook warehouse is saving $8,000 per day. With 250workdays, the net savings is $2,000,000 a year.

In addition to the productivity gains, the book-seller would experience a number of other benefitsby using the Kiva system. One such benefit isgreater accountability. Each order is filled complete-ly by a single individual, improving accuracy andaccountability by reducing the number of touch-es on the product. In addition, the booksellerwould incur no downstream dependencies, becauseno one workers productivity depends on the per-formance of workers earlier in a sequential process.Instead, each workers station is complete and self-contained.

The elimination of batch processing is a criticalbenefit. In a Kiva warehouse, everything is done inreal time. An order can literally be filled withinminutes of being received. Location-free replenish-ment is also a feature. Because any station can beused to put product away, the replenishmentprocess is greatly simplified.

Adaptive slotting is yet another benefit. Becausethe resolution of the allocation of storage is binsrather than the faces of static shelves, the systemmuch more easily adapts to changes in stockingpolicies. Every product is automatically given justenough pick faces.

An important feature of the Kiva solution is thatit has no single point of failure. Unlike a conveyor, ifa drive unit fails, it does not stop the whole floor.

The rest of the system continues to operate, andmost likely there is no noticeable impact on pro-ductivity. Another benefit is rapid deployment. Be-cause there is no fixed infrastructure, a 50-stationwarehouse can be brought online in a matter ofweeks rather than months. Kiva has set up small, 2-station systems in a single day.

Kiva systems allow spatial flexibility. Conse-quently, they can accommodate poles, flow intomultiple rooms, and handle other oddities of theenvironment. By incorporating automated lifts, aKiva installation can use mezzanines to fill the ver-tical space. Finally, the system allows for expand-ability. If a warehouse needs more capacity, onesimply adds more pods, drives, and stations.

The benefits of rapid deployment, spatial flexi-bility, and expandability combine in possiblygame-changing ways. The ease with which ware-houses can be brought online and expandedmeans that managers do not need to purchase au-tomation with the capacity to handle the volumeforecast for five years out. Instead, they need onlya big enough building, and they can buy the Kivacomponents to handle the growth as it occurs. Fur-thermore, companies can plan and deploy ware-houses in months rather than years, enabling afaster, more accurate response to changing marketconditions.

AI Techniques UsedThe Kiva system is highly influenced by AI tech-

Articles

14 AI MAGAZINE

Task Planning

Path Planning

Motion Planningand Control

Resource Allocation

Job Manager

Workflow Manager

Inventory Station Agent Drive Unit Agent

Figure 4. The Multiagent Control System.

niques, though many of the techniques used aretextbook implementations of well-known algo-rithms. The software architecture reflects the factthat the Kiva system is, by its very nature, a multi-agent system. Each drive unit and each station is acomputational device that can receive requests andact on them. At the same time, the system embod-ies a massive, real-time resource allocation prob-lem. The resources in question include shelf spaceat the station, drive units, storage pods, inventory,and physical space.

Multiagent SystemEach robot is represented in the system by a driveunit agent (DUA) and each station by an inventorystation agent (ISA). Systemwide resource allocationis centralized in the Job Manager (JM), which alsocommunicates with the customers existing ware-house-management system.5 Agents communicatewith each other through XML messages. Well over100 message types are sent among the agents.

The JM receives customer orders that need to befilled and assigns drives, pods, and stations to car-ry out the tasks. Figure 4 illustrates the multiagentnature of the architecture. The combination of theresource allocation (in the JM) with task planning,path planning, and motion planning (in the DUA)is the control stack with abstraction layers typical-ly seen in the literature (Simmons et al. 2002). TheISA on the station computer manages the GUI andthe picking lights and communicates with the oth-er agents to receive, request, and report accom-plishment of its tasks.

We found four rules useful in deciding how topartition the system into agents: (1) Physical cor-respondence: in most situations, it makes sense forthere to be one agent for each physically distinctobject. (2) Information encapsulation: each agentshould know just enough information to do itsjob. This cuts down on the amount of informationthat needs to be communicated or accessedthrough the database. (3) Single-agent ownership:all of the important data elements are owned byonly one agent. Although multiple agents mayneed the information, only one can permanentlychange it and write it to the database. (4) Separa-tion by job: when there is a resource allocation thatneeds to be done whose antecedents and effectscan be separated from other tasks, it is a candidatefor encapsulating as a separate agent.

These rules also guide feature implementationbecause the information boundaries force sometypes of solutions. The following example illus-trates some of the above points. Suppose the JMdecides that drive X should deliver pod Y to stationZ so that the worker can pick a product off the Aface of pod Y. The JM tells the station to ask driveX to fetch pod Y. The station does not need toknow where pod Y is located; if the drive unit does-

nt already have pod Y, it will look up its locationand go fetch the pod. Similarly, when the stationasks drive X to present the A face of pod Y, it doesnot tell the drive what it intends to do with pod Y,only that it needs to see face A. Once the pick tasksare complete, the station releases the pod, and theDUA, if it has no other tasks for the pod, requestsa storage location.

The benefits of the multiagent architecture canbe divided into two classes: computational and or-ganizational. The computational benefits includea natural decomposition of the computation thatcan be spread across as many servers as necessary.In addition, the multiagent design makes it clearwhere to focus effort when making the system ro-bust to failures. Every agent must be able to handlenot just error responses but also the outright fail-ure of the agents it communicates with. The orga-nizational benefits include code compartmental-ization, which makes it easier to know where toput certain functionality. The multiagent design al-so establishes clear boundaries of ownershipamong the software developers.

Path PlanningThe grid constitutes a two-dimensional graph ofpaths that may be given weights at design time.The drive units use a standard implementation ofA* to plan paths to storage locations and invento-ry stations. The DUAs also maintain a list of high-level goals and are responsible for prioritizing thegoals and accomplishing them as efficiently as pos-sible. For instance, more than one station may askfor the same pod, and a station may ask to seemore than one face of a pod. The DUA decideswhich station to visit first and in what sequence toshow the faces to minimize travel time. The DUAuses a simple AI-style planner to make these deci-sions.

Resource AllocationOur overarching design goal is to keep the pickersand replenishment workers as busy as possiblewith the least amount of hardware, warehousespace, and inventory on hand. One can imaginekeeping everyone busy by having 100 robots and acomplete copy of all of the products per worker,but it would be unnecessarily expensive. Clearly,good resource allocation algorithms are critical togetting the work done with a cost-effective amountof hardware.

Although one could describe the resource allo-cation task as one large, global optimization prob-lem, it is impractical to do so for a variety of rea-sons. First, the resource allocation decisions mustbe made in real timethere is no window in whichto do the offline computation because most cus-tomers are receiving orders throughout the day forsame-day shipping and are constantly reprioritiz-

Articles

SPRING 2008 15

ing jobs to match truck schedules. Second, theproblem descriptions are quite large and may in-clude tens of thousands of orders and hundreds ofthousands of bin choices. Third, the optimal solu-tion also depends on the actual paths and interac-tions of the vehicles, which is dynamic. Fourth,the system includes constant human interactions,with their variable, and unpredictable, responsetimes. The human variability and the dynamic na-ture of the rest of the system mean that any opti-mized solution is likely to be fragile.

Instead of attempting global optimization, wetake the approach of making decisions on the flyusing utility-based heuristics, where the utility ismeasured as the cost to the warehouse owner. Thedecisions that the system makes using utility met-rics include job assignment, pick-task assignment,replenishment-task assignment, and pod storage.

Job assignment is the task of assigning orders toworkers at stations. The system costs go downwhen we can pick more than one item off a pod,which is more likely to happen when the orders atthe station share common products. Thus, theheuristic looks at the similarity of this job to otherjobs already assigned to the station. The assign-ment problem is sometimes further constrained bythe fact that some stations or workers may havespecialized capabilities, like gift wrapping, that arerequired by the order.

Pick-task assignment involves the following: oncea job is assigned to a station, we pick the pods anddrive units that will be used to bring over the prod-ucts that go into the carton. The heuristic com-bines distance and the number of needed productsthat are available on the candidate pods whenmaking this selection decision.

Replenishment-task assignment. When the timecomes to replenish a case of product, the systemdecides where to store it. There are obviously bin-packing issues involved in selecting appropriatebins based on the dimensions of the case, but thereare other features to consider, like the current lo-cation of the pod.

Pod storage. When a station is done with a pod,the system selects an open parking spot for thepod. The location of the selected storage cell affectsnot only the time to free up the current drive unitbut also the time it will take to deliver that pod thenext time it is needed. The heuristic balances thesecosts and keeps the pods that are more likely to beused closer to the stations and the pods with slowproducts in the back of the room.

We believe that the utility-based approach givesan unprecedented amount of flexibility and adapt-ability to the system. The system runs well througha wide range of configurations and operationalchallenges. At one point during a deployment in acustomers warehouse, an electrician needed todrive a large cherry picker onto the floor to do

some electrical work in the rafters. Using Kivassoftware tools, we were able to block off a large areaof the floor and give the cherry picker access to thepole. The system routed the robot traffic aroundthe keep-out zone and was able to continue to sup-ply the workers with inventory so that they wereable to continue filling orders.

Fielded SystemsIn the preceding sections, we used a hypotheticalwarehouse to illustrate some of the drawbacks ofexisting automation and contrast them with a Ki-va implementation. We now present some resultsfrom actual Kiva deployments, to the extent per-mitted by the customers. In the spring of 2005, Ki-va set up two small installations and ran each forthree months. Since both systems were pilots, wedid only minimal integration with the existingwarehouse-management systems. More recently,Kiva installed a permanent system in a warehousein Pennsylvania for the office supply company Sta-ples.

The first pilot was conducted in a candy ware-house that supplied samples to salespeople. In thecandy warehouse before the Kiva pilot, the pickerdrove around the warehouse with a forklift fetch-ing the needed products from pallet racks and re-turned them to a sorting and packing station. Withthe Kiva system, the worker remained at the sta-tion while the candy samples came to her. Thepicker did five to six times more orders with Kivathan with the preexisting, manual system. By theend of the pilot, the Kiva robots had expandedthrough doorways into three different rooms ofthe warehouse. The project was very successful,and the warehouse manager at that facility did notwant the system taken out when the pilot contractexpired.

The second pilot took place in a distributioncenter for Staples within the business unit that fillsdelivery orders for office supplies. After a success-ful three-month pilot that proved both the effec-tiveness and robustness of the Kiva system, Staplespurchased a permanent installation for one of itswarehouses in Pennsylvania. This second systemcame online during the summer of 2006 with 30robots and five stations. Over the next threemonths, the office supply company purchasedmore Kiva equipment, until the system more thanquadrupled in size and now routinely accounts forhalf of the facilitys output each day. At well over150 robots working 24 hours day, we believe thatthis warehouse represents the largest MVS in exis-tence in a single facility. Most importantly to ourcustomer, pick workers on the Kiva side fill ordersat more than twice the rate of workers using theold conveyorized system (West 2007).

In the spring of 2007, Kiva and Staples brought

Articles

16 AI MAGAZINE

a second facility online near Denver, Colorado.This facility incorporated a new product from Kivacalled OrderFetch. In the Pennsylvania facility, asingle worker will build a carton, fill it, and thenpush it onto the conveyor to be routed around thebuilding to quality control and shipping. With theOrderFetch system, the conveyor is completelyeliminated. Instead, the cartons are transportedaround the warehouse using the same robots thatmove inventory. Figure 5 shows a station at whichthe OrderFetch pods arrive at the workers left andthe inventory pods arrive in front of the worker.Using the same laser and pick-to-light technologiesas a regular station, the Kiva system directs theworker to move products from the inventory podto the totes or cartons. The robots are used to movethe OrderFetch pods through their life cycle: at theinduction station the carton is put on the pod, atthe picking station the carton is filled with inven-tory, at the QC station the cartons inventory isverified, and finally at the shipping station the car-ton is removed and put on a truck. The empty Or-

derFetch pod is now free to return to the inductionstation to receive new, empty cartons.

In addition to these installations, Kiva has a per-manent demonstration facility in Woburn, Massa-chusetts, to showcase the many features of thepicking system. The demonstration warehouse hasseveral different station configurations, 250 inven-tory pods, and 60 drive units. The facility also in-cludes a vertical lift to demonstrate using the Kivasystem in conjunction with mezzanines. Figure 6shows the demonstration facility.

Design and DevelopmentThe Kiva Material Handling System (MHS), includ-ing all of the agents mentioned above and severaltools that are used by the warehouse staff to con-trol the order flow and to manage the hardware,was written in Java. A MySQL database is used topersist the data, though the software can use anySQL database. The vast majority of the softwarewas designed and written by a team of 13, led by

Articles

SPRING 2008 17

Figure 5. An OrderFetch Station in Which the Worker Fills Totes from Inventory Pods.

The same robots that move inventory move the tote pods.

two Ph.D.s with AI backgrounds and two morewith expertise in control systems. The hardwarewas designed by a similarly sized team of mechan-ical and electrical engineers.

In addition to the MHS, we have developed anelaborate simulation of the robotic fulfillment sys-tem using a discrete event-programming language.We use this tool to design customer installationsand to explore algorithms and analyze their sys-temic impacts. The simulation includes detailedmodeling of the drive-unit motion, pod geometry,and human operations and can be fed simulatedorder data or actual customer data. We have simu-lated installations with as many as 150 picking sta-tions, 1,500 drive units, and 20,000 storage pods.Such a facility would have the capacity of thelargest customer site we know of.

Although Kiva has built a working and cost-ef-fective system for everyday use in warehouses,there is plenty of room for improvement. To makeit easy for researchers to study MVS problems likethose found in the Kiva system, we have created anopen-source, stylized warehouse environment

called AlphabetSoup (Hazard, Wurman, and DAn-drea 2006). The AlphabetSoup warehouse mustbuild words out of letter tiles. The letter tiles arestored in buckets and moved around the ware-house by bucketBots.6

Conclusion

So far, the Kiva system has been well received bythe marketplace. As the Kiva approach spreadsthrough the industry, there are likely to be manyinteresting new computational and organization-al problems that need to be solved. In this article,we describe some of the interesting aspects of thesystem that clearly draw on an AI heritage, whilehighlighting some of the very challenging alloca-tion and design problems. In addition to the is-sues outlined here, there are a large number of ro-botic research topics that would benefit systemslike Kivas.

We have found the opportunity to work onsystems involving hundreds of robots to be veryenergizing, and we hope it will inspire others to

Articles

18 AI MAGAZINE

Figure 6. The Kiva Demonstration Facility.

work on related issues. In the very near future, welook forward to the opportunity to stand on anobservation platform in a Kiva deployment andwatch several hundred mobile robots busily get-ting work done.

AcknowledgmentsBuilding a working MVS requires a core set of greatmechanical, electrical, and software engineers. It isyet another thing to turn it into a commercialproduct and manage the manufacture, assembly,and deployment of these systems. We thank theworld-class Kiva employees who have breathed lifeinto this vision.

Notes1. It is an extraordinary engineering and scientific ac-complishment that the Mars rovers are still in operationthree solar years past their expected useful lives.

2. This statistic is based on single-unit picks and has beenreproduced for extended periods in the Kiva test facility.

3. This statistic was verified when a small Kiva demon-stration system was brought to a drugstore distributioncenter where operators picked at nearly 700 lines perhour.

4. The actual ratio of drive units to workers varies de-pending on attributes of the products and order profiles.

5. Most of Kivas customers have existing warehouse-management software that passes orders into the Kivasystem.

6. The project details and source code can be found at re-search.csc.ncsu.edu/alphabetsoup/.

ReferencesBoutilier, C.; Shoham, Y.; and Wellman, M. P. 1997. Eco-nomic Principles of Multiagent Systems. Artificial Intelli-gence 94(1): 16.

Butenko, S.; Murphey, R.; and Pardos, P. M., eds. 2003.Cooperative Control: Models, Applications and Algorithms.Berlin: Springer.

Gilmour, K. 2003. Amazon Warehouse, Amazon Adven-ture. Internet Magazine September.

Hazard, C. J.; Wurman, P. R.; and DAndrea, R. 2006. Al-phabet Soup: A Testbed for Studying Resource Allocationin Multivehicle Systems. Paper presented at the 2006AAAI Workshop on Auction Mechanisms for Robot Co-ordination, Boston, MA, July 17.

Jennings, N. R. 1996. Coordination Techniques for Dis-tributed Artificial Intelligence. In Foundations of Distrib-uted Artificial Intelligence, ed. G. OHare and N. Jennings,187210. New York: John Wiley.

Jennings, N. R., and Bussmann, S. 2003. Agent-basedControl Systems: Why Are They Suited to EngineeringComplex Systems? IEEE Control Systems Magazine 23(3):6173.

Konolige, K.; Fox, D.; Ortiz, C.; Agno, A.; Eriksen, M.;Limketkai, B.; Ko, J.; Morisset, B.; Schulz, D.; Stewart, B.;and Vincent, R. 2004. Centibots: Very Large Scale Dis-tributed Robotic Teams. Paper presented at the Ninth In-ternational Symposium on Experimental Robotics (ISER04), Singapore, 1821 June.

Lesser, V. R. 1999. Cooperative Multiagent Systems: A Per-sonal View of the State of the Art. IEEE Transactions onKnowledge and Data Engineering 11(1): 133142.

Malone, T. W.; Fikes, R. E.; Grant, K. R.; and Howard, M.T. 1988. Enterprise: A Market-like Task Scheduler for Dis-tributed Computing Environments. In The Ecology ofComputation, ed. B. Huberman. Amsterdam: Elsevier,North Holland.

Rosenschein, J. S., and Zlotkin, G. 1994. Rules of En-counter. Cambridge: The MIT Press.

Simmons, R.; Smith, T.; Dias, M. B.; Goldberg, D.; Hersh-berger, D.; Stentz, A.; and Zlot, R. 2002. A Layered Archi-tecture for Coordination of Mobile Robots. In MultiRobotSystems: From Swarms to Intelligent Automata, ed. A.Schultz, L. Parker, and F. Schneider. Dordrecht, TheNetherlands: Kluwer Academic Publishers.

Wellman, M. P., and Wurman, P. R. 1998. Market-AwareAgents for a Multiagent World. Robotics and AutonomousSystems 24(12): 11525.

West, E. 2007. These Robots Play Fetch. Fast Company117(July/August): 49.

Peter R. Wurman is an associate pro-fessor in the Department of ComputerScience at North Carolina State Uni-versity, currently on leave while work-ing at Kiva Systems, Inc. in Boston,Massachusetts.

Raffaello DAndrea received his B.Sc.degree in engineering science from theUniversity of Toronto in 1991, andM.S. and Ph.D. degrees in electrical en-gineering from the California Instituteof Technology in 1992 and 1997. Hewas an assistant, and then an associ-ate, professor at Cornell Universityfrom 1997 to 2007. He is currently a

full professor of automatic control at ETH Zurich. He isalso an engineering fellow of systems architecture and al-gorithms at Kiva Systems.

Mick Mountz founded Kiva Systemsin January 2003 after spending thir-teen years in high technology productdevelopment, manufacturing, opera-tions, and marketing. Prior to found-ing Kiva, Mountz worked on a busi-ness process team at Webvan design-ing a next-generation distributionstrategy for grocery home delivery,

during which he experienced first-hand the high cost oforder fulfillment and the inflexibility of existing tech-nologies. Mountz earned a B.S. in mechanical engineer-ing from the Massachusetts Institute of Technology andan M.B.A. degree from Harvard Business School.

Articles

SPRING 2008 19

20 AI MAGAZINE

The Fourth Conference The Fourth Conference

on Artificial Intelligence on Artificial Intelligence

and Interactive Digital Entertainmentand Interactive Digital Entertainment

AIIDE08 the Fourth Conference on Artificial Intelligence andInteractive Digital Entertainment is intended to be the defini-tive point of interaction between entertainment software develop-ers interested in AI and academic and industrial AI researchers. AI-IDE08 will include invited speakers, research and industry presen-tations, project demonstrations, and product exhibits. Whiletraditionally emphasizing commercial computer and video games,we invite researchers and developers to share their insights and cut-ting-edge results on all topics at the interface of entertainment andartificial intelligence, including serious games, entertainment ro-botics, and beyond. Because AIIDE08 crosses disciplinary bound-aries, submissions will be evaluated based on their accessibility toboth commercial game developers and researchers in addition totheir technical merit.

For details and a complete call for papers, please visit www.aaai.org/aiide08.php

AIIDE08 is sponsored by the Association for the Advancement of Artificial Intelligence (AAAI).

October 2224, 2008, Stanford University, Stanford, California