Cooperative and Coordinated Assembly for Heterogeneous Robots viaDistributed Mobile Agents

Juan Rojas* Electrical and ComputerEngineering Department

Center for Intelligent SystemsVanderbilt University

Nashville, TN 37235, USArojas70@gmail.com

Richard A. Peters II** Electrical and ComputerEngineering Department

Center for Intelligent SystemsVanderbilt University

Nashville, TN 37235, USArap2@vuse.vanderbilt.edu

Abstract— Robotics technology is quickly evolving and de-manding robots to perform more actions and with greatercomplexity. Some tasks must be executed through teams ofhomogeneous or heterogeneous teams of robots. Complexrobotic systems are making used of distributed multi-agentarchitectures to facilitate the development, integration, anddeployment of such systems. Similarly, modular control isplaying an important role in rendering more flexible andadaptive controllers for complex systems. This paper presentsa team of heterogeneous robots performing an autonomous andcollaborative assembly task that is grounded on a distributedmulti-agent architecture and modular control basis approach.We conducted experimental results to assess the efficacy of thesystem and concluded that multi-agent multi-robotic collabo-rative systems with modular control approaches is a viableapproach to generate complex robotic behavior.

I. INTRODUCTION

Robotics technology is quickly evolving and demandingrobots to perform more actions and with greater complexity.Such robot systems need to perform those tasks with in-creased flexibility and autonomy [1]. Robotics is too movingtowards multi-robotic systems in which robots of differentmorphologies collaborate or coordinate with one another [2].

In order to achieve the implementation of complex roboticsystems, researchers have experienced that distributed multi-agent architectures or “mobile agents” has facilitated thedevelopment, integration, and execution of such systems[3]. Multiple frameworks have been built over the lastdecade in an effort to facilitate the development of roboticsystems. Mobile agents usually seek to abstract complexbehavior through a hierarchical taxonomy of low-level tohigh-level primitives [4], supported by strong encapsulationof code to allow fore modularity, scalability, and reusability.Distributed architectures allow to implement hardware andsoftware mechanisms across different computers, which isan increasingly important attribute as more robotic systemscontain more than one computer on-board [5].

Similarly, complex robotic systems are making use ofadaptive and flexible controllers to attenuate the limitationsposed by monolithic controllers and unstructured environ-ments [6]. In [7], an adaptive control strategy used observersand parameter update laws to estimate the stiffness and ge-ometry of objects in the environment for a 2 DoF manipulatorin simulation. In [8], a self-tuning proportionalintegralderiva-

tive (PID) controller scheduled tasks adaptively for a real-time task scheduler.

This paper seeks to assess the viability of this implemen-tation under a multi-robot, collaborative assembly task, withflexible and adaptive controllers. To this end, the authorsdeployed an agent-based distributed robotic system to per-form a collaborative assembly task through the use of an alsomodular robust and adaptive control approach known as thecontrol basis approach. The modularity and the flexibility ofthe control approach works in concert with modular agentsto bootstrap robust but flexible controllers that generate areactive and autonomous system. The authors chose to tasktwo heterogenous robots: an anthropomorphic dual-armed,six DoF, and pneumatically actuated robot ISAC [9] and arigid, point-to-point, six DoF industrial manipulator HP3JCrobot to perform a collaborative assembly task. To test theflexibility of the system, the task was executed by havingrobots switch roles. For the first experiment, the HP3JC robotwould serve as a pusher while ISAC would serve as a holder,and for the second experiment those roles would be reversed.

II. T HE INTELLIGENT MACHINE ARCHITECTURE

The intelligent machine architecture was developed at theCenter for Intelligent Systems at Vanderbilt University withthe governing principles of being a decentralized agent ar-chitecture that would facilitate the development, integration,and execution of complex robotic systems [4], [10]. To thisend, the architecture was designed to be decentralized, multi-lingual, scalable, and reusable. The architecture offers awell-defined agent model, runtime environment, data connection,and user development tools.

The agent model can implement atomic components thatencapsulate hardware or sensor resources, skills or behaviors,an environment, and finite state machines (FSMs). Theseatomic components can be combined in a hierarchical treeto form compound agents. An example could be a visualagent composed on multiple atomic components that encap-sulated frame grabbing, image buffers, color segmentation,and tracking behaviors amongst others. The agent modelfollows a taxonomy by which components are divided intofour categories: (a) Mechanisms - they represent sensor orhardware resources, skills or behaviors; (b) Representations -they represents visual, auditive, or numeric data; (c) Engines

- they represent event-driven state machines; and (d) Agents -a hierarchical component able to be comprised of multi-typecomponents.

The IMA Architecture uses Microsoft’s DCOM as its com-munication protocol. The agent model interfaces to the archi-tecture’s runtime environment though simple IDE interfaces.The runtime environment is composed of distributed layer, acontrol layer, and an application layer. The distributed layeris responsible for handling the messaging that takes placeacross components within or across computers. The controllayer maintains an organized record of all components inthe system, and the application layer provides the End-Userinterfaces and development tools. IMA counts with an intu-itive Agent Construction tool - Distributed Agent Designer(DAD), a visual debugging tool - Manager Book, and a low-level command-line debugging tool - Command Console.DAD is an GUI where one can create new components orreuse existing ones. Component interface parameters can bemodified on- or off-line for testing and system verificationprocedures. Lastly, FSMs can be manually or autonomouslytriggered in DAD.

IMA was used to create hundreds of components, tens ofagents, across six computers and 2 robots. The architectureencapsulated low-level behavior by controlling sensors asvaried as frame grabbers, force-torque sensors, encoders,pan-tilt actuation, servo motor actuation, pneumatic actu-ation, and open-and-closing grasps. Middle-level abstrac-tions included control basis controllers, visual tracking, pathplanning, homing routines, and object recognition. All ofthese middle-level abstractions were initialized, triggered,and stopped through the use of event-driven finite statemachines. At the highest level there were ’brain’ like agentsthat oversaw the smooth execution of the assembly strategyaccording to the roles enacted by each robot (these hadto be chosena priori as learning was not part of thisdemonstration).

III. THE CONTROL BASIS APPROACH

A control basis decomposes a complex control systeminto a set of modular control elements that when connectedappropriately synthesize a variety of behaviors. A controlbasis consists of any number of closed loop controllers(that represent primitive actions) derived from a set ofcontrol laws. As asserted by Huber [6], the control lawsare designed to yield asymptotically stable and predictablebehavior for different robots. That is, each carefully selectedcontrol law is designed to be robust under a wide range ofconditions and largely independent of robot kinematics. Eachcontrol law discretizes the continuous space into discretebasins of attraction. The control law can compensate for alimited range of perturbations and uncertainties while stillconverging to the attractor. Similar approaches are foundin the literature [11], [12], but the control basis frameworkis different in that it factors controllers intoobjectivesandcan combine any number of controllers through the use ofnullspace compositions in any order to achieve a wide rangeof behaviors. A careful selection of a small set of control

laws is realized as controller objectives implemented throughthe use of selected sensor and actuator resources to produceflexible structural solutions. In effect, the approach allowscontrol elements to be re-used and generalized to differentsolutions depending on the context [13]. A solution thatworks in concert with mobile agent paradigms.

A. Mathematical Derivation

We begin by describing a primitive closed-loop controllerand subsequently detailing a methodology to combine andoptimize the results of two controllers (primitive or com-pound). For our system, two robot manipulators with 6 DoFare used, whereq ∈ ℜ6 is a vector of joint angles for bothmanipulators, andx ∈ ℜ6 is the vector of position andorientation for a robot’s end-effector.

Primitive controllersφi, wherei = 1 ∼ n, are elements ina basis of controllers,Φ, such thatφi ∈ Φ. A primitivecontroller optimizes a partitioned portion of a designatedcontrol space and can be understood as the minimizationof a discrete basin of attraction. The basins of attractionare formulated through artificial potential functions definedover a typed domain (such as cartesian positions), which aredefined as the square of the error:

φi(ρ) = ρTρ (1)

where the error,ρ, is the difference between the referenceinput and the plant input,ρ = qref −qdes, at every time step.

Each controller reaches its objective by performing greedydescent,∇φi, on the artificial potential function, whileengaging sensor and motor resources. The minimization ofthe surface potential function in a specified domain space,Xi, is defined as:

∇Xiφi =

∂φi

∂Xi

. (2)

Each primitive is bound to a selected subset of inputcontrol resourcesγj ∈ Γj and output control resourcesγk ∈ Γk relevant to the task. In order to bind input andoutput control resources to the controller, correspondingsensor transforms,sj , and an effector transforms,ek, areused. Thesensor transformmaps incoming sensory resourcesto a specified domain space such thatsj : Γj → Xi. Toensure that a task is guaranteed to operate within the regionof a corresponding basis we require that the output range ofa sensor transform matches the artificial potential functiondomain’s data type. Similarly, theeffector transformmapsthe control law error’s result to an appropriate output space,Yk. The mapping is typically effected using a Jacobian matrixas in Equation 3.

ek(Γl) =

(

∂φxγ1

∂yk,∂φxγ2

∂yk, ...,

∂φxγ |Sl|

∂yk

)T

, (3)

where,φxγ1represents the controller update for a sensor

control resourceγ1. yk is a corresponding point in the outputspace andΓl = γ1, γ2, ..., γ|Sl| is a subset of selected controlresources for a given task. The effector too is a function ofΓl. In order to match an effector transform with an artificial

potential function, the rowspace ofek(Γl) must match thepotential function’s data type.

In conclusion, a closed-loop controller is implementedwhen the error between the incoming sensor informationand the reference position are minimized within the discreteartificial potential basin,∇xi

φi(Xref − sj(Γj)), and havingthe gradient result mapped onto the output configurationspace through an effector transform,ek(Γl). Given that theinput data is of the same domain type as the artificialpotential function, and the effector transform is of the samedimensions as the potential function, the controller’s output,∇yk

φi, is defined as:

∇ykφi = ek(Γl)

T∇xiφi(xref − sj(Γj)). (4)

For convenience, the above expression is expressed in sim-plified notation as:

φi |sj(Γj)

ek(Γl)(xref ). (5)

To concurrently optimize multiple control laws in a sys-tem, we use a method originally implemented by Platt [14]based on the Moore-Penrose pseudo inverse to project asecondary control update to the nullspace of the primary ob-jective’s equipotential manifold. In this context it is said thatwithin the compound controllerπ the secondary controllerφ2 is subject-tothe primary controllerφ1, and expressed as:

∇y(φ2 ⊳ φ1) = ∇yφ1 +N (∇yφT1 )∇yφ2, (6)

where,N (∇yφ

T1 ) ≡ I − (∇yφ

T1 )

+(∇yφT1 ), (7)

and, I, is the identity matrix,y is an n-dimensional space,and ∇yφ

T1 is a (n-1) dimensional space orthogonal to the

direction of steepest descent [15]. For convenience, Equation(7) is written as:

πk : φ1 ⊳ φ2 (8)

The nullspace operator,N (∇yφTi ), encompasses anullspace

compositiontechnique that optimizes the concurrent execu-tion of two controller. Unlike traditional methods [16], thereis no need to specify how control resources will be sharedacross sub-controllers as long as the same control resourcesare used.

B. Control Policy Implementation

Once a number of defined primitives,Φ, sensor transforms,s, and effector transforms,e, have been determined for acontrol problem, a policy must be enacted to complete thetask out of the combinatoric basis:Φ×2s×2e. By selectinga well defined set of basis controllers, the sequencing of sub-goals eases the need for complex control layers or switchingcriteria. Sequencing of primitive or compound controllerslimits the set of tasks that can be addressed and improvepredictions across controller sequences [6]. The sequencing,in effect, becomes an instruction set that strings subgoalstoachieve an overall task [13].

In this demonstration, the finite state automata consists oftwo states for the pushing robot and one state for the holding

robot. The transition policy amongst these controllers isexplained here first and their derivation presented in sectionIV. For the pushing robot, the first state starts the controlcomposition that advance the male truss towards the femaletruss. The second state, deploys the insertion of the trusses.For the holding robot, a counter-balancing controller thatopposes any motion if presented.

In conclusion, a sequence of concurrently combined con-trollers encodes an instruction set for complex tasks. Themodularization of a control problem in this way preventsmonolithic control and reduces the need for complete andaccurate system models; thus easing complexity [6]. Theenaction of a control policy of this kind using the controlbasis framework allowed two robots of very different mor-phologies to perform cooperative assembly tasks flexibly androbustly.

IV. DEVISING A CONTROL BASIS FORCOOPERATIVE ASSEMBLY TASKS

The assembly task performed by our robots work consistsof an insertion where a male truss is inserted into a femalefixture. Such an insertion begins by having the male trusspositioned at a point in space in front of the female trussthrough a mobile platform. The male truss may offset fromthe female truss in all three planes as long as it remainsin the workspace of the robot team. The insertion proceedsby having the male truss move in a linear fashion to anoptimal insertion point in front of the female truss. To thisend, the male truss has an inverted chamfer at the end tosimplify the entry, while the female fixture has a cylindricalend of a diameter slightly larger than the truss’. A numberof controllers were designed to respond to the push-roleconsisting of two stages: (a) a guarded approach, and (b) acompliant insertion [17]. For an insertion task to be executedsuccessfully a robot must be able to displace the male trussto an optimum location for insertion. If the final position ofa male truss during an approach motion ends at a locationoutside the interior hole of a female fixture, the assemblycannot succeed.

Similarly, if during the approach, the male truss jamsthe fixture, a successful insertion is difficult. A series ofprimitive controllers are used, concurrently combined andse-quenced to produce two hierarchical controllers: theGuardedMove Controller and the Compliant Insertion Controller.The former generates a guarded approach that positions thetool at an optimum location for insertion. Upon reachingan appropriate insertion position the finite state machinemoves to the next state initiating the latter controller, whichdrives an insertion. To do so successfully, misalignments arecorrected to decrease friction and resolve jamming and wedg-ing phenomena. Both of these controllers were used by anindustrial robot; while, a modified version of these was alsoused by a dual-arm humanoid robot (the modified versionaccounted for the combined effect of two serial-link chains).Additionally, a third compound controller was designed forthe hold-role. This controller seeks to emulate the rigid holda human would enforce when a second person pushes a part

Concurrent

Controllers

Pusher Holder

Push-Hold Coordination Scheme

Legend

Moment

Position

Guarded Move

Controller

Compliant Insertion

Controller

Force

Moment

Virtual Counter Balance

ControllerNullspace

Projections

Controller

Transition

- Force

Moment

Fig. 1. The HP3JC inserts a male truss into a female counterpart held byISAC. Pushing requires two compound controllers effected through the nullspace approach. Holding requires one compound controller.

to produce an insertion. The controller seeks to counter-actthe experienced forces while simultaneously displacing theorientation of the part to facilitate the entry of the matingtruss. An illustration of the push-hold coordination schemecan be seen in Fig. 1.

A. Guarded Move Controller

The guarded move controllerπGM , uses a dominantposition controller,φp, and a subordinate moment residualcontroller,φmr. The order is decided empirically by prioritiz-ing the need for an optimal insertion location. The positioncontroller displaces the rigidly held truss to such location,(xref ), and a subordinate moment controller minimizesperturbations if contact is made during the trajectory. Theposition controller receives a 3D reference cartesian positionfrom a stereo visual system that detects color fiducial marksplaced at the fixture’s tips [18].

For the position primitive, the sensor transform convertsimage coordinates to cartesian positions:sp(γvisual sys),while the effector transform maps the updated cartesianposition to the robot’s current joint configuration:ep(γjoint).On the other hand, the moment residual controller has asensor transform,smr(γmoment), that returns the momentsexperienced by the F/T sensor, and has an effector transformemr(γtorque), that converts torque updates into joint angleupdates. The composite guarded move controller,πGM , issynthesized by having the moment controller be subject-tothe position controller and defined as:

πGM = φmr |smr(γmoment)emr(γtorque)

(mref )⊳φp |sp(γvisual sys)

ep(γjoint)(xref ).

(9)

B. Compliant Insertion Controller

The compliant insertion controller minimizes residual mo-ments and forces experienced during the assembly’s insertionstage. As stated earlier, experimental practice suggests thatthe aligning of the truss takes precedence over its positionduring the insertion stage. Hence, the hierarchical controller

is composed of a dominant moment residual primitive,φmr,and a subordinate force residual controller,φfr. The subor-dinate controller uses a force reference(fref ) to generate thedriving force to execute the insertion. The dominant momentresidual controller aligns the truss as it is inserted into thefixture. The alignment is a function of the experienced forcesproduced as the truss collides with the fixture’s interior wall.The compliant insertion controller is defined as:

πCI = φfr |sfr(γforce)

efr(γtorque)(fref ) ⊳ φmr |

smr(γforce)

emr(γtorque). (10)

Screen-shots of the assembly demon are shown in Fig. 2.

C. Counterbalance Controller

As part of the push-hold schemes, a composite controllerwas devised to implement a force guided system that wouldmaintain the static fixture’s position in place while updatingits orientation. Following the design evidence of the compli-ant insertion controller, the counterbalance controller,πCB,seeks to facilitate the insertion process and is composed ofadominant moment residual controller and a subordinate forcecontroller:

πCB = φfr |sfr(γforce)

efr(γtorque)(−fref)⊳ φmr |

smr(γforce)

emr(γtorque)(11)

Though similar to the compliant insertion controller,πCB,opposes the force applied by an incoming truss reference anddisplaces the end-effectors so as to create an optimal entryangle for the mating truss. In this way, it minimizes momentand force residuals throughout the task. A similar version ofthe three prior controllers was adapted to be used with ISAC[19].

V. EXPERIMENTS

The demonstration seeks to achieve collaborative and co-operative assembly under a multi-agent distributed architec-ture. The latter also encapsulates the controllers introduced

Fig. 2. Experiment 6: Two heterogeneous robots cooperate toperform ajoint assembly using force sensing under a push-push coordination scheme.

in Sec. IV to implement parts insertion as outlined in Sec.IV-B. Before presenting the experimental set-up details, thehardware is described below.

The testbed consists of an: HP3JC with a JR3, six-axis F/Tsensor, and a Barret Hand mounted on the wrist; and an in-house built humanoid robot, ISAC. The anthropomorph hastwo manipulators, each actuated by 12 pneumatic McKibbenartificial muscles. Each of ISACs end-effectors consist of anATI six-axis F/T sensor and a machined aluminum bracketspecially designed to hold the truss. Truss’, both male andfemale were made from commercial PVC piping. The maletruss was composed of two 0.5 in. pipes connected by anelbow connector. At the truss’ tool-tip an inverted chamferwas used to facilitate its entry into the female counterpart.The female truss was a 1.0 in. pipe connected through at-connector to two 1.0 in. pipes that were held rigidly byISAC’s aluminum brackets. Additionally, color segmentationwas used with empirically set parameters for a low-pass filterand morphological operations. ISAC used image processingto compute the cartesian coordinates of both trusses withvery good results and little noise.

A. Experiment 1: HP3JC-Push, ISAC-Hold

In the first demonstration, the industrial robot drove a maletruss into a female fixture held by ISAC. The HP3JC robotused theπGM and theπCI controllers to actively perform theinsertion, while ISAC used theπCB controller to counteract,but optimize entry forces exerted by the industrial robot.Note the reference parameter for the compliant insertioncontroller—a force reference—is what enacts the forwardmotion of the truss. In other words, this parameter affectsthe speed and force of the insertion. Force sensing, in thissense, drives the insertion, while adjusting the position ofthe truss in the vertical and horizontal planes.

Additionally, three metrics were used to measure theassembly tasks’ performance: (a) time-to-completion, (b)thesum of the absolute value of moment residuals in the x-, y-,and z-directions (referred to hereafter as “moment errors”),and (c) the reference force parameter. The latter was usedto distinguish between faster and slower insertions drivenbythe industrial robot. Faster insertions used a force referencevalue of Fx=40 lbs and slower insertions used a value ofFx=20 lbs. Insertions were assumed complete after the malefiducial mark was covered.

Sensory data for one trial is presented for both robots inFig. 3(a) and Fig. 3(b) in three sub-plots representing the:(i)force and torque signatures, (ii) the moment residuals, and(iii) the force residuals.

The data for this demo shows the assembly was completedin 54 seconds. The duration of this and other experimentsmay seem significant. The primary reason for such durationswas the inability to access the industrial robot’s low-levelcontrol loop. Such impediment forced us to operate throughthe industrial robot’s API which prevents pre-emptive motionand significantly delayed the overall response of the system.With respect to moments, note that residual in the y-directionwas reduced efficiently by both the HP3JC and ISAC, and

5 10 15 20 25 30 35 40 45 50−4

−2

0

2

JR3 Sensor Force Torque Data

Time (seconds)

For

ce (

lbs)

Tor

que

(in−

lbs)

FxFyFzTxTyTz

0 10 20 30 40 50 60 70 80 90 100−4

−2

0

2

4CB: Error for Dominant Moment Controller

Sample Index

Mom

ent E

rror

TxTyTz

10 20 30 40 50 60 70 80 90

10

20

30

40

CB: Error for Subordinate Force Controller

Sample Index

For

ce E

rror

FxFyFz

(a) Force data for the HP3JC-pushing robot

0 10 20 30 40 50 60−4−2

024

ATI Left FT Sensor

Time (seconds)

For

ce (

lbs)

Tor

que

(in−

lbs)

FxFyFzTyTxTz

0 10 20 30 40 50 60−2−1

01

ATI Right FT Sensor

Time (seconds)

For

ce (

lbs)

Tor

que

(in−

lbs)

FxFyFzTyTxTz

5 10 15 20 25 30 35−2−1

01

CB: Error for Dominant Moment Controller

Sample Index

Mom

ent E

rror

TyTxTz

5 10 15 20 25 30 35−0.2

00.20.40.6

CB: Error for Subordinate Force Controller

Sample Index

For

ce E

rror

FxFyFz

(b) Force data for the ISAC-holding robot.

Fig. 3. Exp 1: Force signatures for the HP3JC robot and ISAC.

moments in the z-direction maintained a small presence inboth robots. The presence of moments in the z-directionindicate the approach by the truss also contained a horizontaldisplacement. Small residual moments in the y- and z-directions were expected given that there is a difference indiameters between the male and female, which allows thetruss’ to exert mutual load from the spacing between them.Additionally, for ISAC, the virtual counter balance controllerseeks to eliminate force errors by adjusting the female fixtureposition. Forces in the x- and z-directions converged to zero.The y-direction residual force error is a response to thepresence of moment in the z-direction from the male truss,which had not fully converged by the time the assembly wasfinalized.

A summary of metric results across trials is shown in Fig.4. The first three trials were run under the slower forcereference value Fx=20, which in general, yielded slowerassemblies as opposed to the trials run with Fx=40. Withrespect to moment residuals, the HP3JC robot experiencedmoment errors greater than those experienced by ISAC. Thelast five experiment trials used the faster force reference

1 Fx=20 2 Fx=20 3 Fx=20 4 Fx=40 5 Fx=40 6 Fx=40 7 Fx=40 8 Fx=40 0

10

20

30

40

50

60

70

80Time−to−Completion vs. Sum of Abs. Value of Moment Residuals

Trials

Mag

nitu

de

TimeHP3JC MomentISAC Moment

Fig. 4. Exp.1: Results summary across 8 trials.

value Fx=40. The higher force reference led togenerallyshorter completion times and hinted at larger moment errorsfor both robots than the ones registered in the first three trials.Two of the first three trials contained larger than normalmoment errors due to one wedging in one trial and jammingin another.

B. Experiment 2: ISAC-Push and HP3JC-Hold

The second demonstration reversed roles and assignedISAC as the active robot in the insertion task. This experi-ment, as opposed to the previous one, began by having themale truss held by the HP3JC at a ready-position in frontof the female fixture held by ISAC. The humanoid’s virtual-contact compliant insertion controller,πV CI was responsiblefor driving the insertion. In this experiment, a force referenceparameter of Fx=-0.5 lbs was used by the subordinatecontroller of the virtual compliant insertion controller forISAC. The results are shown in Fig. 5.

Note that the moment residual errors are a result of theaverage moment’s contribution from both the right and leftF/T sensors. A slower speed was selected for the motion ofthe pneumatic actuators to achieve greater accuracy. Fromclock time 1–200 seconds, two important patterns are present

0 50 100 150 200 250

−6−4−2

02

ATI Left FT Sensor

Time (seconds)

For

ce (

lbs)

Tor

que

(in−

lbs)

FxFyFzTyTxTz

50 100 150 200 250

−10−5

05

ATI Right FT Sensor

Time (seconds)

For

ce (

lbs)

Tor

que

(in−

lbs)

FxFyFzTyTxTz

20 40 60 80 100 120−1

0

1

CB: Error for Dominant Moment Controller

Sample Index

Mom

ent E

rror

TyTxTz

20 40 60 80 100 120

−0.6−0.4−0.2

0

CB: Error for Subordinate Force Controller

Sample Index

For

ce E

rror

FxFyFz

Fig. 5. Exp 2: Quick changes in torques due to stiction effects caused byISAC’s compliant nature.

in the data: a) there is a roughly constant increment in themoment residual error in the y-direction, and b) the forcereference force is gradually canceled over this period. Bothpatterns indicate the existence of stiction in the task. This isfurther corroborated by the quick fall seen in the left torquereading in the y-direction and the quick increase in the forcereference value Fx in the negative direction. The controllersare unable to decrease residuals caused by stiction until thecorrective force generated by the controllers overcome thesticking forces present through the artificial muscles. Afterthe sticking forces were overcome, the insertion took placequickly. The sudden completion of the task did not allow thecompound controllers enough time to converge back to theirreference goals.

For ISAC the averaged moment errors generated in thisexperiment were greater than the ones in experiment 1. ISACexperiences greater stress as it drives the insertion than it didwhen it used the counter balance controller. On the otherhand, the averaged moment residual errors experienced byISAC are lower than the ones experienced by the HP3JCrobot. This suggests that the use of artificial muscles easesimpact or stress induced as compared to rigid industrialrobots.

A summary of the results across six trials is found in Fig.??. All trials in this experiment were run with the sameforce reference value. Three trials experienced longer times-to-completion due to more prominent stiction phenomena.

VI. D ISCUSSION

As experimental results showed, the collaborative as-sembly task across a heterogeneous multi-robot team wassuccessfully tasked. The multi-agent architecture fulfilledits design paradigm and was able to encapsulate low-leveland high-level abstractions. The middleware communica-tion layer functioned well throughout the experiments anddistributed a vast amount of visual and numeric data todifferent components in a decentralized manner. A numberof finite state machines were run under different agent enginerepresentations to automate all aspects of the robotic system.

1 Fx=−0.5 2 Fx=−0.5 3 Fx=−0.5 4 Fx=−0.5 5 Fx=−0.5 6 Fx=−0.5 0

50

100

150

200

250

300Time−to−Completion vs. Sum of Abs. Value of Moment Residuals

Trials

Mag

nitu

de

TimeISAC Moment Error

Fig. 6. Exp.2: Summary for results across 6 trials.

Additionally, the control basis framework allowed for themodular implementation of the basis controllers within theoverall paradigm of our agent architecture. Basis controllerswere successfully compounded and sequenced. The controlpolicy to enact the controllers was successfully implementedat the highest level of abstraction allowing for a smoothtransition of compound control basis controllers.

The overall system was flexible in that by selecting welldefined control laws, a variety of controllers for manip-ulation, insertion, and counterbalancing were deployed intwo very different robots with two very different physicalcharacteristics. The results presented above show that thecontrollers reduce moment and force errors over time overthe duration of the insertion task. Even in situations were dis-turbances like stiction were present (Exp. 2), the controllerswere able to overcome such phenomena and successfullycomplete the assembly task.

While there positive aspects of this model have beenpresented, the author would also like to comment on thechallenges posed by such a system as well. That is, thatas the number of robots increase, and along with it, thenumber of sensory and hardware modules, so does the sizeof multi-agent architecture. While these architectures aredesigned to scale easily and facilitate integration, thereisa very significant overhead needed to set them up. Moredevelopment tools that can aid in the automatic initializationand resetting, and the switching on–or–off of debugginginformation would be very desirable.

VII. C ONCLUSION AND FUTURE WORK

This paper presented a team of heterogeneous robotsperforming an autonomous and collaborative assembly taskthat is grounded on a distributed multi-agent architectureand modular control basis approach. The modularity and theflexibility of the agent-based architecture and the controlapproach worked in concert to bootstrap robust, flexible,and decidedly reactive controllers. Experimental resultscon-cluded that multi-agent multi-robotic collaborative systemswith modular control approaches is a viable approach togenerate complex robotic behavior.

For future work, the authors would like to extend the role-playing that robots play in assembly tasks, mimicking humanbehavior. On occasion, when two entities (humans, or twoarms of a human) are trying to assemble an object, and theinsertion is not easily accomplished, humans tend to try topush from both ends simultaneously. This kind of assemblyis prone to higher forces, quicker motions, and error buthumans practically choose such an approach in an attemptto assemble parts. Such a task would be an interesting testto examine the robustness and flexibility of our system.

VIII. ACKNOWLEDGMENTS

The authors gratefully acknowledge the contribution ofNational Research Organization and reviewers’ comments.

REFERENCES

[1] T. Brogardh, “Present and future robot control development–an indus-trial perspective,”Journal of Annual Reviews in Control, vol. 31, no. 1,pp. 69–79, 2007.

[2] J. Rojas and R. A. Peters II, “Preliminary results in force guided as-sembly for teams of heterogeneous robots,”SPIE’s Defense, Security,and Sensing. Accepted and pending publication., April 2009.

[3] J. Kramer and M. Scheutz, “Development environments forau-tonomous mobile robots: A survey,,”Autonomous Robots, vol. 22,no. 2, p. 101132, 2007.

[4] R. E. Olivares, “The intelligent machine architecture version 2.5: Arevised development environment and software architecture,” Master’sthesis, Vanderbilt University, 2003.

[5] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. B. Foote, J.Leibs,R. Wheeler, , and A. Y. Ng, “Ros: an open-source robot operatingsystem,,” inICRA workshop on Open-Source Software, 2009.

[6] M. Huber, “A hybrid architecture for adaptive robot control,” Ph.D.Dissertation, University of Massachusetts, Sept. 2000.

[7] M. Namvar and F. Aghili, “Adaptive force control of robots inpresence of uncertainty in environment,”In the Proceedings of the2006 American Control Conference, pp. 3253–3258, June 2006.

[8] P. Yuan, “An adaptive feedback scheduling algorithm forrobot as-sembly and real-time control systems,”In the Proceedings of the IEEEInternational Conference on Intelligent Robots and Systems, pp. 2226–2231, October 9-15 2006.

[9] J. Rojas and R. A. Peters II, “Sensory integration with articulatedmotion on a humanoid robot,”Journal of Applied Bionics and Biome-chanics, vol. 2, no. 3-4, pp. 171–178, 2005.

[10] R. T. Pack, “Ima: The intelligent machine architecture,” Ph.D. Dis-sertation, Vanderbilt University, 2201 West End Avenue, Nashville,Tennessee 37235, May 1999.

[11] J. Jameson and L. Leifer, “Automatic grasping: An optimizationapproach,”Proceedings of the IEEE Transactions on Systems, Man,and Cybernetics, vol. 17, no. 5, pp. 806–813, Sept. 1987.

[12] J. Son, R. Howe, and G. Hager, “Preliminary results on grasping withvision and touch,”In the Proceedings of the IEEE Conference onIntelligent Robots and Systems, vol. 3, pp. 1068–1075, Nov. 1996.

[13] O. Brock, A. Fagg, R. Grupen, R. Platt, M. Rosenstein, andJ. Sweeney, “A framework for learning and control in intelligenthumanoid robots,”International Journal of Humanoid Robots, vol. 2,no. 3, pp. 301–336, 2005.

[14] R. Platt, A. H. Fagg, and R. A. Grupen, “Nullsapce composition ofcontrol laws for grasping,”In the Proceedings of the IEEE Interna-tional Conference on Intelligent Robots and Systems, pp. 1717–1723,2002.

[15] R. J. Platt, “Learning and generalizing control-basedgrasping andmanipulations skills,” Ph.D. dissertation, University ofMassachusettsAmherst, 2006.

[16] J. A. Coelho and R. A. Grupen, “A control basis for learningmultifingered grasps,”Journal of Robotic Systems, vol. 14, no. 7, pp.554 – 557, 1997.

[17] H. Inuoe, “Force feedback in precise assembly tasks,”In ArtificialIntelligence: An MIT Perspecrive, vol. 2, pp. 219–241, 1981.

[18] J. Rojas and R. A. P. II, “Preliminary results in force-guided assemblyfor teams of heterogeneous robots,” inProceedings of the SPIEConference on Defense and Security, vol. 7332, no. 1, 2009.

[19] J. Rojas, “Autonomous cooperative assembly by force feedback usinga control basis approach,” Ph.D. dissertation, VanderbiltUniversity,2009.