NRI-Large: Collaborative Research: Human-robot Coordinated Manipulation and Transportation of Large Objects

PI: Stergios Roumeliotis (UMN) Co-PIs: Demoz Gebre-Egziabher (UMN), Vijay Kumar, Dan Lee (UPenn), Steve LaValle (UIUC), Paul Oh (Drexel), Ioannis Poulakakis (U Del)

Objective •  Develop the science and technology necessary for

rea l i z ing human-robot cooperat ive object manipulation and transportation

References [1] Bureau of Labor Statistics [2] S. Calinon, P. Evrard, E. Gribovskaya, A. Billard, and A. Kheddar. Learning collaborative manipulation tasks by demonstration using a haptic interface. ICRA’09 [3] A. Thobbi, Y. Gu, and W. Sheng. Using human motion estimation for human-robot cooperative manipulation. IROS’11 [4] M. Vahedi and A. F. van der Stappen. Caging polygons with two and three fingers. IJRR’08 [5] K. Yokoi, F. Kanehiro, K. Kaneko, S. Kajita, K. Fujiwara, H. Hiukawa. Experimental study of humanoid robot HRP-1S. IJRR’04

Motivation •  People often collaborate to lift, hold, and carry

heavy objects (e.g., panels, pipes, pieces of furniture, etc.) through areas inaccessible to hand trucks or forklifts (e.g., narrow passages, stairs)

•  6M workers engaged in manually carrying, lifting freight and stock (construction sites, warehouses, etc.) Back injuries: nation’s primary workplace safety problem; 75% of them during lifting; on avg. 10 working days lost per incident [1]

•  Humanoid co-workers: Possess sensors for perceiving the world, arms/hands for grasping and manipulating objects, are bipedal and can maneuver in human-centric areas w/ steps & stairs

•  Complementary capabilities of humans (perception, cognition) and humanoids (strength, stamina)

Broader Impact •  Socio-economic impacts of flexible human-

humanoid robot material-handling unit − Increase productivity and reduce cost of manufacturing, construction, and warehousing: no need for semi-permanent infrastructure; easily reconfigurable production/assembly lines − Reduce lifting related accidents and injuries − Improve quality of life of the elderly and people with disabilities (home automation)

•  K-12 Education and outreach activities − RoboTech Fellows program for pairing K-12 students and educators with NRI researchers for enriching STEM curriculum, invigorating robotics competitions, reaching out to and attracting under represented groups − Public engagement through interactive demonstrations involving humanoid robots at museums, libraries, and schools

Research Plan •  Research embodiment: Humanoid co-worker

that (i) acts as a follower carrying most of the load; and (ii) communicates indirectly based on body posture and actions of the human (e.g., pointing at the object to be lifted, pulling the object towards its destination)

•  State Est imation and Environment Perception: Develop active sensing and information fusion algorithms for determining the robot’s pose and creating a 3D map of the area and the structures within it (e.g., location, dimensions of objects to be manipulated, stairs, doorways, etc.)

•  Human-posture Estimation: Construct analytical and learned models of human motion for fusing data from IMU sensors on the person’s body and visual and force/torque sensors on the robot to determine the person’s current posture and predict her motion

•  Grasping and Manipulation: Introduce cooperative grasp planning and manipulation algorithms based on models of human-robot grasp synergies that satisfy the dynamic constraints of locomotion and ensure safe operation

•  Coordinated Locomotion: Design locomotion controllers that allow humanoid responsiveness to the human’s current motion and intentions when navigating in human-centric environments

•  Planning: Design humanoid motion planning algorithms for avoiding hazards and feedback strategies for responding to dynamic changes in the environment

•  Safety: Deve lop pr inc ip led approach, methodology, and algorithms for all research thrusts that predict and avoid hazardous configuration, detect potential failures, and promptly react to guarantee human safety

Related Work •  Cooperative object lifting (HRP2 [2], Nao [3]) •  Cooperative object carrying (PR2 [4], HPR2 [5]) •  Current limitations: − Motion within obstacle-free, flat, open areas − Use regularly shaped objects of known dimensions and grasping points − Employ models of human-humanoid locomotion that do not explicitly consider dynamic coupling

Collaborators •  Boeing, Polaris (manufacturing) •  Vecna Robotics (hospital automation) •  Innovative Design Labs (medical devices) •  Willow Garage (robotics) •  Gillette Children's Specialty (healthcare)

Preliminary experiments with the Hubo robot

Experiments in cooperative manipulation and whole arm grasping with the PR2