Laboratory for Manufacturing Systems and Automation - LMSgenerates QR AR informs station controller...
Transcript of Laboratory for Manufacturing Systems and Automation - LMSgenerates QR AR informs station controller...
Laboratory for Manufacturing Systems and Automation - LMS7th CIRP Conference on Assembly Technologies and Systems –
CATS 2018
Dr. George Michalos
Laboratory for Manufacturing Systems and AutomationUniversity of Patras
Application of Wearable Devices for Supporting Operators in Human-
Robot Cooperative Assembly Tasks
❑ Introduction
❑ Approach
❑ Wearable Device for Supporting Operators Collaboration with Robot
❑ Application Implementation
❑ System Integration
❑ Case Study
▪ Automotive case study
▪ White goods case study
❑ Conclusions
❑ Future Work
❑ Acknowledgments
Contents
Introduction: Challenges
Production environments volatility is not only external (demand):- Products have become modular and are updated on-production- Processes become hybrid depending on who is executing them- Behaviour of resources operating in them
- Humans –unique skills and characteristics- Resources – becoming intelligent and reconfigurable (e.g. robotics)
Enabling Humans for Changeability :- Past: Expert teams acting as assistants – intervening on site- Recent Past: Cognitive support by providing access to information - Lately: Customized information
- at the right place, - on the right time - in the right form
Wearable Technologies: Evolution
New devices with higher resolution and faster response in a compact size.
Augmented Reality DevicesSmartwatches
Wearables are of course not limited to these: smart gloves, mini projectors, exoskeletons and
many other are becoming available.
• Wireless connectivity (Bluetooth, Wifi, 3G/4G)
• Customizable Applications
• Pairing with phones/ PCs
• Input: Touch, voice etc.
• Processing power
• Sensors (Acceleration, gyro, etc.)
• Biosensors (heart rate, sweat etc)
• Lightweight
• All day battery
People have become familiar through every day use
IntroductionThe latest trends in EU manufacturing foster the Human-Robot co-existence in a collaborative environment,
sharing workplaces and tasks.
Introduction
- Robots can handle high payload and
repetitive tasks
- Human workers contribute with their
cognition capabilities and dexterity
• The goal is to provide a robust interaction and communication between Human and Robot resources.
• Enabling changeability by providing customized information at the right place, time and form
• Increase the operator’s experience and acceptance
Approach: Overall design of an HR system
Motion of the robot
Support provision on different organizational
levels
Multiple visualization means can be used
Instructions
The robot stops,moves
Parts & models
Restricted Areas
Wearable Device for Supporting Operators Collaboration with Robot (4/)
Approach: A Human – Robot collaborative assembly cell
Human Operator
Product
High Payload Robot
Applications for supporting the operator Supporting Operators Collaboration
with Robot (4/)
Approach: A Human – Robot collaborative assembly cell
AR glasses application Smartwatch application
Pairing process• Presents a unique ID as QR code
• Enables the connection with AR glasses and Execution System
• Running on various hardware devices
Wearable Device for Supporting Operators Collaboration with Robot (1/5)
AR scans the QR
Smartwatchgenerates QR
AR informs station
controller
Station controller connects these
devices
Task complete
• Pressed by the operator when a human task is done
• Station controller can synchronize the tasks among resources
Wearable Device for Supporting Operators Collaboration with Robot (1/5)
Station controller publish current
Human Task
Smartwatchinforms when this task is completed
Station controller coordinates the rest execution
Audio Commands interface
• Enables the Audio commands functionality
Wearable Device for Supporting Operators Collaboration with Robot (2/5)
• Speech recognition and translation in commands
(Android speech recognition API level 3)
• Increase the usability of the application
Manual guidance interface
Wearable Device for Supporting Operators Collaboration with Robot (3/5)
• Application provides two kinds of manual guiding the robot
1. Physical interaction
Operator using smartwatch enables the closed loop force control system and is able
to hand guide the Robot
Manual guidance interface
Wearable Device for Supporting Operators Collaboration with Robot (4/5)
• Application provides two kinds of manual guiding the robot
2. Arrow key buttons
Operator guides the Robot relatively with its TCP by pressing the arrow buttons
AR operator support application interface
Wearable Device for Supporting Operators Collaboration with Robot (5/5)
• Control the output information at AR glasses
1. Show/hide the production info
2. Show/hide 3D models
3. Show/hide robot’s trajectory
On long click checks the connection with the Station controller
Green indicates that it is connected
Red indicates that it is disconnected
Application Implementation (1/2)
Screen Interfaces
The Smartwatch application consist of:
• 10 stage buttons in total
• divided in 3 levels
Stage Button:
• The main button executes the presented functionality
• The label button presents:
- Current Stage title
- The connectivity with Station controller
Stage Levels:
• Each level contains similar functionalities
On single click sends the command to the Station controllerOn swipe left/right navigate through Stage Buttons on the
same level
On swipe up/down navigate through Stages Levels
Navigation between Stages:
Application Implementation (2/2)
Overall layout of all stages
2nd level
1st level
3rd level
System Integration
System Integration
Smartwatchrunning Android
AR glassesapplication
developed with
WIFI
ETHERNET
Station Controllerhandles the messages using
TCP and Web socket
Robot Controllerreceives commands
from station controller for execution over TCP
Force Torque sensorpublish the measured
data through
Assembly of car’s rear wheel axle
Application Examples (1/4)
▪ 1 High payload Robot
▪ 1 Real axle (25 kg)
▪ 2 Rear wheel group (12 kg)
▪ 4 Clips for fixing cables
▪ 8 screws for wheel groups
Hybrid Assembly Steps:
1. Loading Axle (Robot)
2. Loading Right Wheel Group (Robot)
3. Screwing the Right Wheel Group on the Axle (Human)
4. Assembly of the cable group (Human) – Loading the Left
Wheel Group (Robot)
5. Screwing the Left Wheel Group on the Axle (Human)
6. Assembly of the cable group (Human)
Assembly of car’s rear wheel axle (video)
Application Examples (2/4)
Link to
video
Refrigerator assembly stations
Application Examples (3/4)
▪ 2 Small payload Robot
▪ 1 Preassembly of cabinet
▪ 1 Flexible panel (polionda)
▪ 1 Conveyor system
▪ Foam sponges and pieces of tape
Hybrid Assembly Steps:
1. Loading Polionda (Human)
2. Hold polionda and manual guide Robot (Human - Robot)
3. Sealing the front section (Robot)
4. Assembly and fix the back side of polionda (Human) – Sealing the middle section (Robot)
5. Sealing the back section (Robot)
Refrigerator assembly stations (video)
Application Examples (3/4)
Link to
video
Conclusions
• Added value:
- Integrating humans in the manufacturing execution workflow with a dynamically configurable way
- Improve the worker experience compared to existing approaches (e.g. stationary touch screens
or physical buttons)
- Same application can be installed in various android smartwatches
- The interface with ROS framework makes easier the integration with other robotics application.
• Obstacles:
- Time for a user to get familiar with the application
- Navigation among different buttons could be difficult in case of many features.
- Customization time for adopting each use case requirements
Conclusions
Future work
Based on the end user feedback it is planned to:
• Improve the usability of the application :
- Swiping through buttons
- Touching gestures
- Pairing process with AR glasses
• Use additional sensors that smartwatches usually acquire
- Accelerometer , gyroscope
- Vibrator
• Develop programs for quicker and easier customization of the application from
non experts users
Future work
Acknowledgments The work of this paper has been partially funded by EC
research projects:
• ROBO-PARTNER – Seamless Human-Robot Cooperation for
Intelligent, Flexible and Safe Operations in the Assembly Factories of
the Future”
(Grant Agreement: 608855)
www.robo-partner.eu
• THOMAS - Mobile dual arm robotic workers with embedded
cognition for hybrid and dynamically reconfigurable manufacturing
systems”
(Grant Agreement: 723616)
http://www.thomas-project.eu
Acknowledgments
THANK YOU!
Laboratory for Manufacturing Systems and AutomationUniversity of Patras
www.lms.mech.upatras.gr