New Paradignms for Human-Robot INteraction Using Tangible User Interfaces
Agricultural robot sprayer: Evaluation of user interfaces in field experiments
-
Upload
george-adamides -
Category
Technology
-
view
175 -
download
5
Transcript of Agricultural robot sprayer: Evaluation of user interfaces in field experiments
Agricultural Robot Sprayer and Evaluation of User Interfaces in Field
Experiments
George AdamidesSenior Agricultural Research OfficerAgricultural Research Institute
Presentation overview
Technical characteristics of the Agricultural Robot Sprayer
User Interfaces for HRI Design & implementation
Field Experiment design & implementation
Findings
Conclusion
Summit XL by Robotnik
Technical characteristics of the Agricultural Robot Sprayer
http://www.robotnik.es/en/products/mobile-robots/summit-xl
Summit XL technical characteristics
• The SUMMIT XL is a medium-sized high mobility all-terrain robot with extreme performance.
• The Summit XL has skid-steering kinematics based on 4 high power motor wheels
– Dimensions 693x626x417 mm– Weight 30 Kg– Load capacity 20 Kg– Speed 3 m/s– Traction system 4 wheels– Batteries 8x3.2V LiFePO4 (~5 hours autonomy)
• 45 minutes for full charge– Temperature range 0oC +50oC – Max climbing angle 45o
– Controller ROS embedded PC with Linux Real Time– Communications WiFi 802.11n– Connectivity Internal: USB, RS232, GPIO y RJ45– External: USB and power supply 12 VDC
Technical characteristics of the Agricultural Robot Sprayer
Main parts of Summit XL robot
Building the Robot Sprayer
Technical characteristics of the Agricultural Robot Sprayer
September 2012
The Sprayer• Serena electric sprayer
Weight (with full tank) 13.6 kgMeasurements: 315x145x400Water flow rate with a fan nozzle 26 liters/hBattery life 11.30 hFull cycle charging 10hCapacity 10 liters
Technical characteristics of the Agricultural Robot Sprayer
Sprayer installation• Modbus IO• Sprayer case• ROS module programming
<?xml version="1.0"?> <launch> <!-- start modbus_io node --> <param name="modbus_io_node/ip_address" value="192.168.2.185" /> <param name="digital_outputs" value="8"/> <param name="digital_inputs" value="8"/> <param name="analog_outputs" value="2"/> <param name="analog_inputs" value="2"/> <node pkg="modbus_io" type="modbus_io_node" name="modbus_io_node" output="scr een"/> </launch>
ssh [email protected] summit> ping 192.168.0.185 3. Launch the modbus_io module roslaunch modbus_io test_io.launch 4. Test the digital outputs rosservice call /modbus_io/write_digital_output 0 false rosservice call /modbus_io/write_digital_output 0 true rosservice call /modbus_io/write_digital_output 1 false rosservice call /modbus_io/write_digital_output 1 true ... Outputs 5,6,7 and 8 are RELAYS
5. Once it is working, modify the summit_xl_complete launch file in order to launch the IO module every time you start the robot and roscd summit_xl_complete edit the launch/summit_xl_complete.launch file and add the following line: <!-- start modbus_io node --> <include file="$(find modbus_io)/launch/test_io.launch"/>
Sprayer case and installation by AgroWise
Building the Robot Sprayer – modified version
December 2012
Technical characteristics of the Agricultural Robot Sprayer
Initial trial-out findings
• Issues with the camera – Viewing angle– Drops on dome cover
• Issues with PC screen– Lighting/shading/reflection
• Issues with wireless connection
– Distance
• Issues with Bluetooth connection
– PS3 (distance)
Technical characteristics of the Agricultural Robot Sprayer
Improving the Agrirobot
• Hardware – Installation of two USB cameras to improve
Peripheral vision and End-effector vision– Bigger wifi antenna– Moved higher the spraying nozzle
• Software – Installation and configuration of the
mjpeg_server ROS module and the Apache webserver
– Installation and programming the pr2_keyboard_teleop ROS module for the Summit XL navigation and the sprayer on/off state
Technical characteristics of the Agricultural Robot Sprayer
Peripheral Visual aid
Technical characteristics of the Agricultural Robot Sprayer
End-effector visual aid
Technical characteristics of the Agricultural Robot Sprayer
WiFi Antenna
Building the Robot Sprayer – current version
Technical characteristics of the Agricultural Robot Sprayer
May 2013
Technical issues and troubleshooting
Short-movie
Technical characteristics of the Agricultural Robot Sprayer
Presentation overview
Technical characteristics of the Agricultural Robot Sprayer
User Interfaces for HRI - Design and Implementation
Field Experiments Design & Implementation
Findings
Conclusion
Reality Based Interaction (RBI) styles [1]
New interaction styles that draw strength by building on users’ pre-existing knowledge of the everyday, non-digital world to a much greater extent than before.Examples of RBI: VR, AR, TUI, ubiquitous and pervasive computing, handheld or mobile interaction…
[1] Jacob, Robert JK, et al. "Reality-based interaction: a framework for post-WIMP interfaces." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2008.
User Interfaces for HRI – Design and Implementation
User Interface for Robot Teleoperation – Development Phases
User Interfaces for HRI – Design and Implementation
Spraying
Designing for HRI Awareness
HRI Awareness [2]
Given one human and one robot working on a task together, HRI awareness is the understanding that the human has of the location, activities, status, and surroundings of the robot; and the knowledge that the robot has of the human’s commands necessary to direct its activities and the constraints under which it must operate.
LASSO technique [3]
• Location Awareness• Activity Awareness• Status Awareness• Surroundings
Awareness• Overall Mission
Awareness[3] Jill L. Drury, Holly A. Yanco & Keyes, B 2007, 'LASSOing HRI: analyzing situation awareness in map-centric and video-centric interfaces', Proceedings of the ACM/IEEE international conference on Human-robot interaction.
[2] Scholtz, J.; Young, J.; Drury, J.L.; Yanco, H.A., "Evaluation of human-robot interaction awareness in search and rescue," Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 IEEE International Conference on, vol.3, no., pp.2327,2332 Vol.3, 26 April-1 May 2004
Robot teleoperation through Human-Robot user interfaces
Mental model
User Interfaces for HRI – Design and Implementation
Phase 1. Using ROS command line
~> roslaunch usb_cam low_res.launch
User Interfaces for HRI – Design and Implementation
~> export ROS_MASTER_URI=http://V3:11311~> rostopic list~> rosrun image_view image_view image:=/logitech_usb_webcam/image_raw
Robot PC Remote PCssh summit@V3
• Using ROS command line to display Peripheral and End-Effector cameras
• Web interface of Axis Ethernet camera
ROS environment with three cameras
User Interfaces for HRI – Design and Implementation
• Installed the mjpeg_server module• Installed the Apache web server
The mjpeg_server is a streaming server that subscribes to requested image topics in ROS and publishes those topics as MJPEG streams via HTTP
Preparing for phase 2
User Interfaces for HRI – Design and Implementation
Phase 2. First attempt for Web UI in HTML
User Interfaces for HRI – Design and Implementation
Improved version of the Web UIin PHP
UI Design and Implementation by Istognosis
UI for drivingMain camera & Peripheral camera
UI for spraying (rejected)Main camera & spraying camera
UI for spraying Main camera & ssupport cameras
Phase 4. Using a patriot wireless tracker & digital glasses
User Interface Design and Implementation by Istognosis
Presentation overview
Technical characteristics of the Agricultural Robot Sprayer
User Interfaces for HRI – Design & Implementation
Field Experiments Design & implementation
Findings
Conclusion
Field experiments Design & Implementation
Setting up the stage
Field experiments Design & Implementation
Preparing the robot
Field experiments Design & Implementation
Path and obstacles
Field experiments Design & Implementation
Grape clusters (targets)
Field experiments Design & Implementation
Experiment design & implementation
1 PC Screen + PS3 + Main Camera Only
2 PC Screen + PS3 + Main & Support Cameras
3 PC Screen + Keyboard + Main Camera Only
4 PC Screen + Keyboard +Main & Support Camera
5 AR Glasses + PS3 + Main Camera Only
6 AR Glasses + PS3 + Main & Support Cameras
7 AR Glasses + Keyboard + Main Camera Only
8 AR Glasses + Keyboard + Main & Support Cameras
Field experiments Design & Implementation
Tasks were randomized and were conducted in two day visits. Four tasks were carried out on day 1 and the remaining tasks on day 2 (not consecutive, period between experiments varied 2 to 10 days)
25 participants:AgronomistsAgricultural TechniciansAgricultural Laborers
Experiment procedures• Consent form• Pre-Questionnaire
– Demographics, Immersion Tendency Questionnaire, General Self-Efficacy Scale, Santa Barbara Sense of Direction Scale, CEW Fluency Scale
• Briefing and getting familiar with the UIs• Post-Questionnaire after each run
– SUS, Presence, NASA TLX– Metrics (collisions, path divergence, targets sprayed,
targets missed, percent completed, duration)
• Experiment duration ~3 hours per participant to complete 8 tasks
Field experiments Design & Implementation
The Agrirobot in the field
Photo Album
Field experiments Design & Implementation
Spraying – extended nozzle antenna
The Agrirobot UI findings
Screenshots
FindingsMain camera VS Main Camera and Support Cameras
Identifying obstacles
Identifying obstacles
Peripheral visionIdentifying obstacles
Shading Issues (left wheel)
Spraying
Stopped Spraying
Shading Issues (left wheel)
Shading issuesIdentifying obstacles
Shading Issues (wheels)
Identifying red grape clusters
Identifying green grape clusters
The Agrirobot VS obstacles in the field
Image album
Keep going…
Presentation overview
Technical characteristics of the Agricultural Robot Sprayer
User Interfaces for HRI – Design & Implementation
Field Experiments Design & Implementation
Findings
Conclusion
Preliminary results
NASA Task Load Index per UI
Findings
User Interfaceswith PS3 controller
User Interfacesusing keyboard controller
Preliminary results
Effectiveness: Number of grape clusters sprayed per UI
Findings
User Interfaceswith main andSupport (3) cameras
User Interfaceswith main (1) camera
Preliminary findingsMean number of collisions per UI
UI # Cameras N Minimum Maximum Mean Std. Deviation
UI1 - 1 Camera
25 0 3 ,76 ,779
UI2 - 3 Cameras
25 0 4 ,60 ,913
UI3 - 1 Camera
25 0 7 1,36 1,777
UI4 - 3 Cameras
25 0 2 ,56 ,861
UI5 - 1 Camera
25 0 5 1,28 1,242
UI6 - 3 Cameras
25 0 4 ,84 ,987
UI7 - 1 Camera
25 0 4 ,80 1,118
UI8 - 3 Cameras
25 0 3 ,28 ,678
Presentation overview
Technical characteristics of the Agricultural Robot Sprayer
User Interface Design
Field Experiment design & implementation
Findings
Conclusion
In Summary – Problems faced and overcome
• Transformation of a “off-the-shelf” robot into a robotic sprayer
• Pilot trials revealed issues with WiFi, Bluetooth, camera view points
• A lot of –smaller or bigger- practical, “non-research” problems, turned into valuable experience for the future
In Summary - what we did• Designed and implemented several
user interfaces• Used WIMP and RBI interaction styles• Field experiments
Conclusions• Yes, it is feasible! (Agri Robot tele-operation )• The user interface design does make a
difference• There are many small, practical issues to
resolve– Agricultural task are demanding and take place in a
difficult environment– Many issues to overcome
• PS3 Bluetooth, WiFi, monitor shading/light, web cameras• Robot wheels, sprayer hose• ROS module programming
• Promising findings
Future work• Incorporation of sensor information in the
UI to include ultrasound sensor information, battery-life (under development)
• Robot improvements – Servos for extending sprayer antenna and
USB/Ethernet cameras control/rotation – Sprayer antenna with multiple nozzles
• Learnability issues need further study• Long hours? • Cost-benefit analysis
Spraying short movie
Thank you for your attention!
Discussion / Coffee time