Intelligent Systems Division (ICTC TI)

Collaborative and Assistant Systems (CAS)

Research

ROSES Partnership OpportunitiesRich Keller

rkeller@mail.arc.nasa.gov2/17/05

Collaborative and Assistant Systems: Major Themes

• Human-centered systems– Work-systems design, modeling, simulation, and evaluation

– Procedure design, modeling, validation, training, and execution support

– Integrated field tests

– Multi-modal interfaces and human / automation / robot interaction

• Situation awareness and decision support systems

– Integrated simulations supporting what-if analysis

– Contingency planning and assistance

• Complex information and knowledge management – Heterogeneous data storage, search, integration, and analysis

– Semantic data organization, metadata management, and ontologies

– Lessons learned and expertise capture

• Software Composition and Integration – Middleware, components, services, frameworks and portals

Major CAS Projects

• Human-centered systems– Clarissa (ISS DTO) and Advanced Astronaut Assistants

– Regulus Open-source spoken dialog toolkit

– Mobile Agents and Advanced EVA technologies

– Optimal human interface synthesis

• Situation awareness and decision support systems

– Simulation-based Acquisition

– Intelligent Launch and Range

– UAV Collaboration and Coordinated Mission Mgmt

• Complex information and knowledge management – Simstation and Digital Shuttle

– Netmark and PMT (major rollout and EVM integration)

– SemanticOrganizer for Science and Accident Investigation

– Aviation data integration, mining, and trending

• Software Composition and Integration – CIP for MER Extended Mission– MCT Fabric demonstration for MSL

Human-Centered Systems

• Some technology pitfalls:– “Build it and they will come.”– “… but it’s a great technology.”– “Lets just ask them what they want.”

Technology

People & Culture

Physical Environment

• Human-Centered Systems: A systematic methodology for designing systems that optimize the teaming of humans and machines.

• Methodology:

• Work-systems design, modeling, simulation, and evaluation

• Procedure design, modeling, validation, training, and execution support

• User-centered design

• Integrated field tests

• Crew and team organizational design and modeling

• Flight deck and mission operations facilities design and evaluation

HCC & Fatigue Countermeasures: Improved data understanding and

Enhanced situational awareness

CIP: Customizable data navigation, search, and information management

Viz: High fidelity terrain modeling and analysis

MAPGEN: Activity plan development and analysis

MERBoard: Collaborative information analysis and sharing

MER science and uplink team members MER science and uplink team members have estimated thathave estimated that overall science overall science return increased by more than 20%.return increased by more than 20%.

Ames Capabilities for Mars Exploration Rover (MER) Mission Operations

Collaborative Information Portal

The technology developed for the CIP project has application to the broader Information Infrastructures technology (InfoCore) research goals. The long term objective of this research is to provide a seamless information infrastructure where data from large numbers of heterogeneous information systems are easily accessible and presented in a “smart” human-centered user interface. This will enable improved decision making, reduce risk, and maximize leverage of the vast information resource and corporate knowledge developed by NASA and industry.

A Web-based information management system being developed to support the surface operations phase of the Mars Exploration Rover (MER) Mission in 2004. This system will provide Mission Operations and Science Teams with rapid and intuitive access to a broad range of Mission status and planning information to assist in the daily decision making process.

Integrated MER Mission Engineering & Science Data Management

MERBoard/Xboard Collaborative Workspace

Collaborative Content Creationfor Mission Planning

Robot Mule tracks Astronauts & takes photos when commandedRobot in “follow me” mode

Voice annotation is recorded and transmitted to database in habitat & to RST on earth

Astros can work fully in parallel, talking to personal agents

Mobile Agents: Coordinating Human-Robot Interactions

Utah Field Tests 2003 and 2004

• 50 Participants over 17 days

• 3 NASA centers & 2 universities

• Diverse scenarios, rough terrain

• 2 geologists; authentic science

Analog Simulation of Mars Surface Exploration

Cart and instrument

package

Cameras: • environmental pan/zoom Webcam• zooming closeup sensor videocam

• wide-field mat still camera

ScienceOrganizerInformation Repository

Experiment management

tools

Microbial mats

Greenhouse server w/ intelligent agent

Science team

• experiment design• experiment scheduling• automated instrument control• automated monitoring• automated data acquisition

Remote control XYZ positioning table

Greenhouse Collaboratory: A remote control scientific experimentation facility

Greenhouse

Rotor Wing

Mishap Has site:

Found at:

Thrust bearingThrust bearing

System under investigation:

Has subpart:

Operated by:

Has hypothesis:

Improper Bolt

Preload

Has design record:

Supported by:

Design schematic

Rotor assembly

Wind tunnel

Rotor pitch link

John Smith

Semantic Network overlays the SemanticOrganizer Information Repository

SemanticOrganizer Interface

A customizable knowledge management system supporting distributed teams of science and engineering investigators

Digital information repository stores heterogeneous investigation documents, information, and metadata; Repository features:

• semantic cross-linkage for rapid access to interrelated info (e.g., evidence & hypotheses)

• intuitive, domain-specific terminology• automated knowledge acquisition via inference

Serves as an archival record of investigation

Major SemanticOrganizer Applications

• Early Microbial Ecosystems/ NASA Astrobiology Institute

• Astrobiology Center for the Study of Biosignatures

• Tulane/NIH Malaria Study

• Mars Analog Drilling Mission

• Simulated Human-Robot Mars Surface Exploration

• Mars Exploration Rovers

• Columbia Accident Investigation

• CONTOUR Spacecraft Mishap Investigation

• Helios UAV Crash

• Moffett Airshow Investigation

• Canard Rotor Wing Investigation

ScienceOrganizer

InvestigationOrganizer

Right side panel displays metadata for the repository node being inspected

Left side panel uses semantic links to display all information related to the repository node shown on the right

Scientific KnowledgeManagement

Decision Support: Intelligent Launch & Range Operations

Spoken Dialog Assistant Systems

• Problem:Problem:– Astronauts perform numerous tasks where they need to both

obtain information about activities and use their hands.– Use of a keyboard in zero-gravity is cumbersome.

• Solution:Solution:– Ames has been developing spoken dialogue systems since

1999 to assist astronauts and allow them to interact with the computer using natural language.

– Systems are able to track the dialogue to maintain context

• CLARISSA Flight Experiment: Intelligent Procedures and CLARISSA Flight Experiment: Intelligent Procedures and ChecklistsChecklists – Allows astronaut to interact using natural language with an

intelligent procedure/checklist system.– Eliminates the need for an astronaut to read the procedure

while one does the procedure.– Strong support from the astronaut office.– Path-finding flight experiment opening the door to a wide range

of intelligent astronaut assistant technology.