Multimodal Intelligent Interactive Development Environment

BAE SYSTEMS PROPRIETARY INFORMATIONThe information contained in this document is the property of BAE Systems, and further dissemination is prohibited without the written permission of BAE Systems.

UNCLASSIFIED

UNCLASSIFIED

Multimodal Intelligent Interactive Multimodal Intelligent Interactive Development EnvironmentDevelopment EnvironmentApplication Programming Interface8/12/2009

BAE SYSTEMS PROPRIETARY INFORMATION

UNCLASSIFIED

UNCLASSIFIED

MIIDEWhiteboard

(MIDOS)

PAMS LTAE

ModelReasoner

Get context

Evaluate

Preferences& constraints

Model & Knowledge

Management

Deploy and Simulate

TaskModels

Results & Rationale

Context

Validation results

Simulator

DeviceModels

InferenceRules

System OverviewSystem Overview

Results & rationale

Context Models


UNCLASSIFIED

UNCLASSIFIED

• MIDOS – user interaction core

• MKM – Models and tools• Continuous two way

interaction between MIDOS and MKM

• Domain specific plug-insMIDOS

Domain independent

Domain specific

Domain perspective

Model & Knowledge

Management

Model Compiler

MIIDEDomain meta model

The model

Simulator/Validator

Generated Artifact

User models

Model

MIIDE Plug-In ArchitectureMIIDE Plug-In Architecture

Generated


UNCLASSIFIED

UNCLASSIFIED

ChallengesChallenges

• Incremental interaction between MIDOS and MKM to guide the user• Intelligently narrowing down the future options• Allowing individual user to develop domain dialects for interacting with

MIDOS• Evolving the domain meta model


UNCLASSIFIED

UNCLASSIFIED

Simulator

MIIDEWhiteboard

(MIDOS)

PAMS LTAE

ModelReasoner

SDR Simulator(Testsys)

Ontology.xsd

Deplo

y

getOntology()

deploy()

evaluate()

S-expression

evaluate()

Model & Knowledge

Management

deploy()

Models

Component APIComponent API


UNCLASSIFIED

UNCLASSIFIED

sd getOntology

MIDOS Model & KnowledgeManagement

MIDOS User

StartStart

Intialize()

getOntology()

:ServiceDescription, CapabilityGroup

How may I help you?()

getOntology()getOntology()


UNCLASSIFIED

UNCLASSIFIED

• An instance of a ServiceDescription defines all of the domain objects to be used by MIDOS.

• A set of components defines a system that will provide one or more capabilities.

class Serv ice Diagram

Serv iceDescription

- serviceName: string

Env ironmentSpec

- environmentName: string- value: string

ComponentSpec

- componentName: string

ResourceCost

- amount: string- resource: ResourceSpec

ReliabilityInEnv ironment

- reliability: double- environmentCondition: EnvironmentSpec

CapabilitySpec

- defaultValue: boolean- capabilityName: string- value: string- description: string

Default assumes 100% reliability, if condition not present

ResourceSpec

- resourceName: string

0..*

typesOfComponents

0..*

environmentConstraints

0..*

resourcesUsed

0..*

typesOfCapabilities

0..*

typesOfEnvironments

0..*

typesOfResources

0..*

resourcesSupplied

getOntology() resultsgetOntology() results


UNCLASSIFIED

UNCLASSIFIED

ServiceDescription

serviceName SDR

List<CapabilitySpec>

capabilityName value

Video Recording Standard

Video Recording High-def

Onboard Storage None

Onboard Storage Low capacity

Onboard Storage High capacity

Audio Recon Yes

Audio Recon No

Weapon Delivery Yes

Weapon Delivery No

List<EnvironmentSpec>

environmentName value

Weather Clear

Weather Cloudy

Weather Precipitation

Weather Heavy Precipitation

Terrain Mountain

Terrain Valley

Terrain Plain

Terrain Ocean

List<ResourceSpec>

resourceName

Power

Memory

Weight

typesOfCapabilities

typesOfResources

typesOfEnvironments

Service DescriptionService Description


UNCLASSIFIED

UNCLASSIFIED

ServiceDescription

serviceName SDR

List<ReliabilityInEnvironment>

reliability environmentCondition

50% Precipitation


List<ResourceCost>

resource amount

Weight 100 lbs

Memory 96 MB

List<ComponentSpec>

componentName Predator

componentName CameraList<ReliabilityInEnvironment>

Reliability environmentCondition

25% Precipitation

0% Heavy Precipitation

List<ResourceCost>

resource amount

Weight 15 lbs

Power 25 Watts


resourcesSupplied

resourcesUsed

typesOfComponents

Service Description – ComponentsService Description – Components


UNCLASSIFIED

UNCLASSIFIED

Mission Type

Recon KineticWeaponDelivery: YesPrecision Guidance: Yes

VideoAudioAudio Recon: YesOnboard Storage: Yes

Close Air SupportVideo Recon: StandardRetransmission: Yes

Destroy TargetTrack: Yes

High-DefinitionHD Video Recon: YesHD Onboard Storage: Yes

StandardVideo Recon: YesRetransmission: Yes


CapabilityGroup

- name: string

Capability

- name: string- value: string

0..*

capabilitiesInGroup

0..*subGroup

getOntology() results: CapabilityGroupgetOntology() results: CapabilityGroup


UNCLASSIFIED

UNCLASSIFIED

Evaluate will be called each time the user selects or removes a capability (either implicitly or explicitly). This will return to MIDOS 0..n valid configurations with reasons, which will allow MIDOS to tailor the information it presents to the user.

evaluate()evaluate()

sd ev aluate


Model Reasoner

MIDOS User

selectCapability()

evaluate(Request, Context)

evaluate(ServiceDescription,Configurations, Request, Context)

:ReasoningResults

update()


UNCLASSIFIED

UNCLASSIFIED

Each request has a set of required components and capabilities and a set of weighted capability preferences.


Context

- serviceName: string

Resource

- available: string- resourceCost: double- resourceType: ResourceSpec

resourceCost establishes a relative cost across different resources; can be used by reasoners to minimize the cost of resources used to implement a given capabil ity set

:Env ironmentSpec

0..*

resourcesenvironment

0..*

The context can contain values for each resource (Power, Weight, etc.) and values for the environment (Weather: Cloudy, Terrain: Mountain, etc.)


Request

- requestName: string- serviceName: string- description: string

CapabilityPreference

- leftBetterThanRightBy: double

Capability


Capability


Capability


Component

- name: string

1..*

left

0..*

capabilityPreferences

0..*

requiredCapabilities

1..*

right

0..*

requiredComponents

evaluate() input: Request and Contextevaluate() input: Request and Context


UNCLASSIFIED

UNCLASSIFIED

The ReasoningResults returned from the evaluate method call will contain an ordered list of 0..n results and the explanation of why no configuration was selected or a summary of why the n-configurations were selected.

Each result will contain:•validConfiguration: This configuration will contain all of the required capabilities and components and may contain some of the preferred capabilities.•comparisons: 0..n comparisons between the validConfiguration and other configurations that details the difference between the two and why the validConfiguration ranks higher.

evaluate() resultsevaluate() resultsclass Serv ice Diagram

Configuration

- configName: string- serviceName: string

Capability


:ComponentSpec

ReasoningResults

- explanation: string

Result

Rationale

- difference: string- reason: string

1..*

capabili ties components

1..*

1

compareTo

1

validConfiguration

0..*

comparisons

0..*

results


UNCLASSIFIED

UNCLASSIFIED

The user requests to add image filtering and sets the priority of the capabilities.

Storyboard ExampleStoryboard Example


UNCLASSIFIED

UNCLASSIFIED

List<Capability>

name value

Predator Yes

Audio Recon Yes

Video Recon Standard

Non-LOS Retransmission

Yes

Precision Guidance Yes

List<Component>

name

Camera

Request

requestName Request1

serviceName SDR

List<CapabilityPreference>

leftBetterThanRightBy 10

List<Capability>

name value

Image Filtering Yes

List<Capability>

name value

Image Filtering No

capabilityPreferences

left right

requiredCapabilitiesrequiredComponents

Example RequestExample Request


UNCLASSIFIED

UNCLASSIFIED

List<EnvironmentSpec>

name value

Weather Precipitation

Terrain Mountain

List<Resource>

resourceType resourceCost available

Power 2 20 Watts

Other resource constraints will be determined by what is supplied by the selected components in a configuration. In this example, the system might know that an additional 20 Watts is being provided by solar panels mounted on the Predator.

Context

serviceName SDR

resources environment

Example ContextExample Context


UNCLASSIFIED

UNCLASSIFIED

capabilities components reason

Predator: Yes

Audio Recon: Yes

Video Recon: Standard

Non-LOS Retransmission: Yes

Precision Guidance: Yes

Hardware: Predator, GPS Device, Camera, Microphone, High-Power Modem

Software: Flight Computer Interface, Internal Network Stack, Audio Acquisition, Video Acquisition, SNMP Agent

Result 1:

Result 2:

capabilities components reason

Predator: Yes

Audio Recon: Yes

Video Recon: Standard

Non-LOS Retransmission: Yes

Precision Guidance: Yes

Hardware: Predator, GPS Device, Camera, Microphone, High-Power Modem

Software: Flight Computer Interface, Internal Network Stack, Audio Acquisition, Video Acquisition

Uses less resources

Reasoning ResultsReasoning Results


UNCLASSIFIED

UNCLASSIFIED

Deploy can be called multiple times to test different configurations. Each call will present a simulation to the user and run generated unit tests that return if the configuration is valid.

sd deploy


Simulator

MIDOS User

deploy(Configuration)

simulate()

simulation()

:isValidConfiguration

deploy()deploy()


UNCLASSIFIED

UNCLASSIFIED

• Add a new CapabilityGroup• Reason about preferences at CapabilityGroup level• Allow the user to add a new capability and use the PAMS DTA for SDR to

dynamically generate valid configurations• Use the DESERT domain constraints to provide better reasons why

configurations are not valid• Drive dialog from set of potential configurations

Possible ExtensionsPossible Extensions

Multimodal Intelligent Interactive Development Environment

Documents

Transcript of Multimodal Intelligent Interactive Development Environment