Recognizing Teamwork Activity in Observations of Embodied Agents Linus J. Luotsinen School of...

Recognizing Teamwork Activity in Observations of Embodied Agents

Linus J. LuotsinenSchool of Electrical Engineering and Computer Science

University of Central Florida

B. Sc. University of Dalarna, Sweden, 2002M. Sc. University of Central Florida, 2004

Ph.D. Dissertation DefenseNovember 06, 2007

Committee Members

School of Electrical Engineering and Computer Science– Dr. Ladislau Bölöni

– Dr. Avelino Gonzalez

– Dr. Kenneth Stanley

Department of Statistics and Actuarial Science – Dr. Liqiang Ni

Outline

Introduction

Data Acquisition and Knowledge Engineering

Teamwork Activity Recognition using Hidden Markov Models

Team-Oriented Feature Extraction

Role-Based Teamwork Activity Recognition

Tuning the Performance of Teamwork Activity Recognition

Conclusions

Covered in candidacy

Introduction

Recognize teamwork activity in a stream of positional agent traces, and annotate them with the recognized actions

Video demonstration

Team merge (t=0)Convoy (t=100) Goal

(t=200)

Applications

Surveillance– Recognize multi-agent activity in surveillance video feeds

Training– Identify discrepancies and deviations from the actions

performed by an expert team

Smarter agents– Model the opponent team to imitate or countermeasure its

actions

Automated annotation– Automatically index large databases for fast content retrieval

After Action Review Digital Video Recorder

Challenges

Observation noise– Position traces can be distorted by inaccuracies in sensors and

localization algorithms

Alignment problems– Movement performed at different location and orientation

Scaling problems– Movement performed at different physical scale

Temporal scaling– Movement happens slower or faster in time

Terrain distortion– Movement distorted because of adaptation to the terrain

Challenges

Movement variants– Movement in alternative ways that map to the same label

Uncertainty regarding the role of the agents in the team

Role count variants– Movement with different number of agents in the team

Agents changing their roles during the team action

Outline

Introduction

Data Acquisition and Knowledge Engineering– Software tools we developed– Datasets we acquired, segmented and labeled





Conclusions

Software Tools we Developed

Teamwork Scenario Editor (TSE)– Interactive knowledge engineering tool

– Video editor interface

– Visualize large datasets and geographical areas

All knowledge engineering work in this thesis was performed using the TSE

Acquire the datasets

Find teamwork activities

Label the activities

Datasets we Acquired, Segmented and Labeled

Real-world warfare exercise Recorded over a three days Data collected from hundreds of soldiers and tanks

equipped with GPS devices, laser range finders and laser range detectors

Datasets we Acquired, Segmented and Labeled

Mini-MOUTOTBSAF

Military Operations in Urban Terrain (MOUT)

Military Movement

We extract military movement patterns from the warfare databases

In the military domain the movement techniques and formations are selected based on the situational awareness of the team

Traveling– Enemy contact not expected– Fast movement speed

Traveling overwatch– Enemy contact possible– Medium movement speed– Characterized by continuous movement of lead unit and alternating advancement

of rear units

Bounding overwatch– Enemy contact expected– Slow movement speed– Alternating or successive bounds

Military Movement

Formations are used in combination with movement techniques

Formation is selected based on visibility needs, firepower focus and so on

Column– Leader followed by rear units– Fire power in all directions (flanks, front and rear)

Line– Fire power in the front

Wedge– Fire power in the front and in the flanks

Echelon– Enemy contact expected in the front or in the echeloned flanks– Used when one flanks are secured by obstacles

Outline

Introduction






Conclusions


A spatio-temporal pattern recognition problem– Recognize teamwork from X and Y coordinates over time for

multiple agents

Why Hidden Markov Models?– Mathematically sound

– Temporal by nature

– Successfully applied in the past (e.g. speech recognition)

Ways of encoding teamwork activity in HMMs– Knowledge engineering

– Learn from a set of representative examples

Workflow Overview

ImportExternal database

Teamwork Scenario Editor

Export

Visualization Identification

Teamwork Activity Recognition

Learning

Teamwork Activity Models

Teamwork Database

Representative Examples

Classification

Real-time data stream

Annotated Teamwork Behavior

The Hidden Markov Model

The HMM consists of a number hidden states

Transition probabilities–

Emission probabilities–

– Gaussian PDF

Initial probabilities–

HMM with 3 hidden states

The Hidden Markov Model

Learning algorithms– Baum-Welch

Optimize by maximizing

– Segmental K-means Optimize by maximizing , where is the optimum hidden

sequence given by the Viterbi decoding algorithm

Classification– Determine the probability that the input sequence was generated

by the HMM

– Forward evaluation algorithm

Baseline Input Format for the HMM + Basic Preprocessing The input is a vector of the agent positions

– VT = {v1, v2, v3,…, vt}– vt = {x1, y1, x2, y2, x3, y3, …, xn, yn}

It is very unlikely that the team action will be repeated in the same location and position!

We perform a pre-processing of the input data which allows us to recognize team actions happening at arbitrary locations and orientation:

– Translation Align with team centroid

– Rotation Align with x-axis

This is not sufficient, many other distorting factors can happen: scaling, terrain distortion, different ordering of the agents – we are dealing with these later in this presentation

Experimental Setup

Test problem– Real-world military warfare exercise

Train and test data– Six activities (extracted using TSE)

– Artificial activities added for testing

Results: Recognition Accuracy

Classification accuracy is 82% Matches the performance of the knowledge engineering

approach

As presented at the AAMAS-07 conference

Results: Real-Time Analysis

HMM with 4 states and 6 classes ~9.4ms

Hidden States Mean Time (ms)

1 2.543

2 4.743

3 7.02

4 9.394

5 12.06

Shortcomings: A Lot of Assumptions!

We assumed that the teamwork activities were performed at the same scale

We assumed that there is no interaction with the environment

Observation input is of fixed arrangement, hence, we assumed that recognition is performed on the same team the HMM was trained for

We assumed that all activities can be modeled using the same number of hidden states in the HMMs

Dimensionality and state space for larger teams will quickly grow out of control

We do not recognize the roles of the agents in the teamwork activity– Who is the leader and who is the follower?

The recognizer is not practical!

Outline

Introduction






Conclusions


Improve recognition accuracy and robustness by replacing the “vector of positions” input with a collection of team-oriented features

– The features are extracted through pre-processing– Some of the features replace the existing input vector– Some of the features are the input of the role recognizer (and there is some

overlap)– It requires a more complex recognition workflow (which will be shown later,

together with the role recognition module)

Extract semantically rich features from the agent position traces

Discretization process– Intuitive descriptions of teamwork activities– For classifiers using discrete input

Three feature function classes– Agent-oriented features– Environment-oriented features– Team-oriented features

Calculated over a sliding window in time

Features are matched with the ways humans would understand the scene

Agent-Oriented Feature Functions

Focus on individual agents

Enhance performance of teamwork recognition

Used to recognize (likely) roles

Curvature

The rate at which a curve changes direction

Finite difference approximation with central difference

Environment-Oriented Feature Functions

Agent and team interactions with environment

Environmental objects– Physical or virtual

– Static or dynamic

Domain specific examples– Frontline in war

– Offside line in soccer

– Line-of-scrimmage in football

A1

Obsta

cle

Orientation

Path

t=0

t=1

t=2

Angle

LOS

Team-Oriented Feature Functions

Extract features relative to the team

Specifically designed for teamwork activity recognition

CRPV Discretization

Centroid-Relative Position Vector (CRPV)

Positions are calculated relative the centroid position and orientation

Dimensionality is reduced (compared to previous approach)

Translation, rotation and scale invariant

An evolution is the Role-Relative Position Vector (RRPV)– Privileged agent

Cohesion

Measures the bonding together of the team– Derived from the Principal Component Analysis– PCA: Dimensionality is reduced by restricting attention to the

directions along the scatter cloud that are the greatest

Calculate eigenvalues and eigenvectors from the position-based scatter matrix

– In the 2D case there are two eigenvalues and two eigenvectors

Cohesion is the maximum eigenvalue

CohesionDirection is the direction of the eigenvector with maximum eigenvalue

CohesionGradient is the change in cohesion over a sliding window

Cohesion

Position-based scatter matrix

Team-Oriented Feature Functions

Agent-oriented features can be used by replacing the team with a virtual agent following the team centroid

Outline

Introduction






Conclusions


The goal is to improve robustness of the teamwork recognizer

What are the role assignments in the team?

The baseline HMM input we presented previously is a fixed arrangement

– {1, 2, 3, 4} not the same as {2, 3, 4, 1}

Brute force solution – Inefficient: One HMM for each permutation

Role recognition module– Represent teamwork activity based on roles, rather than by agents

Extended Workflow Overview

Import

External database


Export

Visualization Identification

Teamwork Activity Recognition Framework

Learning

Teamwork Activity Models

Teamwork Data Format

Representative Examples

Classification

Real-time data stream

Annotated Teamwork

Activity

Role Models

Recognition

Mapping

Role Recognition


Role-Oriented Feature Extraction

Assignment

Learning Role Models

Role models are represented by decision trees– Trained from observations with ID3 algorithm

– Pruned to minimize effects of overfitting

– Intuitive (white-box)

– Visualizes the exact features which were used in classification Follow leader uses CRPV

Feature functions for role recognition:– Agent-oriented feature functions

– CRPV feature (assuming that bystander agents are filtered out)

Role Recognition

Calculate the role assignment probability Pr( ai , rj )

– The probability that agent ai plays role rj

Extract observation sequences from the movement trace of each agent

Input each observation in the sequence to the decision tree classifier

The output is a sequence of class frequency vectors Role assignment probability: CRPV

f={2, 4, 0, 0}

NorthWest

f={4, 6, 0, 0}

NorthEast

f={0, 0, 17, 0}

SouthEast

South

f={0, 0, 277, 319}

f={0, 0, 25, 0}

SouthWest

Velocity

North

f={66, 113, 0, 0}

Low

Acceleration

High

f={107, 80, 0, 0}

Decelerating

f={34, 7, 0, 0}

Constant

f={106, 109, 0, 0}

Accelerating

Role Assignment and Mapping

Identify the best match of role to agent assignments by searching the role assignment probabilities– Re-map the team-oriented feature vectors

Multiple role assignment– Each agent can play multiple roles

Role Assignment and Mapping

Unique role assignment– Each agent can play one role

Experimental Setup

Warfare exercise dataset– Same as used previously– Extended with seven activities– Four agents and four roles

Teamwork Activity Recognizer– Gaussian Mixture Model (GMM)– Multiple internal HMMs for each activity to accommodate for complexity

variations in teamwork activities

GPS readings are not always available (e.g. positions from opponents)

Simulate noisy observations– Offset position with a randomly generated number following the Gaussian

distribution multiplied with a noise magnitude

Mean accuracy and standard deviation was calculated using stratified 10-fold cross-validation

Activity Sequences Observations

Traveling column 29 319

Traveling line 30 330

Traveling box 33 363

Bounding overwatch 9 189

Wedge 17 187

Team split 15 192

Team merge 15 177

Results: Creating the Idealized Team Actions

Assume perfect role recognition

Best parameter configuration– Four hidden states

– Three mixture components

Results: Performance Evaluation with Unknown Team-Organization

Shuffled input

Results: Performance Evaluation with Unknown Team-Organization

Accuracy with noise and unknown team-organization– 92.62% with standard deviation 5.53%

– Previous accuracy was 82% (without noise)

Role recognition improves robustness

Team-oriented feature functions improves accuracy

Outline

Introduction






Conclusions


Many components in the recognizer’s workflow are customizable– HMM: Number of hidden states and choice of emission

probability representation

– Feature extraction: Width of sliding window and discretization thresholds

– Role recognition module: Choice of features, parameters in the ID3 learning algorithm and the choice of learning algorithm

The probability density function (PDF) has a significant impact on the accuracy and robustness of the recognizer

Implementation Choice Multi-modal Continuous Parameters

Simple HMM Yes No States (n), Clusters (k)

HMM with Gaussian No Yes States (n)

HMM with GMM Yes Yes States (n), Components (C)

Simple Hidden Markov Model

Discrete (histogram-based) probability density function

Inputs are sequences of discrete values

Vector quantization– Compress observations– Training of a codebook– Cluster assignment

Advantages– Multi-modal distribution

Disadvantages– Extra cluster parameter to train codebook– Can not estimate unseen observations– Requires more training data

Hidden Markov Model with Gaussian PDF

Used in the initial recognizer

Advantages– Continuous

– Can estimate unseen observations

Disadvantages– Unimodal distribution

– Example: How to model U-turn to the left and right in the same HMM?

– Matrix inversion of covariance matrix can be problematic Reduce expressiveness by enforcing diagonal covariance matrices

Hidden Markov Model with GMM PDF

Used in the extended recognizer

Advantages– Continuous

– Can estimate unseen observations

– Multi-modal distribution

Disadvantages– Extra parameter to determine number of mixture components

– Parameter estimation is computationally expensive

Results: Simple HMM

Best recognition accuracy– 60 clusters with 18 hidden

states– Accuracy is 75.05% ± 14.89%

Difficulties with– Bounding overwatch– Team split– Team merge

Results: HMM with Gaussian PDF

Best recognition accuracy – 3 hidden states– Accuracy is 91.29% ± 6.69%

Fails when using more than three states (not enough training data)

– Bounding overwatch– Team split– Team merge

Reduce expressiveness of the PDF

Results: HMM with GMM PDF

Best recognition accuracy – 3 hidden states– 2 mixture components– Accuracy is 93.90% ± 4.68%

Summary

Simple HMM– Performed poorly

– Filters out too much information

HMM with Gaussian PDF– Fails when using the general covariance matrix when n > 3

HMM with Gaussian Mixture Model PDF– Best choice

– Least sensitive to changes in parameter configurations

Implementation Choice Mean Max Min Std. dev.

Simple HMM 62.91% 75.05% 44.52% 5.44%

HMM with Gaussian 83.27% 91.28% 77.76% 4.29%

HMM with GMM 88.30% 93.90% 83.19% 2.17%

Outline

Introduction






Conclusions

Conclusions

We developed a knowledge engineering tool for visualization, identification and extraction of teamwork datasets

We developed a teamwork activity recognizer capable of encoding teamwork activity models from representative observations

We improved accuracy of the recognizer using team-oriented feature functions

The robustness of the recognizer was enhanced by integrating a role-recognition module in the recognition workflow

We studied the importance of probability density function choices in our recognizer

Contributions


Corpus of labeled teamwork activity

Team-oriented feature extraction

Learning teamwork activity from observation

Role-based teamwork recognition

Performance tuning of the teamwork recognizer

Future Work

Team membership assignment

Larger teams with 100 or more agents

Preliminary studies– Kalman filter

Noise reduction of observation data Fusing information from multiple (uncertain) sources Applied to estimate life-expectancy in a disaster response simulation

– Genetic programming approach to learn agent strategies Applied to a simple foraging game in the single agent domain

Publications Teamwork Recognition of Embodied Agents with hidden Markov models. L. J.

Luotsinen, H. Fernlund and L. Bölöni, IEEE 3rd International Conference on Intelligent Computer Communication and Processing (ICCP07), September 6-8, 2007.

Automatic Annotation of Team Actions in Observations of Embodied Agents. L. J. Luotsinen and H. Fernlund and L. Bölöni, Sixth International Conference on Autonomous Agents and Multiagent Systems (AAMAS-2007), May, 2007, Honolulu, Hawaii.

A Study of the Robustness of Agent Performance in Nine Popular Agent Implementation Paradigms. L. J. Luotsinen, M. A. Khan and L. Bölöni, IEEE 3rd International Conference on Intelligent Computer Communication and Processing (ICCP07), September 6-8, 2007.

A Comparison Study of Twelve Paradigms for Developing Embodied Agents. Ladislau Bölöni, L. J. Luotsinen, Joakim N. Ekblad, T. Ryan Fitz-Gibbon, Charles Houchin, Justin Key, Majid Ali Khan, Jin Lyu, Johann Nguyen, Rex Oleson, Gary Stein, Scott Vander Welde, and Viet Trinh. Software: Practice and Experience, 2007.

Comparing Apples with Oranges: Evaluating Twelve Paradigms of Agency (Book chapter), L. J. Luotsinen, J. N. Ekblad, T. R. F. Gibbon, C. Houchin, J. Key, M. A. Khan, J. Lyu, J. Nguyen, R. Oleson, G. Stein, S. V. Weide, V. Trinh and L. Bölöni, ProMAS-2006, Fourth International Workshop on Programming Multi-Agent Systems, LNAI 4411, pp. 95-114, 2007.

Comparing Apples with Oranges: Evaluating Twelve Paradigms of Agency. L. J. Luotsinen, Joakim N. Ekblad, T. Ryan Fitz Gibbon, Charles Houchin, Justin Key, Majid Ali Khan, Jin Lyu, Johann Nguyen, Rex Oleson, Gary Stein, Scott Vander Weide, Viet Trinh and Ladislau Bölöni, ProMAS-2006, Fourth International Workshop on Programming Multi-Agent Systems, Hakodate, Japan, May, 2006.

A Two-Stage Genetic Programming Approach for Non-Player Characters. L. J. Luotsinen, Joakim N. Ekblad, Annie S. Wu, Avelino J. Gonzalez and Ladislau Bölöni, FuturePlay, The International Academic Conference on the Future of Game Design and Technology, East Lansing, Michigan, October, 2005.

Collaborative UAV Exploration of Hostile Environments. L. J. Luotsinen, Avelino J. Gonzalez and Ladislau Bölöni, 24th Army Science Conference, Orlando, Florida, December, 2004.

Publications (Pending)

Role-Based Teamwork Activity Recognition in Observations of Embodied Agent Actions. L. J. Luotsinen and L. Bölöni, Submitted to: Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-2008), May, 2008, Estoril, Portugal.

A Robust Method for Estimating Noisy Measurements Applied to Disaster Response Operations. L. J. Luotsinen, M. A. Khan and L. Bölöni, 2008.

Building a World Model under Communication Constraints for Disaster Response Applications. M. A. Khan, L. J. Luotsinen and L. Bölöni, 2008.

Questions?

Recognizing Teamwork Activity in Observations of Embodied Agents Linus J. Luotsinen School of...

Documents

Transcript of Recognizing Teamwork Activity in Observations of Embodied Agents Linus J. Luotsinen School of...