31 January 2007 Craig E. Ward 1

Large-Scale Simulation Experimentation and Analysis

Database Programming Using Java

31 January 2007 Craig E. Ward 2

Agenda

Background on problem presented by the simulations.

Architecture of the Scalable Data Grid (SDG) solution.

Java technology choices. Future Directions for SDG Questions

31 January 2007 Craig E. Ward 3

Simulation Environment

Joint Semi-Autonomous Forces (JSAF) environment in Urban Resolve experiments.

Distributed across multiple workstations and two 128-node clusters.

Goal of simulating 1,000,000 entities.

31 January 2007 Craig E. Ward 4

Urban Resolve 2015 Sites

31 January 2007 Craig E. Ward 5

Logging Simulation Data

Data loggers captured simulation data from JSAF environment

Before analysis, data moved to central store.

Experiments generating multiple terabytes of data.

Data difficult to manage and analyze.

31 January 2007 Craig E. Ward 6

Scaleable Data Grid SDG is a system for distributed collection and

analysis of large amounts of data. Initial prototype deployment for JSAF

environment in Urban Resolve 2015 experiment.

Goals Make the analysis of the data more useful. Make the management of data more convenient.

31 January 2007 Craig E. Ward 7

SDG n-tier Architecture Java applications

Services between nodes

Services to client simulations

Web application User Interface

Database engine MySQL

31 January 2007 Craig E. Ward 8

Multidimensional Cubes Use the raw data to create multidimensional

cubes of dimensions of interest. Dimensions “roll up” into categories of greater

abstraction.

Sensors

Targets

Detection Status

Detection Cube

31 January 2007 Craig E. Ward 9

SDG Cube Implementation Concepts Cube Description

What the possible ranges are for indices (coordinates) and measures in a Cube.

Cube View A selection of a subset of the possible indices and

measures of interest in a query.

Cube A instantiation of a Cube that matches a Cube View. Comprised of Elements with indices and measures.

31 January 2007 Craig E. Ward 10

Distributed Communication Communication between processes and

nodes with Remote Method Invocation (RMI).

Cube Manager classes register handles with RMI servers

Managers return handles to Cubes Descriptions and Views are serialized and

passed to clients.

31 January 2007 Craig E. Ward 11

Distributed Communications

31 January 2007 Craig E. Ward 12

JDBC

Cubes do not exist in their entirety outside of the database tables.

Tables store simple elements of the Cubes. Measures at the coordinates in relational

table. Elements are very simple Java objects.

31 January 2007 Craig E. Ward 13

Cube Generator Processing

31 January 2007 Craig E. Ward 14

Threaded Processing

Interface to simulations using sockets and pooled Thread objects.

Internal mechanisms use Java 5 Executor classes and thread-safe queues from the concurrent processing package. Generator Query Aggregation

31 January 2007 Craig E. Ward 15

SDG Web Application HTML

Forms for query selection. Display of results of queries.

Java Server Pages Create HTML pages.

JavaScript Manipulate DOM to display form.

Java servlets Interface between backend and JSP pages.

31 January 2007 Craig E. Ward 16

Java Technologies used JDBC

Interaction with the database.

RMI Communication between nodes.

Threads “Traditional” and new Java 5 concurrent classes.

Sockets Interface to simulations.

31 January 2007 Craig E. Ward 17

Technologies Not Used Java Data Objects

Complicates build process. Hibernate

Another product to configure and distribute. Java EE

Too complex. Clustered JDBC (Sequoia)

Requires that tables be uniform.

31 January 2007 Craig E. Ward 18

Future SDG Development (?) Enhance web application with JSF, Ajax. Caching of cube view results. Develop ad hoc query capability. Implement an algebra for manipulating

cubes. Dynamic update of cube descriptions. Dynamic reconfiguration of

communication links.

31 January 2007 Craig E. Ward 19

Questions & Comments