Test & Evaluation/Science & Technology Program Industry ... · Integrated Planning of Tactical, ......
-
Upload
doannguyet -
Category
Documents
-
view
216 -
download
0
Transcript of Test & Evaluation/Science & Technology Program Industry ... · Integrated Planning of Tactical, ......
Test & Evaluation/Science & Technology Program
C4I & Software Intensive Systems Test (C4T)Test Technology Area (TTA)
Big Data Analytic Technology Needs for T&E
May 11, 2017Mr. Gil TorresPhone: 805-989-9521Email: [email protected] A. Approved for public release: distribution unlimited.
Big Data Data
• Varieties of Big Data data– Structured Data (20%) (Sensor data, Comm, transaction data, man-machine
interface I/O, Schema / RDBMS– Unstructured Data (Imagery, voice, text, chat, email, Docs)– Semi-structured (Self describing -Label/ Value pairs, XML)
• Velocities of Big Data data– Near real Time Situation Awareness– Real time tactical and tactical edge communications– Time Critical interactions– Low latency Distributed Networks
• Volumes of T&E Big Data data– Theater data stream 270 TB of NTM data/year– Future Sensor data Petabytes to Yottabytes
DISTRIBUTION A.
C4TOverview
Innovate T&E: Joint, Early, Often, and Agile
Complex Warfare EnvironmentsBattle increasingly sophisticated adversaries in increasingly complex environments*
Innovative approaches to how we fight, posture our force, and leverage our asymmetric strengths and technological advantages*
Distributed Testing (DT)
• Remove Test Infrastructure Biases• Cross Domain Solutions (CDS) and Multi-Level
Security (MLS)• Assess Big Data Warfighter Systems• Testing Warfighter Systems Employing Agile
Communications• Emulate Contest/Dense Communications
Environments
Modeling & Simulation (M&S)
Test Automation (TA)
• Advance Big Data Collection, Analysis & Visualization
• Improve Test Execution thru Automation/Cloud Computing
• Advance Testing for Next Generation of Handhelds
• Improve Automated Control of Targets
* 2014 Quadrennial Defense Review
• Required Fidelity in Live and Simulated Environments
• V&V Techniques• Aggregation Techniques• Run-time Performance for RT
Applications• Systems, Communications,
Environmental Representations
C4I & Software Intensive Systems Test (C4T)Test Technology Area (TTA)
BIG DATA & CLOUD COMPUTING
Summary of C4T TTA Projects
C4TFAST GRaB (AE)
TNTI
Concept ExplorationTechnology
Characterization & Transition
Engineering, Integration, &
Experimentation
Project Initiation
Technology Demonstration
Technology Assessment
Technology Transition
Concept Formulation(BAA Process)
Transitioned(Tracked)
IPT3N
Development Timeline
Test Automation
Distributed TestingModeling & Simulation LVCT
RAID F-35
WAF BB
RAID PRISM
Domain Project Title Project Acronym
Performer
Test Automation Real-time Automated Insight Engine for Data to Decision RAID F-35 Jet Propulsion Laboratory
Test Automation Real-time Automated Insight Engine for Data to Decision Patuxent River Infrared Signature Measurements
RAID PRISM Jet Propulsion Laboratory
Test Automation Integrated Planning of Tactical, Test Support, & Tactical Engagement Networks
IPT3N Scalable Network Technologies
Modeling & Simulation FAST Gaussian Ray Bundle FAST GRaB NUWC Newport
Modeling & Simulation Weapons Analysis Facility: Broadband Capability WAF BB NUWC
Distributed Testing Technologies for Net-centric Test Interoperability TNTI Leidos Corporation
Distributed Testing Technologies for Interoperability of Live Platforms with Virtual & Constructive Simulation for Distributed Test
LVCT Lockheed Martin Rotary and Mission Systems
FAST GRaB
Simulated Test Environment of Battlefield Robustness
Sensor Platform/Aircraft Movement SimulationUpdate Rate (200 msec)
Terrain Generator100 Meters Per Grid Tile Rocket Sensor Guidance Software
System Under Test (SUT)
Rocket Fly Out Simulation
Update Rate (100 msec)Non-Continuous
Ground Target/VehicleMovement Simulation
Update Rate (1 sec)Non-Continuous
Sensor Update Rate (50 msec); Guidance Update Rate (100 msec); Accuracy ≈ 6m (CEP)
Mature technologies that increase the scope of modeling & simulation across the Department of Defense (DoD) distributed T&E infrastructure.
Modeling & Simulation Technology Use CaseM&S for T&E of Systems in Complex Environments (Representations and V&V)
Use of M&S in T&E will increase Big Datasets
Automated Testing in a Cloud Environment Executing 1000’s of Test Case Scenarios in Parallel
Test Automation Technology Use CaseSoftware Intensive Systems Testing in a Cloud Environment
New virtualization and cloud-based test technologies are needed to automatically create, execute, and control a large number of test cases operating in parallel creating Big datasets.
Software Intensive Systems (SIS) Typically Operate Within a System-of-System (SoS) to Assess Effectiveness
Automated Testing in a Cloud Environment Executing 1000’s of Test Case Scenarios in Parallel
Automated Tools For Testing System
Effectiveness &
PerformanceVirtualized HW
Aspects of System(s) Under Test
Test Harness Technologies • Support automated/agile testing (individual system)• Minimize labor-intensive test procedures • Support of regression, developmental, operational,
and interoperability testing
Virtual Test Harness Virtual Test Harness
Virt
ual T
est H
arne
ssVirtual Test Harness
System Under Test
Software Intensive Systems (SIS) Typically Operate Within a System-of-System (SoS) to Assess Effectiveness
Automated Tools For Testing System
Effectiveness &
PerformanceVirtualized HW
Aspects of System(s) Under Test
Test Harness Technologies • Support automated/agile testing (individual system)• Minimize labor-intensive test procedures • Support of regression, developmental, operational,
and interoperability testing
Virtual Test Harness Virtual Test Harness
Virt
ual T
est H
arne
ssVirtual Test Harness
System Under Test
Automated Testing Creates Big Datasets with Big Data Analytic Needs
Test Automation Technology Use CaseNextGen Handhelds and Widgets
“On-Demand” Test Services for Speed to Fleet for Mobile Applications and Technologies
Big Data Analytics to Improve Speed to Fleet for Mobile Applications & Technologies
DISTRIBUTION A.
Test Automation Domain Technology Use Case T&E Big Data Rapid Analysis
TACTICALTSPI
STRUCTUREDDATA
C2 Operations/Analysts
Improve the test communities ability to analyze and make decisions from large volumes of structured and unstructured test data.
SEARCH:Parallel Search Engine
Technology
TARGET LOCATION
Test Conductor
Advanced Semantic Algorithms
UNSTRUCTUREDDATA
Use Smart Algorithms &Database Design Technologies
to Structure Data
VIDEO CHAT
TEXT VOICE
SCREENS
DISTRIBUTION A.
T&E Big Data Rapid Analysis Technology Needs (1)
• Software algorithms that rapidly process large video datasets captured in multiple file format with multiple focal lengths (e.g. metric zoom lens) and scale factors to correlate into precise angular measurements.
• Algorithms that rapidly process video captured with multiple focal lengths and scale factors into precise angular (e.g. TSPI) measurements;
• Pattern Recognition for T&E Data;• Advanced Multi-Variate Time Series Analytic Techniques;• Integrating Simulation results with collected data to better understand system-
under-test performance;• Advanced Big Data Visualization Techniques Tailored to the T&E Domain; Low
Latency Federated Query on Disparate Big Data Sets;• Artificial Intelligence Techniques applied to T&E data analysis;• Rapid Data Ingest into the big data analysis system;
DISTRIBUTION A.
T&E Big Data Rapid Analysis Technology Needs (2)
• Sample big data representative repositories to evaluate C4T warfighter platforms with reusable and customizable data capture, T&E algorithms, data warehouse and analytics for Big Data warfighting platform analytics’;
• DoD T&E data types fusion, federation & integration technologies;• Near real time analytics for streaming DoD data types; • Hybrid cloud interfaces for efficient Community of Interest (COI) shared Big
Data infrastructure and tools
DISTRIBUTION A.
Distributed Testing Technology Use CaseAssess Big Data Warfighter Systems
Improve ability to create and assess big data warfighter systems under emulated wartime modes of operation
Joint Mission Environment
Distributed Test Infrastructure
Test Control
Distributed Common Ground System (DCGS) -C4I cross Service System employing Data-to-Decisions• Data generators and
simulators for Velocity, Variety, and Volume
• Platform analytics & fusion assessments
• Big Data environment replication
• Effectiveness and Performance of Individual system within SoS
• Automate regression testing
Test Control & Analysis
T&E Instrumentation
Live Warfighter Systems
Virtual and Constructive Systems/Environments
DISTRIBUTION A.
Assess Big Data Warfighter Systems Technology Needs (1)
• Data generators and simulations to stimulate platform inputs with representative velocity, volume, and data type variety requirements including error inputs, and verification and validation of generated datasets.
• Big Data Analytics to evaluate Warfighter Systems; various data types and sources storage, federation, transformations and integration; Platform analytics and responses to inputs; Evaluating platforms Big Data use and share, Effectiveness/performance evaluations
• Analysis of Big Data System fusion algorithms• Analytics to evaluate the correctness of the Big Data platform analytics• Big Data T&E infrastructure monitoring, visualization and control for Big Data
Platform distributed event analytics for real time analysis of streaming input data types including ones that can be located near the source or before storage for batch processing
DISTRIBUTION A.
Assess Big Data Warfighter Systems Technology Needs (2)
• Analytics for Big data birth to deployment artifacts evaluation• Analytics for Big Data platform analysis T&E data history usage• Analytics for Big Data platform deployment monitoring• Automate regression testing for Big Data Platform changes to its big data
capture or analytics• Big Data IT infrastructures: Cloud and noSQL data bases• Analytics to drill down to individual system performance and contribution, and
then determine the impact to the effectiveness of the SoS performance
DISTRIBUTION A.
15
C4T Test Automation ProjectReal-time Automated Insight Engine for Data to Decision (RAID) F-35
Jet Propulsion Laboratory / Pasadena, CA Description: Big data analytical technology developed to automatically analyze, extract, & manage actionable knowledge from terabytes of data per test day.Enables: Automated reuse of knowledge to enable DT quality testing throughout the lifecycle of weapon systems & provide the potential for discovery of “unknown-unknowns” problems.Current Status: Developing RAID knowledge management framework & proof-of-concept insight extraction demo using F-16 data. Continue to try to get F-35 engine data.Transition Partner / Date: Edwards 771st Flight Test Squadron, TRMC Knowledge Management, Edwards 412th Range Squadron / 3QFY18
FY17 Accomplishments: Improved RAID human-like Artificial Intelligence (AI) software to
reuse tester knowledge for data analytics in realistic environment with high degrees of uncertainty.
Integrate reasoner with teleo-reactive prog. Enhanced Associative Symbolic Memory (ASSUME) software to
support probabilistic inferencing. Demonstrated using RAID to accomplish target identification with
Edwards AFB DIADS data. Deliverables (Mo/Yr):Prototype RAID-DIADS software to Edwards AFB. RAID Phase 2 Final Report.
Key Future Events: Enable correlation of test data with knowledge base models and
ontologies. Initial demonstration of RAID capability addressing a
representative F-35 T&E scenario.
DISTRIBUTION A.
16
C4T Test Automation Domain ProjectReal-time Automated Insight Engine for Data to Decision (RAID)
Patuxent River Infrared Signature Measurements (PRISM)
Jet Propulsion Laboratory / Pasadena, CA Description: Develops big data technologies to analyze infrared (IR) videos that utilizes Intelligent Neural Network (NN) based segmentation and 2D/3D correlation used for system identification, tracking and performance assessment.Enables: Real-time automatic segmentation of multiple band IR videos allow rapid data analysis from man-months to hours. Real-time feedback of IR signature quality during test runs. Current Status: Developing neural network based segmentation algorithm & automatic multi-band signature segmentation software. Transition Partner / Date: PAX ATR / 3QFY18
FY17 Accomplishments: Completed segmentation algorithm refinement. Completed applying neural net directly to MW. Completed Initial C++ GUI test program. Mid-Year progress review.
Deliverables (Mo/Yr): Delivered incremental software builds and user’s manual to PAX
ATR PRISM team.
Key Future Events:Complete target feature segmentation and implementation in
complex neural network.Complete testing and optimization Software. Implement real-time automatic segmentation system.
DISTRIBUTION A.
17
C4T Test Automation DomainDecision Engine for Structured & Unstructured Data (DESU)
Morgan State University / Baltimore, MD
Improve Near Real-Time & Post-Test Analysis for T&E Unstructured Big Data
Description: Developing Big Data technologies for rapid analysis of voluminous structured & unstructured heterogeneous data (audio/video/text) in support of data to decisions for automated event pattern discovery.Enables: Assessment of large volumes of test data, real-time data fusion & analysis capability.Current Status: Developing event classifiers & algorithms to correlate within/across data sets.Transition Partner / Date: PAX ATR 4QFY15
FY15 Accomplishments: Completed 100% of Phase 3 exit requirements. Acquired and tested PAX 3-D Shearlet Algorithm.
Deliverables (Mo/Yr): Phase 2 Report & Software.
Key Future Events: None, project successfully completed.
DISTRIBUTION A.
Big Data Analytic Technology Needs for T&EClosing
• Big Data and associated Analytics presents technological challenges for the T&E community game changing innovations are needed NOW
• C4T TTA Broad Area Announcement defines investment topics for Industry and Academia to propose new technology projects– Primary Big Data TopicsT&E Big Data Rapid AnalysisAssess Big Data Warfighter Systems
– Big Data Topics Resulting from New Technology InnovationsM&S for T&E of Systems in Complex EnvironmentsSoftware Intensive Systems Testing in a Cloud EnvironmentT&E for NextGen Handhelds and Widgets
DISTRIBUTION A.
C4TPoints of Contact
• Executing Agent– Mr. Gil Torres– 805-989-9521 – [email protected]
• Deputy Executing Agent– Ms. Gail Holmes– (401) 832-6972 – [email protected]
DISTRIBUTION A.