Post on 21-Sep-2020
Presented to:
Presented by:
Autotestcon 2017
Tony Conard
Schaumburg, Illinois
NAVAIR, Automated Test Systems (ATS)
Program Manager
TPS Life Cycle Panel ‘TPS Requirements and Procurement’
TPS Acquisition Challenges
Potential conflicts: some
say “do this” and some
say “do that”
Need to sort through policies and
processes and initiatives to balance
the scale to meet mission needs
2
Challenge: Balancing Policies, Processes, Guidance, with successful program
execution to deliver products and services that meet the warfighters needs
2
3
AIR-0.0
Vice Admiral Paul Grosklags
Commander, Naval Air Systems Command
• Increase Readiness, Affordability and Speed
• Reduce Cost and Increase Speed of Products to Fleet
• Smaller empowered program / project teams
• Significantly reduce “derived requirements” – Across the Board!
• Well understood risk assessments
• Tailoring in vs. out and “right sizing” program structures and requirements
“Top priorities for 2017 have not
changed; we still are focused 100%
on readiness of our existing aircraft
and how to get new capabilities to
the fleet more quickly.”
Delivering Rapid Capability
Challenge: Delivering Rapid Capabilities at value to the Warfighter
Acquisition programs are managed by
application of Systems Engineering (SE) in
accordance with DoDI 5000.02, Enclosure 3 Optimizes operational capability
Total system performance
Minimizes total ownership costs
Shall develop a Systems Engineering Plan
(SEP) to describe the program’s overall
technical approach Tailored to the application of SE
NAVAIRINST 4355.19E defines System
Engineering Technical Reviews (SETR) as an
integral part of process “…SETR process is tailorable in scope such that it
can be “right sized”…
Independent technical “health” assessment of the
design throughout the development cycle
Systems Engineering
Challenge: Use SEP To “tailor/ right-size” SETR to fit complexity of Project
4
• Requirements / Technical Baseline Management
– Define high level requirements (ORD/CDD/CPD/RAD) and develop System
Performance Specification (SPS)
– Requirement Analysis Document performance parameters and the SPS are
derived from the legacy peculiar support equipment (PSE) and unit under test
(UUT) specifications
– Systems Engineering Plan (SEP) developed & tailored to efforts
– SOW will define engineering and logistics requirements, reviews, and audits
• Risk, Opportunity & Issue Management
– Risks and issues are captured/managed during risk assessment meetings held
by the Risk Management Board (RMB) IAW the Risk Management Plan (RMP)
5
4
3
2
4 3 2 1
1
LIK
EL
IHO
OD
CONSEQUENCE
5
4
3
2
1
Requirements & Risk Management
5
Challenge: Efficient Requirements, Risks, Issues, and Opportunity Management
Systems Engineering
6
Tailor, “right size” for program
Cybersecurity & OTPSs
• Cyber process replaces Information Assurance (IA) or DIACAP process
with Risk Management Framework (RMF)
• Incorporate cybersecurity early and robustly in the OTPS acquisition and
development to reduce risk of a cyber threat
• Ensure mission risks and resilience are central to program and
operational decisions
• NAVAIR OTPS Cyber Impacts- What we know for now… All OTPSs are considered Platform IT (PIT)
Per OPNAVINST 5239.1D, an Assess & Authorize (RMF steps 4&5) are
required for OTPS procurements
Using RMF process will ultimately determine if an Authority to Operate (ATO)
is required for the TPS effort
If low complexity design (wires, passive devices, simple Ics)-then Assess &
Authorize only
If Hi complexity, external transmissions, processing of none CASS FoT
station controlled S/W , requires ATO & full RMF process
Pursue use of reciprocity, standard controls and overlays for OTPS
7
RMF Process Overview
8
RMF Process
NGOR Overview
• The NAVAIR Generic OTPS RFP (NGOR) is a crucial component of the NAVAIR
acquisition process that provides a standard, tailorable RFP for the procurement of
Operational Test Program Sets (OTPS)
• NGOR is policy, NAVAIRINST 13630.5A states that the Aviation Support Equipment
PMA260 is responsible for developing and maintaining a generic OTPS RFP package
• NGOR follows the Uniform Contract Format:
SECTIONS B-M
ATTACHMENTS: (1) STATEMENT OF WORK (SOW)
(2) SYSTEM PERFORMANCE SPECIFICATION/MIL-PERF-32070A Addendum
(3) GENERAL ACCEPTANCE TEST PROCEDURE (GATP)
(4) TECHNICAL DATA PACKAGE (TDP) CONTRACT REQUIREMENTS
(5) TECHNICAL MANUAL CONTRACT REQUIREMENTS (TMCR)
(6) PROVISIONING STATEMENT OF WORK (PSOW)
(7) ADDRESSEE LIST
(8) DISTRIBUTION STATEMENTS
(9) DD FORM 254 (CONTRACT SECURITY CLASSIFICATION SPECIFICATION)
(10) UUT List
(11) STORAGE CASE DRAWINGS
CONTRACTS REQUIREMENTS DATA LIST (CDRLs)
• Current baselines: NGOR v17.1; tailoring options provided
• NGOR Link can be found on DoD ATS website: http://www.acq.osd.mil/log/mpp/ats.html
• For assistance, questions & comments contact the NGOR IPT at NGOR@navy.mil
9
NGOR in the Acquisition Lifecycle M
at
ur
it
y
Le
ve
l
Time
SRR-II
PDR
CDR
TRR
SVR-PRR
SRR-I
Performance Baseline CCA PPP CARD PESHE RFP IMP TEMP Spec SEP RMP
RFP
MS-C
FRPDR
Initial Product Baseline Est 2D Drawings Reviewed Test Plan Approved (K) H/W-S/W CDRLs Reviewed IMS Updated SEP Updated Spec Updated
Operational Config. Est. Test Plan Approved (Govt) Test Procedures Reviewed Test Schedule Delivered
ASR
SRB
RAD
Final Product Baseline Est. Test Reports Signed
Contractor Sys Baseline Spec Updated IMS Baselined
Allocated Baseline Est. IMS Updated SEP Updated
AoA
IOC
Desk
Audit Desk
Audit
Requirements/Need Established
APBA
Prog
Designation
Spec
(Draft)
10
NGOR program events are tailored to fit program scope
Cybersecurity & OTPSs
11
Fosters interoperability amongst ATS(modular and open systems)
Promote innovation and international technology transfer
Allows information to be passed across TPS Life Cycle
boundaries (design, other levels of maintenance)
Promotes Industry development of COTS related tools
Can be a cost effective approach due to data reuse across ATS
interfaces
TPS Related Standards • IEEE 1641.1 Standards for Signals and Test Definitions
• IEEE 1671.1 ATML for Test Information
• IEEE 1671.2 ATML for Instrument Description
• IEEE 1671.3 ATML for Unit Under Test
• IEEE 1671.4 ATML for Test Configuration
• IEEE 1671.5 ATML for Test Adapter
• IEEE 1671.6 ATML for Test Station
• IEEE 1445 Digital Interchange Format (DTIF)
• IEEE 1636.1 Test Results
• IEEE 1636.2 Maintenance Action Information
Need to understand barriers that impact application of standards
Standards In TPS Lifecycle
Cybersecurity & OTPSs
12
Benefits of Standards to TPS Lifecycle
ATEs meeting emerging avionics complex technologies
• Testing High Speed busses, LVDS, JTAG interfacing, Ethernet,
Firewire
Sustainment of multiple ATS environments
• Atlas, TestStand/Lab Win/View, Direct test
• ATLAS is standard signal definition language, but has limitations
• Lack of trained programmers that are efficient with ATLAS
• Transitioning older, outdated languages to today’s environments
• Managing obsolescence across many COTS development/runtime
environments
Interoperability between TPS environments and ATE
Style guides for modern TPS languages
TPS sustainment QA and development tools
Cyber in TPS and ATE environments
Lack of consistent use of IEEE Standards in TPS Life Cycle
TPS Life Cycle Challenges & Opportunities
Summary
Challenge: Balancing Policies, Processes, Guidance, with successful program execution to deliver
products and services that meet the warfighters needs
Overarching DoD and Services specific TPS Acquisition policies provide guidance
System Engineering, SETRs and Cyber approaches should be “tailored” via SEP and
program docs to fit the program’s complexity and risks
NAVAIR’s NGOR provides a standard tailorable RFP for procurement of OTPSs − Can also be used for other customers
Purse Continuous Process Improvement (CPI)
Identify Risk and Opportunities via IPT to stakeholders
Balance all factors to provide rapid- “Speed to the Fleet” Products to warfighters within
C/S/P baselines
13