Post on 13-Jan-2016
SpaceLiner 100 Propulsion Task ForceCandidate Technology Evaluation and Assessment
&
Prioritization WorkshopHuntsville, Al.
April 10-11, 2001 Russel Rhodes NASA KSC
Purpose of Review• Provide a review of the “Technology Evaluation Process”
developed by the SPST in support of the SL-100 Functional Requirements and Key Objectives.– Focus is on Criteria Development and weighting used in Workshop– Supportive of Civil, DOD, and Commercial needs for Third Generation
Space Transportation Systems
• Challenge the affected technical community to consider the knowledge and information from this process in developing and justifying the technologies capable of satisfying SL-100 key objectives.– “Example: Two orders of magnitude decrease in costs & 10,000 times
safer than today's Space Transportation Sys.”
The Evaluation Tool Development Process for Technical Benefit
Define Customer
What Doesthe Customer
Want?
Develop“Whats”
Customer
Weight the“Whats”
Determine the“Hows” for
Each “What”
Determine the“Hows” for
Each “What”
Establish Correlation of Each“How” to Every “What”
SynergyGroup
SynergyGroup
Establish RelativeMerit of Each “How”
Result
List of Prioritizedand Weighted
“Hows” WhichProduce the “Whats”
That the Customer Wanted
Evaluate Each Candidateon Each “How”
Evaluate Each Candidateon Each “How”
Prioritized List of Propulsion TechnologiesLikely to Achieve Goals
Prioritized List of Propulsion TechnologiesLikely to Achieve Goals
SynergyGroup
Tech. Candidates
The Attributes of a Space Transportation System
Affordable / Low Life Cycle Cost Min. Cost Impact of Payloads on Launch Sys.
Low Recurring CostLow Cost Sensitivity to Flight GrowthOperation and SupportInitial AcquisitionVehicle/System Replacement
DependableHighly ReliableIntact Vehicle RecoveryMission Success
Operate on CommandRobustnessDesign Certainty
Environmental Compatibility Minimum Impact on Space Environ.
Minimum Effect on AtmosphereMinimum Impact all Sites
Public SupportBenefit GNPSocial Perception
ResponsiveFlexibleCapacityOperable Process Verification
Auto. Sys. Health VerificationAuto. Sys. Corrective ActionEase of Vehicle/SystemIntegrationMaintainableSimpleLaunch on DemandEasily SupportableResiliency
SafetyVehicle SafetyPersonnel SafetyPublic SafetyEquipment and Facility Safety
During the Technology R&D Phase:
During the Program Acquisition Phase:
Affordable / Low Life Cycle Cost Cost to Develop
Benefit Focused Schedule Risk Dual Use Potential
Affordable / Low Life Cycle Cost Cost to Acquire
Schedule Risk Technology Options Investor Incentive
Operating
Programmatics
How do weimprove in allthese phases?
The Attributes Prioritized
But these are still
qualitative “whats” …
we required a more useful
expression of “how”...
3 fold scoring used -importance, plus need to
improve and whereare we now.
0.000 1.000 2.000 3.000 4.000 5.000 6.000 7.000 8.000
Mission Success
Launch on Demand
Capacity
Flexible
Ease of Veh./Sys. Integration
Low Cost Sens. to Flt. Growth*
Min. Impact on Space Enviro.
Min. Cost Impact of Payloads on Launch Sys.
Social Perception
Process Verification
Equipment and Facility Safety
Intact Vehicle Recovery
Vehicle Safety
Public Safety
Personnel Safety
Min. Effect on Atmosphere
Min. Environ. Impact all Sites
Initial Acquisition
Veh./System Replacement
Resiliency
Design Certainty
Highly Reliable
Robustness
Benefit GNP
Maintainable
Auto. Sys. Health Verification
Auto. Sys. Corrective Action
Operate on Command
Simple
Easily Supportable
Operation and Support
Measurable Criteria - “How”
Used an IterativeStructured Process
Process forces theconsideration ofimprovement incustomer wants
Enviro
nmen
tal
Safety
Depen
dable
QFD Matrix for SL 100 Space Transportation
Propulsion Design Criteria Weighting (Operational Phase)
(Qua
lity
Cha
ract
eris
tic)
B
en
efi
t C
rite
ria
#pol
lutiv
e or
tox
ic m
ater
ials
(-)
# ac
res
perm
anen
tly a
ffect
ed (
-)
# of
kee
pout
zon
es (
-)
Am
ount
of e
nerg
y re
leas
e fro
m
unpl
anne
d re
actio
n of
pro
pella
nt (
-)
# of
tox
ic fl
uids
(-)
# of
pro
puls
ion
sub-
syst
ems
with
fa
ult
tole
ranc
e (+
)
# of
con
fined
spa
ces
on v
ehic
les
(-)
Am
ount
of r
espo
nse
time
to in
itiat
e sa
fe a
bort
(-)
% o
f tra
ject
ory
time
avai
labl
e fo
r ab
ort
(+)
# of
act
ive
com
pone
nts
requ
ired
to
func
tion
incl
udin
g fli
ght
oper
atio
ns (
-)
# of
com
pone
nts
with
dem
onst
rate
d hi
gh r
elia
bilit
y (+
)
# of
sys
tem
s re
quiri
ng m
onito
ring
due
to h
azar
ds (
-)
% o
f pro
puls
ion
subs
yste
ms
mon
itore
d to
cha
nge
from
haz
ard
to
safe
(+
)
# of
diff
eren
t flu
ids
in s
yste
m (
-)
(Demanded Quality) List Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14
ATTRIBUTES WEIGHTAffordable / Low Life Cycle Cost Min. Cost Impact of Payload on Launch Sys. 2.21 1 1 1 1 3 9 3 0 0 3 9 3 3 3 Low Recurring Cost Low Cost Sens. to Flt. Growth* 1.47 3 0 0 1 3 0 3 0 0 1 1 1 1 1 Operation and Support 6.9 9 0 3 9 9 3 9 1 1 9 9 9 9 9 Initial Acquisition 2.49 3 3 1 1 9 3 3 0 0 3 3 3 3 3 Vehicle/System Replacement 2.49 0 0 0 0 3 3 1 0 1 3 1 1 1 3Dependable Highly Reliable 3.45 3 0 0 1 3 9 3 1 1 9 9 9 9 3 Intact Vehicle Recovery 2.3 0 0 0 3 1 9 1 9 9 9 3 1 9 3 Mission Success 0.62 1 0 0 3 3 9 1 0 0 9 9 3 3 3 Operate on Command 6.9 3 0 0 1 9 3 3 0 1 9 9 9 3 9 Robustness 3.45 0 0 0 0 1 3 1 1 1 3 3 3 3 1 Design Certainty 3.45 0 0 0 0 0 3 1 0 0 3 9 3 3 3Responsive Flexible 1.11 1 0 1 0 1 0 0 0 0 1 1 1 0 0 Capacity 1.11 1 0 0 0 3 0 3 0 0 3 3 3 0 3 Operable Process Verification 2.3 3 0 1 0 3 3 1 0 0 3 9 1 1 9 Auto. Sys. Health Verification 6.9 0 0 0 1 3 9 3 1 1 9 3 9 9 3 Auto. Sys. Corrective Action 6.9 0 0 0 1 3 9 3 1 1 3 3 9 9 1
SpaceLiner 100 Propulsion Task ForceSPST Task Charter • SPST Approach Complies with National Space Policy:
• Attributes are Anchored to the National Space Policy:
• SPST Flow Diagram Responds to National Space Policy:
• SPST Will Provide Recommended Functional Requirements Necessary to Develop Space Transportation Architectures, Concepts, and Systems that Surface the Technology Needs Required to Realize the Objectives of SpaceLiner 100 “3rd Generation RLV Feasible Concepts”
• SPST will Assist NASA in the Development of a Prioritized Portfolio of Technologies Capable of Reaching the SpaceLiner 100 3rd Generation RLV Objectives
SPST SPACELINER 100 PROPULSION TASK FORCE
”FUNCTIONAL REQUIREMENTS SUB-TEAM"
OBJECTIVE:• Develop SL-100 functional requirements capable of achieving the 3rd generation RLV objectives
• Develop the Technology Evaluation Criteria and Weights for both Technical Benefit and Programmatics for use in the AHP model to be used at a Technology Evaluation Workshop while anchoring on the existing SPST data base where possible. This sub-team accepted Garry Lyles challenge to further develop and take accountability for his algorithm for “Systems Approach to Safety,
Reliability, and Cost” and anchor the Evaluation Workshop Criteria to the algorithm
SpaceLiner 100 Propulsion Task Force
• In addressing this objective, SpaceLiner 100 Propulsion Technology 3rd Generation RLV Evaluation Criteria--The First Step:
---Select the correct team make-up
---Must be balanced with Designers, Operators, Managers, and Technologists
---Required to bring the needed experience and knowledge together for consensus building to provide quality and creditable process
SpaceLiner 100 Propulsion Task ForceFunctional Requirements Sub-Team Membership
• Russel Rhodes, NASA-KSC - Lead • Uwe Hueter, NASA-MSFC • Walt Dankhoff, SAIC • Bryan DeHoff, Aero.Tech.Serv. • Glen Law, Aerospace Corp. • Mark Coleman, CPIA• Robert Bruce, NASA-SSC• Ray Byrd, Boeing-KSC • Clyde Denison, NGC• Bill Pannell, NASA-MSFC
• Dan Levack, Boeing/Rocketdyne • Bill Escher, SAIC • Pat Odom, SAIC • David Christensen, LMCO • Jim Bray, LM-MAF • Tony Harrison, NASA-MSFC• Keith Dayton/John Robinson, Boeing Co • Andy Prince, MSFC• Carey McCleskey, NASA-KSC• Jay Penn, Aerospace Corp.• John Hutt, NASA-MSFC
•CUSTOMER PROVIDING EVALUATION INPUT:
Uwe Hueter, NASA-MSFC
71% Influence (29% Other Factors)
Influence DiagrammingStresses importance of the
Dependability Attribute Objectives
on achieving SpaceLiner Goals
Safety
Cost
Operable
Responsive
Recurring Cost
Spaceliner Goals
Lyles Algorithm Influence Diagramming
Approach
41%
33%
52%
56%44%
(59% Other)
(67% Other)
(48% Other)
(Life Cycle Cost)Non-Recurring Cost
Dependable(Inherent Reliability)
SpaceLiner-100 Propulsion Task Force SpaceLiner 100 Key Attribute
Influence Relationships
Management Visibility of Influence Achieved when using Weighted Design Criteria for Technology Prioritization ( The How’s to achieve the What’s Desired )
R&D Investment Influence on Achievement of SpaceLiner-100 Key ObjectivesOperations and DDT&E Integrated Key Attributes
Influence Relationships
100XCHEAPER
COST,$/LB
LOW DDT&E ACQUISITION
COST
10,000 X
SAFER
OPERABLE
LOW RECURRING
COSTRESPONSIVE
LOW NON-RECURRING
COST
INVESTORS INCENTIVE
LOW LIFE
CYCLE COST
SAFE
DEPENDABLEINHERENT
RELIABILITY
DUAL USE POTENTIAL
LOW COST R&D
BENEFIT FOCUSED
SHORT SCHEDULE
TECHNOLOGY OPTIONS
LOW RISK DDT&E
SHORT SCHEDULE
R&D
DDT&E
OPERS
OPERS
OPERS
COST FOCUS
ATTRIBUTES KEY
LOW RISK R&D
R&D ATTRTIBUTES
DDT&E ATTRIBUTES
OPERATIONS ATTRIBUTES
L I
F E
C
Y C
L E
C
O S
T
NO
N-R
EC
UR
RIN
G IN
VE
ST
ME
NT
GEN3
GOALS
FLEET
PURCHASE
SpaceLiner-100 Propulsion Task ForceSpaceLiner-100 Assessment/Prioritization Process & Criteria
Sub-Team Products for SL-100 3rd Gen. RLV
• Developed Functional Requirements for SL-100 3rd Generation RLV
• Developed the Influence Diagram Algorithm with SPST 3rd Gen. RLV data base focusing on SL-100 Goals
• Established and weighted desired attributes focusing on the importance and need to improve to achieve the SL-100 performance requirements
• Established and weighted measurable design criteria and provided paretos
• Identified and selected the top 26 good discriminating design criteria with weights for use at evaluating top level technologies
SpaceLiner-100 Propulsion Task ForceSpaceLiner-100 Assessment/Prioritization Process & Criteria
Sub-Team Products for SL-100 3rd Gen. RLV "Continued"
• Established programmatic factors and weighted measurable quality characteristics for both the Acquisition and R & D Phases and provided paretos
• Provided functional requirements, design criteria, and programmatic criteria to Technology Candidate Developers
• Provided Technical Benefit (Design Criteria) and Programmatics Criteria (Acquisition and R & D Phases) for use in the AHP evaluation tool
• Developed a definitions reference document to capture terminology used in this work to provide understanding and communications required for process usefulness
BENEFIT (TECHNICAL) ATTRIBUTES WEIGHTING
Note: Weighting Factors are 1 to 5 with 5 being the most important
Attributes
Column A "Customer
Rank" (J.Mankins &
U.Hueter)
Column B Technology
Improvement "Now"
Column C Technology
Improvement "Plan"
Column D "Improvement
Ratio" (C/B)
Column E "Sales
Points"
Column F Score
(A*D*E)
Weighted Score F/Sum(F)
(Customer)
Affordable / Low Life Cycle Cost
Min. Cost Impact of Payloads on Launch Sys. 4 2 4 2.00 1 8.00 2.209
Low Recurring CostLow Cost Sens. to Flt. Growth* 4 3 4 1.33 1 5.33 1.473
Operation and Support 5 1 5 5.00 1 25.00 6.904
Initial Acquisition 3 1 3 3.00 1 9.00 2.486
Vehicle/System Replacement 3 1 3 3.00 1 9.00 2.486Dependable
Highly Reliable 5 2 5 2.50 1 12.50 3.452
Intact Vehicle Recovery 5 3 5 1.67 1 8.33 2.301
Mission Success 3 4 3 0.75 1 2.25 0.621
Operate on Command 5 1 5 5.00 1 25.00 6.904
Robustness 5 2 5 2.50 1 12.50 3.452
Design Certainty 5 2 5 2.50 1 12.50 3.452Responsive
Flexible 2 1 2 2.00 1 4.00 1.105
Capacity 4 4 4 1.00 1 4.00 1.105
OperableProcess Verification 5 3 5 1.67 1 8.33 2.301
Auto. Sys. Health Verification 5 1 5 5.00 1 25.00 6.904
Auto. Sys. Corrective Action 5 1 5 5.00 1 25.00 6.904
Ease of Vehicle/System Integration 2 1 2 2.00 1 4.00 1.105
Maintainable 4 1 4 4.00 1 16.00 4.419
Simple 5 1 5 5.00 1 25.00 6.904
Launch on Demand 2 1 2 2.00 1 4.00 1.105
Easily Supportable 5 1 5 5.00 1 25.00 6.904
Resiliency 3 1 3 3.00 1 9.00 2.486
Safety
Vehicle Safety 5 3 5 1.67 1 8.33 2.301
Personnel Safety 5 3 5 1.67 1 8.33 2.301
Public Safety 5 3 5 1.67 1 8.33 2.301
Equipment and Facility Safety 5 3 5 1.67 1 8.33 2.301Environmental Compatibility
Minimum Impact on Space Environ. 4 2 4 2.00 1 8.00 2.209
Minimum Effect on Atmosphere 3 1 3 3.00 1 9.00 2.486
Minimum Environ. Impact all Sites 3 1 3 3.00 1 9.00 2.486
Public Support
Benefit GNP 4 1 4 4.00 1 16.00 4.419
Social Perception 4 2 4 2.00 1 8.00 2.209TOTALS= 362.08 100.000
Systems Approach to Dependability, Responsiveness, Safety, and Affordability - Supporting SpaceLiner 100 Functional Requirements -
BENEFIT (TECHNICAL) ATTRIBUTES WEIGHTING
Pareto
Auto. Sys. Health Verification 6.904
Auto. Sys. Corrective Action 6.904
Operate on Command 6.904
Simple 6.904
Easily Supportable 6.904
Operation and Support 6.904
Benefit GNP 4.419
Maintainable 4.419
Design Certainty 3.452
Highly Reliable 3.452
Robustness 3.452
Min. Effect on Atmosphere 2.486
Min. Environ. Impact all Sites 2.486
Initial Acquisition 2.486
Veh./System Replacement 2.486
Resiliency 2.486
Process Verification 2.301
Equipment and Facility Safety 2.301
Intact Vehicle Recovery 2.301
Vehicle Safety 2.301
Public Safety 2.301
Personnel Safety 2.301
Min. Impact on Space Enviro. 2.209 Min. Cost Impact of Payloads on Launch Sys. 2.209
Social Perception 2.209
Low Cost Sens. to Flt. Growth* 1.473
Launch on Demand 1.105
Capacity 1.105
Flexible 1.105
Ease of Veh./Sys. Integration 1.105
Mission Success 0.621
BENEFIT (TECHNICAL) ATTRIBUTES WEIGHTING PRIORITIZED
0.000 1.000 2.000 3.000 4.000 5.000 6.000 7.000 8.000
Mission Success
Launch on Demand
Capacity
Flexible
Ease of Veh./Sys. Integration
Low Cost Sens. to Flt. Growth*
Min. Impact on Space Enviro.
Min. Cost Impact of Payloads on Launch Sys.
Social Perception
Process Verification
Equipment and Facility Safety
Intact Vehicle Recovery
Vehicle Safety
Public Safety
Personnel Safety
Min. Effect on Atmosphere
Min. Environ. Impact all Sites
Initial Acquisition
Veh./System Replacement
Resiliency
Design Certainty
Highly Reliable
Robustness
Benefit GNP
Maintainable
Auto. Sys. Health Verification
Auto. Sys. Corrective Action
Operate on Command
Simple
Easily Supportable
Operation and Support
SpaceLiner-100 Propulsion Task Force SL-100 Propulsion Assessment/Prioritization Process & Criteria
En
vir
on
men
tal
Safe
ty
Dep
en
dab
le
Matrix for SL 100 Transportation
Propulsion Design Criteria Weighting
(Operational Phase)
(Qu
ali
ty C
haracte
ris
tic)
Ben
efi
t C
rit
eria
#pollutive o
r toxic
mate
ria
ls (
-)
# a
cres p
erm
anently a
ffecte
d (
-)
# o
f keepout zones (
-)
Am
ount of energy r
ele
ase from
unpla
nned r
eaction o
f propellant
(-)
# o
f to
xic
flu
ids (
-)
# o
f propuls
ion s
ub-syste
ms w
ith fault tole
rance (
+)
# o
f confined s
paces o
n v
ehic
les (
-)
Am
ount of response tim
e to initia
te s
afe
abort (-)
% o
f tr
aje
cto
ry tim
e a
vailable
for a
bort (+)
# o
f active c
om
ponents
required to function inclu
din
g flight
operations (
-)
# o
f com
ponents
with d
em
onstr
ate
d h
igh r
eliability (
+)
# o
f syste
ms r
equirin
g m
onitorin
g d
ue to h
azards (
-)
% o
f propuls
ion s
ubsyste
ms m
onitored to c
hange from
hazard to
safe
(+)
# o
f diffe
rent fluid
s in s
yste
m (
-)
$ / d
ay o
f dela
y (
-) D
ELE
TE
D
Margin
, m
ass fraction (
+)
Margin
, ave. specific
im
puls
e (
+)
Margin
, th
rust le
vel / engin
e c
ham
ber p
ress(+)
# o
f pote
ntial le
akage / c
onnection s
ources (
-)
# o
f critic
ality
1 failure m
odes (
-)
# o
f m
odes o
r c
ycle
s (
-)
# o
f active e
ngin
e s
yste
ms r
equired to function (
-)
# o
f engin
e r
esta
rts
required (
-)
Desig
n V
aria
bility (
-)
# n
ew
uniq
ue a
pproaches (
+)
Facility c
apitalization c
ost (-)
Hardw
are c
ost (-)
# o
f m
anufa
ctu
rin
g, te
st and o
perations facilites (
recurrin
g) (
-)
Mean tim
e b
etw
een m
ajo
r o
verhaul (+)
Hours to r
efu
rbis
h p
ropuls
ion s
yste
m (
-)
Hours for turnaround (
betw
een launches o
r c
om
mit to n
ew
mis
sio
n) (
-)
# o
f in
spection p
oin
ts (
-)
# o
f checkouts
required (
-)
# o
f purges r
equired (
flig
ht and g
round) (
-)
# o
f ele
ment to
ele
ment in
terfa
ces r
equirin
g e
ngin
eerin
g c
ontr
ol
(-)
Technolo
gy r
eadin
ess levels
(+)
# o
f uniq
ue s
tages (
flig
ht and g
round) (
-)
mean tim
e b
etw
een o
verhaul as %
of syste
m a
cq. cost (-)
DE
LE
TE
D
# o
f expendable
s (
fluid
, parts
, softw
are) (
-)
Ave. Is
p o
n r
efe
r. traje
cto
ry (
+)
Ideal delta-V
on r
ef. traje
cto
ry (
-)
lbs. of fluid
req'd
for w
aste
mgm
t. @
end o
f life
( -
)D
ELE
TE
D
lbs. of propellant req'd
for R
CS
function (
- )
DE
LE
TE
D
lbs. of propellant req'd
for D
elta V
( -
)D
ELE
TE
D
# o
f um
bs. req'd
to L
aunch V
ehic
le (
- )
lbs. In
tg.w
et &
dry m
ass o
f propuls
ion s
ys. ( -
)
# o
f in
-space s
upport sys. req'd
for p
ropuls
ion s
ys. ( -
)
Pow
er r
equired a
s %
of to
tal veh. pow
er (
-)
lbs. of airborne s
upport sys. req'd
( -
)
# o
f active o
n-board s
pace s
ys. req'd
for p
ropuls
ion (
- )
Mis
sio
n s
uccess M
argin
(+) (
Diff. to M
easure?)D
ELE
TE
D
Fuel C
ost $/lb. (-)D
ELE
TE
D
On-board P
ropellant S
torage &
Managem
ent D
ifficulty in S
pace
(-)
Resis
tance to S
pace E
nvironm
ent (+)
Impacts
to P
aylo
ad c
om
pat.(E
MI,T
herm
al,&
Exhaust)
(-)
Req'd
propuls
ion s
ys. volu
me (
-)
Inte
gral str
uctu
re w
ith p
ropuls
ion s
ys. (+)
Thrust contr
ol range (
+)
Min
imum
Im
puls
e b
it (
-)
Transporta
tion trip
tim
e (
-)
ISP
Propellant tr
ansfe
r o
peration d
ifficulty (
resupply
) (
-)
# o
f engin
es (
-)
Mass F
raction r
equired (
-)
% o
f paylo
ad m
argin
(+)
# o
f attain
able
destinations (
+)
% o
f propuls
ion s
yste
m a
uto
mate
d (
+)
# o
f syste
ms w
ith B
IT B
ITE
(+)
# o
f active s
yste
ms r
equired to m
ain
tain
a s
afe
vehic
le (
-)
# o
f physic
ally d
ifficult to a
ccess a
reas (
-)
# o
f diffe
rent propuls
ion s
yste
ms (
-)
# o
f hands o
n a
ctivitie
s r
eq'd
(-)
# o
f m
anhours (
c/o
, handle
, assem
ble
etc
) o
n s
yste
m b
etw
een
on a
nd o
ff c
ycle
s (
Low
Cycle
Fatigue) o
r u
se (
Hig
h C
ycle
Fatigue) (
-)
# o
f ground p
ow
er s
yste
ms (
-)
# o
f hours to r
efu
rbis
h launch s
ite b
etw
een e
ach launch (
-)
# o
f alternate
dedic
ate
d e
mergency a
bort sites r
equired (
-)
# o
f active g
round o
r in-space s
yste
ms r
equired for s
ervic
ing (
-)
# o
f aero-contr
ol surfa
ces (
-)
# o
f m
ajo
r s
yste
ms r
equired to ferry o
r r
etu
rn to launch s
ite (
plu
s
logis
tics s
upport)
(
-)
Syste
m m
argin
(+)
# o
f parts
(diffe
rent, b
ackup, com
ple
x) (
-)
# o
f processin
g s
teps to m
anufa
ctu
re (
-)
Am
ount of real tim
e inspection o
r r
epair (
-)
# o
f hazardous p
rocesses (
-)
# o
f to
ols
required (
-)
# o
f cle
anliness r
equirem
ents
(-)
cost of tr
ansporta
tion r
equirem
ents
(-)
Maxim
um
Q M
ach n
o. P
roduct (-) D
ELE
TE
D
# of active systems required to maintain a safe vehicle (-) 603
(Demanded Quality) List Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 39 48 60 64 87 20 21 22 23 24 25 26 27 28 29 30 38 49 51 52 54 55 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 58 59 31 32 33 34 35 36 37 40 50 53 57 61 62 63 65 66 41 42 43 44 45 46 47 56# of different propulsion systems (-) 582
ATTRIBUTES WEIGHT# of systems with BIT BITE (+) 542
Affordable / Low Life Cycle Cost # of components with demonstrated high reliability (+) 541 Min. Cost Impact of Payload on Launch Sys. 2.21 1 1 1 1 3 9 3 0 0 3 9 3 3 3 9 3 1 3 9 9 3 1 1 0 3 0 3 9 1 1 0 1 3 9 9 3 3 9 9 3 9 9 9 9 1 3 3 1 9 9 0 9 9 3 1 9 3 3 0 1 3 3 9 9 1 3 1 1 9 1 1 9 3 0 0 0 1 3 1 # of hands on activities req'd (-) 534
Low Recurring Cost # of active components required to function including flight operations (-) 527
Low Cost Sens. to Flt. Growth* 1.47 3 0 0 1 3 0 3 0 0 1 1 1 1 1 1 1 0 1 3 1 1 0 0 0 3 9 3 3 9 9 3 3 3 1 0 3 3 3 1 3 3 3 9 3 3 3 1 1 9 0 3 3 9 9 1 3 1 1 3 3 1 1 9 9 3 3 3 0 3 3 1 1 1 1 1 1 0 1 3 # of potential leakage / connection sources (-) 527
Operation and Support 6.90 9 0 3 9 9 3 9 1 1 9 9 9 9 9 3 3 3 9 3 3 3 1 3 0 1 0 9 9 9 9 9 9 9 9 9 9 9 9 0 9 1 9 1 3 9 9 1 3 1 9 1 3 9 9 3 9 0 0 9 9 9 9 9 9 9 9 9 3 9 3 3 9 3 0 1 0 1 3 1 # of systems requiring monitoring due to hazards (-) 523
Initial Acquisition 2.49 3 3 1 1 9 3 3 0 0 3 3 3 3 3 3 3 1 3 1 3 3 1 9 0 9 9 9 1 1 3 1 1 3 9 9 9 3 3 3 1 9 3 9 3 3 3 3 3 9 0 1 9 0 3 3 9 1 0 3 3 9 3 9 3 0 9 1 0 3 1 1 3 9 9 3 9 9 3 3 System margin (+) 508
Vehicle/System Replacement 2.49 0 0 0 0 3 3 1 0 1 3 1 1 1 3 1 1 1 1 1 1 1 1 1 0 1 9 3 3 1 3 1 1 1 1 0 3 3 3 1 1 9 3 3 3 3 3 1 1 3 9 1 9 0 1 3 3 1 0 1 1 3 0 9 1 0 1 0 0 3 1 1 1 9 9 3 9 3 3 3 # of toxic fluids (-) 495
Dependable % of propulsion system automated (+) 488
Highly Reliable 3.45 3 0 0 1 3 9 3 1 1 9 9 9 9 3 9 9 9 9 9 3 9 3 9 0 0 0 1 3 0 0 9 9 9 9 9 9 1 3 3 3 0 3 0 0 9 9 9 1 0 1 1 0 3 1 9 3 3 0 9 9 9 3 9 3 1 3 0 0 9 3 3 9 9 3 3 1 0 3 0 # of unique stages (flight and ground) (-) 483
Intact Vehicle Recovery 2.30 0 0 0 3 1 9 1 9 9 9 3 1 9 3 3 3 3 3 9 3 3 9 3 0 0 0 0 0 0 0 0 0 1 1 3 9 3 3 1 0 3 3 3 1 3 3 9 0 1 1 1 1 0 3 1 3 0 1 3 3 3 0 3 0 1 0 0 9 0 1 0 9 1 0 0 0 0 0 0 % of propulsion subsystems monitored to change from hazard to safe (+) 470
Mission Success 0.62 1 0 0 3 3 9 1 0 0 9 9 3 3 3 3 3 3 3 9 3 3 9 9 0 0 0 0 0 1 1 0 0 1 1 3 3 1 1 3 1 0 9 3 1 9 9 9 9 3 3 9 9 9 9 1 3 1 1 3 3 3 0 9 0 1 0 0 0 0 1 0 3 1 0 0 0 0 0 0 # of in-space support sys. req'd for propulsion sys. ( - ) 465
Operate on Command 6.90 3 0 0 1 9 3 3 0 1 9 9 9 3 9 0 0 1 9 3 1 1 0 3 0 0 0 1 3 3 9 3 3 3 1 3 3 3 0 0 0 0 3 3 0 3 9 3 1 0 1 1 9 1 9 3 3 0 0 9 9 9 3 9 9 3 3 1 3 3 1 0 3 3 0 0 0 0 0 0 Design Variability (-) 464
Robustness 3.45 0 0 0 0 1 3 1 1 1 3 3 3 3 1 9 9 9 9 9 1 0 3 9 0 0 0 0 1 0 0 0 1 0 0 1 1 1 3 1 0 0 3 3 9 3 1 3 3 1 3 9 3 1 3 1 9 3 1 1 1 3 0 1 0 3 0 3 0 0 0 0 9 1 0 0 0 0 1 0 # of active on-board space sys. req'd for propulsion ( - ) 454
Design Certainty 3.45 0 0 0 0 0 3 1 0 0 3 9 3 3 3 9 9 9 1 0 9 3 3 9 0 0 0 0 0 0 0 1 1 0 9 9 3 3 9 9 3 1 3 0 0 3 3 3 0 1 0 0 0 3 3 3 9 0 0 3 3 3 0 3 0 0 0 0 1 0 0 3 9 1 3 0 1 0 1 0 On-board Propellant Storage & Management Difficulty in Space (-) 453
Responsive # of purges required (flight and ground) (-) 428
Flexible 1.11 1 0 1 0 1 0 0 0 0 1 1 1 0 0 3 3 3 0 1 0 1 0 1 0 0 0 0 0 0 1 0 0 0 1 1 1 0 3 3 0 9 3 9 9 3 3 3 3 3 3 9 9 3 9 1 3 9 9 0 1 1 0 1 1 3 0 0 1 1 0 1 9 0 0 0 0 0 0 0 # of confined spaces on vehicles (-) 427
Capacity 1.11 1 0 0 0 3 0 3 0 0 3 3 3 0 3 9 9 1 1 1 0 0 0 3 0 1 1 1 3 3 9 1 1 1 1 1 3 1 9 0 1 9 3 3 9 3 9 3 3 3 3 3 9 3 9 3 3 3 1 3 3 1 1 3 9 3 1 3 0 9 1 3 9 1 0 0 0 0 0 0 Technology readiness levels (+) 425
Operable # of active ground systems required for servicing (-) 420
Process Verification 2.3 3 0 1 0 3 3 1 0 0 3 9 1 1 9 3 3 1 3 3 9 3 3 9 0 0 0 3 1 0 0 3 3 3 9 9 9 9 3 3 3 0 1 0 0 3 3 3 1 0 1 1 1 3 3 3 3 1 0 3 3 3 3 9 9 3 3 0 3 9 3 3 9 9 9 9 3 9 9 0 # of different fluids in system (-) 404
Auto. Sys. Health Verification 6.90 0 0 0 1 3 9 3 1 1 9 3 9 9 3 1 1 1 9 3 3 3 3 9 0 0 0 0 3 3 3 9 9 3 3 3 3 3 0 0 3 0 9 1 0 9 9 3 3 0 3 1 0 0 0 1 3 0 0 9 9 9 3 9 9 3 3 0 0 3 3 0 3 1 0 1 0 0 1 0 # of checkouts required (-) 403
Auto. Sys. Corrective Action 6.90 0 0 0 1 3 9 3 1 1 3 3 9 9 1 0 0 1 3 3 3 3 3 3 0 0 0 0 1 3 3 1 3 3 3 3 3 1 0 0 1 1 9 3 0 9 3 3 1 0 1 3 3 1 3 1 3 0 0 9 9 9 3 3 9 1 1 0 0 3 3 0 3 0 0 0 0 0 0 0 # of propulsion sub-systems with fault tolerance (+) 398
Ease of Vehicle/Sys. Integration 1.11 9 0 3 1 9 1 9 0 0 3 1 3 1 9 0 0 0 9 1 3 3 1 1 0 0 0 0 1 1 3 3 3 9 9 1 9 9 1 1 3 1 9 3 3 9 9 1 3 3 3 1 9 0 9 9 1 0 0 3 3 3 3 9 9 1 3 1 1 9 3 3 3 1 0 0 0 1 1 0 # of inspection points (-) 390
Maintainable 4.42 9 0 3 1 9 3 9 0 0 9 9 3 1 3 1 1 3 9 3 3 9 1 3 0 1 0 0 3 9 9 9 9 3 1 9 9 3 3 0 9 3 9 1 1 9 3 3 1 1 3 1 3 0 3 9 3 0 0 9 9 9 9 9 9 9 3 3 0 9 3 3 9 9 0 1 1 1 1 1 Mass Fraction required (-) 387
Simple 6.90 3 0 0 0 3 3 9 0 0 9 3 9 9 9 0 0 1 9 3 1 3 0 1 0 0 0 0 1 1 1 9 9 9 3 3 9 9 3 0 3 0 3 1 0 3 3 1 3 0 3 0 0 1 1 1 3 0 0 3 9 9 3 9 9 1 3 1 0 3 1 0 3 3 1 1 1 1 1 0 Hours for turnaround (between launches or commit to new mission) (-) 374
Launch on Demand 1.11 1 0 1 1 9 3 3 0 0 9 9 9 1 3 0 0 1 3 3 1 1 0 3 0 0 0 0 3 9 9 9 9 3 3 3 9 3 0 1 3 3 3 1 1 1 3 1 1 1 1 3 9 3 9 3 3 0 1 9 9 3 3 3 9 3 1 3 3 9 1 0 3 1 0 0 0 0 0 0 ISP Propellant transfer operation difficulty (resupply) (-) 371
Easily Supportable 6.90 9 0 3 1 9 3 9 0 0 9 9 3 3 9 1 1 3 9 3 3 3 1 9 0 1 0 3 3 3 9 9 9 9 3 9 9 9 3 0 9 3 9 1 3 9 9 3 3 3 3 1 9 0 3 9 3 0 0 9 9 9 3 9 9 3 3 3 3 9 3 3 9 9 1 1 1 1 1 1 #pollutive or toxic materials (-) 350
Resiliency 2.49 0 0 0 3 3 3 1 0 0 3 9 3 1 1 1 1 1 3 1 0 1 0 3 0 0 0 0 1 3 9 1 1 1 1 1 3 1 0 1 1 1 3 0 1 3 3 1 1 1 1 3 3 3 9 3 3 0 0 1 3 1 1 3 3 3 0 3 3 3 1 9 3 1 0 0 1 0 0 0 # of expendables (fluid, parts, software) (-) 348
Safety Minimum Impulse bit (-) 332
Vehicle Safety 2.30 3 0 1 9 3 9 9 9 9 3 9 9 9 3 3 1 1 3 9 1 3 9 9 0 0 0 0 0 0 0 0 0 9 1 3 3 1 3 1 1 1 1 1 1 1 3 9 3 1 3 1 1 0 3 1 3 1 0 3 3 9 1 1 0 0 0 0 3 3 1 1 9 1 0 0 0 0 0 0 # of criticality 1 failure modes (-) 329
Personnel Safety 2.30 9 0 9 9 9 3 9 9 9 3 9 9 9 3 3 1 1 3 9 1 1 9 9 0 0 0 0 0 0 0 1 0 9 1 3 3 1 1 1 1 1 1 1 1 1 3 3 1 1 1 0 1 9 9 1 3 0 0 3 3 9 3 3 1 1 0 0 3 3 1 1 3 1 0 0 9 0 1 0 # of element to element interfaces requiring engineering control (-) 320
Public Safety 2.30 3 0 1 9 9 3 9 0 1 1 9 3 9 1 1 1 1 3 3 1 1 9 3 0 0 0 0 0 0 0 0 0 9 1 3 1 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 3 0 0 1 3 9 3 1 0 0 0 0 3 3 1 1 3 0 0 0 1 0 0 0 Ave. Isp on refer. trajectory (+) 310
Sample P
iece o
f Actu
al Matr
ix
Sp
aceL
iner
-100
Pro
pu
lsio
n T
ask
For
ce S
L-1
00 P
rop
uls
ion
Ass
essm
ent/
Pri
orit
izat
ion
Pro
cess
& C
rite
ria
Par
eto
of
all D
esig
n C
rite
ria
do
wn
to
To
p 2
6 g
oo
d d
iscr
imin
ato
rs (
*) u
sed
in W
ork
sho
p P
roce
ss
# of
act
ive
syst
ems
requ
ired
to m
aint
ain
a sa
fe v
ehic
le (
-)60
3
2.72
%#
of d
iffer
ent p
ropu
lsio
n sy
stem
s (-
)58
2
*
2.62
%
5
.34%
# of
sys
tem
s w
ith B
IT B
ITE
(+
)54
2
2.45
%
7
.79%
# of
com
pone
nts
with
dem
onst
rate
d hi
gh r
elia
bilit
y (+
)54
1
2.44
%
10.
%#
of h
ands
on
activ
ities
req
'd (
-)53
4
2.41
%
12.
63%
# of
act
ive
com
pone
nts
requ
ired
to fu
nctio
n in
clud
ing
fligh
t ope
ratio
ns (
-)52
7
*
2.38
%
15
.01%
# of
pot
entia
l lea
kage
/ co
nnec
tion
sour
ces
(-)
527
2.
37%
17.3
9%#
of s
yste
ms
requ
iring
mon
itorin
g du
e to
haz
ards
(-)
523
2.
36%
19.7
4%S
yste
m m
argi
n (+
)50
8
*
2.29
%
22
.03%
# of
toxi
c flu
ids
(-)
495
*
2.
23%
24.2
7%%
of p
ropu
lsio
n sy
stem
aut
omat
ed (
+)
488
*
2.
20%
26.4
7%#
of u
niqu
e st
ages
(fli
ght a
nd g
roun
d) (
-)48
3
*
2.18
%
28
.64%
% o
f pro
puls
ion
subs
yste
ms
mon
itore
d to
cha
nge
from
haz
ard
to s
afe
(+)
470
2.
12%
30.7
6%#
of in
-spa
ce s
uppo
rt s
ys. r
eq'd
for
prop
ulsi
on s
ys. (
- )
465
2.
10%
32.8
6%D
esig
n V
aria
bilit
y (-
)46
4
*
2.09
%
34
.95%
# of
act
ive
on-b
oard
spa
ce s
ys. r
eq'd
for
prop
ulsi
on (
- )
454
*
2.
05%
37.0
0%O
n-bo
ard
Pro
pella
nt S
tora
ge &
Man
agem
ent D
iffic
ulty
in S
pace
(-)
45
3
*
2.04
%
39
.04%
# of
pur
ges
requ
ired
(flig
ht a
nd g
roun
d) (
-)42
8
1.93
%
40
.97%
# of
con
fined
spa
ces
on v
ehic
les
(-)
427
1.
92%
42.8
9%T
echn
olog
y re
adin
ess
leve
ls (
+)
425
*
1.
92%
44.8
1%#
of a
ctiv
e gr
ound
sys
tem
s re
quire
d fo
r se
rvic
ing
(-)
420
1.
89%
46.7
1%#
of d
iffer
ent f
luid
s in
sys
tem
(-)
404
*
1.
82%
48.5
3%#
of c
heck
outs
req
uire
d (-
)40
3
1.82
%
50
.34%
# of
pro
puls
ion
sub-
syst
ems
with
faul
t tol
eran
ce (
+)
398
*
1.
79%
52.1
4%#
of in
spec
tion
poin
ts (
-)39
0
1.76
%
53
.90%
Mas
s F
ract
ion
requ
ired
(-)
387
*
1.
75%
55.6
4%H
ours
for
turn
arou
nd (
betw
een
laun
ches
or
com
mit
to n
ew m
issi
on)
(-)
37
4
1.69
%
57
.33%
ISP
Pro
pella
nt tr
ansf
er o
pera
tion
diffi
culty
(re
supp
ly)
(-)
371
1.
68%
59.0
1%#
of p
ollu
tive
or to
xic
mat
eria
ls (
-)35
0
1.58
%
60
.58%
# of
exp
enda
bles
(flu
id, p
arts
, sof
twar
e) (
-)34
8
1.57
%
62
.15%
Min
imum
Impu
lse
bit (
-)
332
1.
50%
63.6
5%#
of c
ritic
ality
1 fa
ilure
mod
es (
-)32
9
1.48
%
65
.13%
# of
ele
men
t to
elem
ent i
nter
face
s re
quiri
ng e
ngin
eerin
g co
ntro
l (-)
320
1.
44%
66.5
7%A
ve. I
sp o
n re
fer.
traj
ecto
ry (
+)
310
*
1.
40%
67.9
7%#
of p
arts
(di
ffere
nt, b
acku
p, c
ompl
ex)
(-)
296
1.
33%
69.3
1%#o
f um
bs. r
eq'd
to L
aunc
h V
ehic
le (
- )
27
6
*
1.25
%
70
.55%
# of
eng
ines
(-)
274
*
1.
24%
71.7
9%R
esis
tanc
e to
Spa
ce E
nviro
nmen
t (+
)
268
*
1.
21%
73.0
0%#
of p
hysi
cally
diff
icul
t to
acce
ss a
reas
(-)
265
1.
19%
74.1
9%#
of a
ctiv
e en
gine
sys
tem
s re
quire
d to
func
tion
(-)
247
*
1.
11%
75.3
0%In
tegr
al s
truc
ture
with
pro
puls
ion
sys.
(+
)
239
*
1.
08%
76.3
8%H
ours
to r
efur
bish
pro
puls
ion
syst
em (
-)23
7
1.07
%
77
.45%
# of
man
hour
s (c
/o, h
andl
e, a
ssem
ble
etc)
on
syst
em b
etw
een
on a
nd
off c
ycle
s (L
ow C
ycle
Fat
igue
) or
use
(H
igh
Cyc
le F
atig
ue)
(-)
229
1.
03%
78.4
8%#
of m
odes
or
cycl
es (
-)22
7
*
1.02
%
79
.50%
# of
gro
und
pow
er s
yste
ms
(-)
226
*
1.
02%
80.5
2%M
ean
time
betw
een
maj
or o
verh
aul (
+)
22
1
1.00
%
81
.52%
Am
ount
of e
nerg
y re
leas
e fr
om u
npla
nned
rea
ctio
n of
pro
pella
nt (
-)21
9
*
0.99
%
82
.51%
Mar
gin,
mas
s fr
actio
n (+
)21
5
*
0.97
%
83
.48%
Mar
gin,
thru
st le
vel /
eng
ine
cham
ber
pres
s(+
)21
1
*
0.95
%
84
.43%
Tra
nspo
rtat
ion
trip
tim
e (-
)21
1
*
0.95
%
85
.38%
# of
eng
ine
rest
arts
req
uire
d (-
)20
1
*
0.91
%
86
.29%
Attributes versus Programmatic Criteria - Matrix / Pareto
Chart for Weighting SL-100 Advanced Space
Transportation (Pre-Operational Phases,
R&D and Aquisition) (Qu
alit
y C
har
acte
rist
ic)
Pro
gra
mm
atic
Cri
teri
a
TR
D-c
ost
to r
each
TR
L -6
(-)
TR
D-t
otal
ann
ual f
undi
ng b
y ite
m a
t pe
ak d
olla
r
Req
uire
men
ts (
-)
TR
D-#
of
new
fac
ilitie
s re
quire
d co
stin
g ov
er
$2M
(-)
TR
D-e
stim
ated
tim
e to
rea
ch T
RL
6 fr
om s
tart
o
f R
&D
(-)
TR
D-C
urre
nt T
RL
(+)
TR
D-#
ope
ratio
nal e
ffec
tiven
ess
attr
ibut
es
pr
evio
usly
dem
onst
rate
d (+
)
TR
D-#
ope
ratio
nal e
ffec
tiven
ess
attr
ibut
es
add
ress
ed f
or im
prov
emen
t (+
)
TR
D-#
tec
hnol
ogy
brea
kthr
ough
s re
quire
d to
dev
elop
and
dem
onst
rate
(-)
TR
D-#
ful
l sca
le g
roun
d or
flig
ht d
emon
stra
tions
requ
ired
(-)
TR
D-#
rela
ted
tech
nolo
gy d
atab
ases
ava
ilabl
e (+
)
TR
D-#
mul
tiuse
app
licat
ions
incl
udin
g sp
ace
tran
spor
tatio
n (+
)
PA
-tot
al s
yste
m D
DT
&E
con
cept
dev
elop
men
t
and
impl
emen
tatio
n co
st (
-)
PA
-infr
astr
uctu
re c
ost:
initi
al s
yste
m
impl
emen
tatio
n (c
apita
l inv
estm
ent)
(-)
PA
-tim
e re
quire
d to
est
ablis
h in
fras
truc
ture
(sch
edul
e of
R&
D p
hase
) (-
)
(Demanded Quality) Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Attributes WEIGHT
Technology R&D Phase
Cost 30 9 3 9 9 9 9 9 9 9 3 1 Benefit Focused 30 9 3 1 9 3 3 9 9 3 3 1
Schedule 15 3 1 3 9 9 3 3 9 9 3 1
Risk 15 1 1 3 3 9 9 9 9 9 9 1
Dual use Potential 10 1 3 1 3 1 1 3 9
Program Acquisition Phase
Cost 25 9 9 3 Schedule 15 3 3 9
Risk 25 1 1 3
Technology Options 10 3 3 3
Investor Incentive 25 9 9 9
RAW SCORE 600 210 400 750 630 550 750 820 640 390 180 550 550 540
SUM OF RAW SCORES TRD 5920
SUM OF RAW SCORES PA 4550
NORMALIZED SCORE 10 4 7 13 11 9 13 14 11 7 3 12 12 12
PA
-tec
hnol
ogy
read
ines
s at
pro
gram
acq
uisi
tion
mile
ston
e: T
RL
6 +
mar
gin
(+)
PA
-tec
hnol
ogy
capa
bilit
y m
argi
n (p
erfo
rman
ce
as
fra
ctio
n of
ulti
mat
e) (
+)
PA
-# o
f ot
her
optio
ns a
vaila
ble
(+)
PA
-# m
ajor
new
tec
hnol
ogy
deve
lopm
ent
item
s
(eng
ines
, ai
rfra
mes
, T
PS
, et
c) (
-)
PA
-# it
ems
requ
iring
maj
or g
roun
d te
st a
rtic
les
and
dem
onst
ratio
n (e
xam
ple:
new
eng
ines
) (-
)
15 16 17 18 19
9 3 3 9 39 3 3 9 3
9 9 9 9 3
9 9 3 9 3
3 3 3 9 3
750 510 450 900 300
16 11 10 20 7
SPST / SL-100 Space Propulsion
Attributes versus Programmatic Criteria - Matrix / Pareto
Note: TRD - Technology Research and Development Stage PA - Program Acquisition Phase
Program Acquisition Phase
PA-# major new technology development items (engines, airframes, TPS, etc) (-) 20%PA-technology readiness at program acquisition milestone: TRL 6 + margin (+) 16%PA-time required to establish infrastructure (schedule of R&D phase) (-) 12%PA-total system DDT&E concept development and implementation cost (-) 12%PA-infrastructure cost: initial system implementation (capital investment) (-) 12%PA-technology capability margin (performance as fraction of ultimate) (+) 11%
PA-# of other options available (+) 10% PA-# items requiring major ground test articles and demonstration (example: new engines) (-) 7%
Technology R & D Phase
TRD-# technology breakthroughs required to develop and demonstrate (-) 14% TRD-estimated time to reach TRL 6 from start of R&D (-) 13%
TRD-# operational effectiveness attributes addressed for improvement (+) 13% TRD-Current TRL (+) 11%
TRD-# full scale ground or flight demonstrations required (-) 11% TRD-cost to reach TRL -6 (-) 10% TRD-# operational effectiveness attributes previously demonstrated (+) 9% TRD-#related technology databases available (+) 7% TRD-# of new facilities required costing over $2M (-) 7%
TRD-total annual funding by item at peak dollar requirements (-) 4% TRD-# multiuse applications including space transportation (+) 3%
SPST / SL-100 Space Propulsion
0 5 10 15 20
PA-# items requiring major ground test articles
and demonstration (example: new engines) (-)
PA-# of other options available (+)
PA-technology capability margin (performance asfraction of ultimate) (+)
PA-time required to establish infrastructure
(schedule of R&D phase) (-)
PA-total system DDT&E concept development
and implementation cost (-)
PA-infrastructure cost: initial system
implementation (capital investment) (-)
PA-technology readiness at program acquisition
milestone: TRL 6 + margin (+)
PA-# major new technology development items
(engines, airframes, TPS, etc) (-)
TRD-# multiuse applications including space
Transportation (+)
TRD-total annual funding by item at peak dollar
Requirements (-)
TRD-#related technology databases available (+)
TRD-# of new facilities required costing over $2M (-)
TRD-# operational effectiveness attributes
previously demonstrated (+)
TRD-cost to reach TRL -6 (-)
TRD-Current TRL (+)
TRD-# full scale ground or flight demonstrations
Required (-)
TRD-estimated time to reach TRL 6 from start of
R&D (-)
TRD-# operational effectiveness attributes
addressed for improvement (+)
TRD-# technology breakthroughs required to
develop and demonstrate (-)
SCORE
Attributes versus Programmatic Criteria - Matrix / ParetoSPST / SL-100 Space Propulsion
Sp
aceL
iner
100
Pro
pu
lsio
n T
ask
For
ce
Sp
aceL
iner
100
Pro
pu
lsio
n A
sses
smen
t/P
rior
itiz
atio
n P
roce
ss &
Cri
teri
a
Tec
hn
olog
y E
valu
atio
n B
enef
its
(Tec
hn
ical
) A
ttri
bu
tes
and
Ass
ocia
ted
Des
ign
Cri
teri
aB
enef
its
(Tec
hn
ical
Wit
h S
ense
of
Goo
dn
ess
and
Nor
mal
ized
Wei
ghti
ng)
Aff
ord
able
/ L
ow L
ife
Cyc
le C
ost
Min
. C
os
t Im
pa
ct
on
La
un
ch
Sy
s.
Lo
w R
ec
urr
ing
Co
st
Lo
w C
os
t S
en
s.
to F
lt.
Gro
wth
*
Op
era
tio
n a
nd
Su
pp
ort
In
itia
l A
cq
uis
itio
n
Ve
hic
le/S
ys
tem
Re
pla
ce
me
nt
Raw
%
Sco
re
Wei
ght
N
o. 4
9 #
of u
niqu
e st
ages
(fl
ight
and
gro
und)
(-)
483
5.3
%N
o. 7
5 #
of
acti
ve o
n-bo
ard
spac
e sy
s. r
eq'd
for
pro
puls
ion
( -
) 4
54
4
.9 %
No.
78
On-
boar
d P
rope
llan
t Sto
rage
& M
anag
emen
t Dif
ficu
lty
in S
pace
(-)
453
4.9
%N
o. 3
8 T
echn
olog
y re
adin
ess
leve
ls (
+)
425
4.6
%N
o. 5
9 M
ass
Fra
ctio
n re
quir
ed (
-)
3
87
4
.2 %
No.
54
Ave
. Isp
on
refe
r. tr
ajec
tory
(+
)
3
10
3
.4 %
No.
70
# o
f um
bs. r
eq'd
to L
aunc
h V
ehic
le (
- )
276
3.0
%N
o. 5
8 #
of e
ngin
es (
-)
2
74
3
.0 %
No.
79
Res
ista
nce
to S
pace
Env
iron
men
t (+
)
26
8
2
.9 %
No.
82
Inte
gral
str
uctu
re w
ith
prop
ulsi
on s
ys. (
+)
239
2.6
%N
o. 8
5 T
rans
port
atio
n tr
ip ti
me
(-)
211
2.3
%
Dep
end
able
Hig
hly
Re
lia
ble
Inta
ct
Ve
hic
le R
ec
ov
ery
Mis
sio
n S
uc
ce
ss
Op
era
te o
n C
om
ma
nd
Ro
bu
stn
es
s
De
sig
n C
ert
ain
ty
Raw
%
Sco
re
Wei
ght
N
o. 1
0 #
of a
ctiv
e co
mpo
nent
s re
quir
ed to
fun
ctio
n in
clud
ing
flig
ht
oper
atio
ns (
-)52
7
5
.7 %
No.
87
Des
ign
Var
iabi
lity
(-)
464
5.0
%N
o. 1
4 #
of d
iffe
rent
flu
ids
in s
yste
m (
-)40
4
4
.4 %
No.
60
# of
act
ive
engi
ne s
yste
ms
requ
ired
to f
unct
ion
(-)
247
2.7
%N
o.
48
# o
f m
od
es
or
cycl
es
(-)
22
7
2.
5 %
No
. 1
6 M
arg
in,
ma
ss f
ract
ion
(+
)2
15
2.3
%N
o.
18
Ma
rgin
, th
rust
leve
l/en
gin
e c
ha
mb
er
pre
ss (
+)
21
1
2.
3 %
No
. 6
4 #
of
en
gin
e r
est
art
s re
qu
ired
(-)
20
1
2.
2 %
Sp
aceL
iner
100
Pro
pu
lsio
n T
ask
For
ce
Sp
aceL
iner
100
Pro
pu
lsio
n A
sses
smen
t/P
rior
itiz
atio
n P
roce
ss &
C
rite
ria
Tec
hn
olog
y E
valu
atio
n B
enef
its
(Tec
hn
ical
) A
ttri
bu
tes
and
Ass
ocia
ted
Des
ign
C
rite
ria
Con
’t
Res
pon
sive
Fle
xib
leC
ap
ac
ity
Op
era
ble
P
roc
es
s V
eri
fic
ati
on
A
uto
. S
ys
. H
ea
lth
Ve
rifi
ca
tio
n
Au
to.
Sy
s.
Co
rre
cti
ve
Ac
tio
n
Ea
se
of
Ve
hic
le/S
ys
tem
In
teg
rati
on
M
ain
tain
ab
le
Sim
ple
L
au
nc
h o
n D
em
an
d
Ea
sil
y S
up
po
rta
ble
R
es
ilie
nc
y
Raw
%
S
core
W
eigh
t N
o.
37
# o
f di
ffer
ent p
ropu
lsio
n sy
stem
s (-
)58
2
6.
3 %
No.
66
Sys
tem
Mar
gin
(+)
508
5.5
%N
o. 3
3 %
of
prop
ulsi
on s
yste
m a
utom
ated
(+
)48
8
5.
3 %
No.
53
# of
gro
und
pow
er s
yste
ms
(-)
226
2.5
%
En
viro
nm
enta
l Com
pat
ibil
ity
Min
imu
m I
mp
ac
t o
n S
pa
ce
En
vir
on
.M
inim
um
Eff
ec
t o
n A
tmo
sp
he
re
Min
imu
m E
nv
iro
n.
Imp
ac
t a
ll S
ite
s
Saf
ety
Ve
hic
le S
afe
tyP
ers
on
ne
l S
afe
tyP
ub
lic
Sa
fety
Eq
uip
me
nt
an
d F
ac
ilit
y S
afe
ty
R
aw
%
S
core
W
eigh
tN
o. 5
# o
f to
xic
flui
ds (
-)49
5
5
.4 %
No.
6 #
of
prop
ulsi
on s
ub-s
yste
ms
wit
h fa
ult t
oler
ance
(+
)39
8
4
.3 %
No.
4 A
mou
nt o
f en
ergy
rel
ease
fro
m u
npla
nned
rea
ctio
n of
pro
pell
ant (
-)21
9
2
.4 %
Pu
bli
c S
up
por
tB
en
efi
t G
NP
So
cia
l P
erc
ep
tio
n
Sp
aceL
iner
100
Pro
pu
lsio
n T
ask
For
ceIn
-Sp
ace
Pro
pu
lsio
n A
sses
smen
t/P
rior
itiz
atio
n P
roce
ss &
Cri
teri
a
Can
did
ate
Tec
hn
olog
ies
PR
OG
RA
MM
AT
IC A
ss
es
sm
en
t C
rite
ria
Pro
gram
Acq
uis
itio
n P
has
eP
A-#
ma
jor
ne
w t
ech
no
log
y d
eve
lop
me
nt
item
s (e
ng
ine
s, a
irfra
me
s, T
PS
, e
tc)
(-)
20
%
PA
-te
chn
olo
gy
rea
din
ess
at
pro
gra
m a
cqu
isiti
on
mile
sto
ne
: T
RL
6 +
ma
rgin
(+
)
16
%
PA
-tim
e r
eq
uire
d t
o e
sta
blis
h in
fra
stru
ctu
re (
sch
ed
ule
of
R&
D p
ha
se)
(-)
1
2 %
PA
-to
tal s
yste
m D
DT
&E
co
nce
pt
de
velo
pm
en
t a
nd
imp
lem
en
tatio
n c
ost
(-)
1
2 %
PA
-infr
ast
ruct
ure
co
st:
initi
al s
yste
m im
ple
me
nta
tion
(ca
pita
l in
vest
me
nt)
(-)
1
2 %
PA
-te
chn
olo
gy
cap
ab
ility
ma
rgin
(p
erf
orm
an
ce a
s fr
act
ion
of
ulti
ma
te)
(+)
1
1 %
PA
-# o
f o
the
r o
ptio
ns
ava
ilab
le (
+)
1
0 %
PA
-# it
em
s re
qu
irin
g m
ajo
r g
rou
nd
te
st a
rtic
les
& d
em
on
stra
tion
(e
x: n
ew
en
gin
es)
(-)
7 %
Tec
hn
olog
y R
& D
Ph
ase
T
RD
-# t
ech
no
log
y b
rea
kth
rou
gh
s re
qu
ired
to
de
velo
p a
nd
de
mo
nst
rate
(-)
1
4 %
TR
D-e
stim
ate
d t
ime
to
re
ach
TR
L 6
fro
m s
tart
of
R&
D (
-)
13
%
TR
D-#
op
era
tion
al e
ffe
ctiv
en
ess
att
ribu
tes
ad
dre
sse
d f
or
imp
rove
me
nt
(+)
1
3 %
TR
D-C
urr
en
t T
RL
(+
)
11
%
TR
D-#
fu
ll sc
ale
gro
un
d o
r fli
gh
t d
em
on
stra
tion
s re
qu
ired
(-)
1
1 %
TR
D-c
ost
to
re
ach
TR
L –
6 (
-)
10
%
TR
D-#
op
era
tion
al e
ffe
ctiv
en
ess
att
ribu
tes
pre
vio
usl
y d
em
on
stra
ted
(+
)
9
%
TR
D-#
re
late
d t
ech
no
log
y d
ata
ba
ses
ava
ilab
le (
+)
7 %
TR
D-#
of
ne
w f
aci
litie
s re
qu
ired
co
stin
g o
ver
$2
M (
-)
7
%
TR
D-t
ota
l an
nu
al f
un
din
g b
y ite
m a
t p
ea
k d
olla
r re
qu
irem
en
ts (
-)
4
%
TR
D-#
mu
lti-u
se a
pp
lica
tion
s in
clu
din
g s
pa
ce t
ran
spo
rta
tion
(+
)
3
%