O Rhodes summit10 - SEAri at MITseari.mit.edu/documents/summit/2010/02-SEAriSummit10_O_DHR.pdf ·...
Transcript of O Rhodes summit10 - SEAri at MITseari.mit.edu/documents/summit/2010/02-SEAriSummit10_O_DHR.pdf ·...
2010 SEAri Annual Research Summit
SEAri Overview and Motivations
Dr Donna H Rhodes (Di t P i i l R h S i ti t SEA i)Dr. Donna H. Rhodes (Director, Principal Research Scientist, SEAri)
October 19, 2010C b id MACambridge, MA
Massachusetts Institute of Technology
SEAri Portfolio & Methods
RESEARCH PORTFOLIO
• Socio-Technical Decision Making
• Designing for Value Robustness
• Systems Engineering Economics
METHODS USED
• Models and Simulations: MATLAB Models Agent-based• Systems Engineering Economics
• Systems Engineering in the Enterprise
• Systems Engineering Strategic Guidance
MATLAB Models, Agent-based Models, STK
• Empirical studies of historical systems programs andsystems, programs, and practices
• Grounded theory, coding/memo writing methodscoding/memo writing methods, latent semantic analysis
• Experiment-based studies: advanced analyses visualizing
seari.mit.edu2010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology 2
advanced analyses, visualizing complex data sets
Selected Examples of SEAri Research and ContributionsResearch and Contributions
CONCEPT DESIGN
How can we incorporate operational variables in tradespace exploration?
How can multi-stakeholder negotiations be augmented?
Examples of Recent Knowledge Contributions
INNOVATIONWh t ff ti Wh t t i ti i t
• tradespace exploration methods• changeability taxonomy • new metrics for several “ilities”
What are effective innovation models/strategies
in government settings?
What uncertainties impact product platform design and how can they be managed?
• 17 survivability design principles• real options framework• fractionated spacecraft study
SYSTEM PROPERTIES (ilities)
How can we measure system adaptability?
How can ilities be traded-off in system decisions?
• leading indicators for HSI • epoch-era analysis• traits of systems thinking teams
system adaptability? in system decisions?
seari.mit.edu 32010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Influence and ImpactInfluence• Briefing to Defense Sciences Board
P t ti t CNO St t i St di G• Presentation to CNO Strategic Studies Group • Panelist at DARPA Complexity Workshop• Participation in ISAT Summer Study
Collaborate• Involvement in INCOSE, IEEE, AIAA• Leadership of INCOSE Doctoral Student Network• Hosting Technical Exchange Meetings• Hosting Technical Exchange Meetings• Cross-University Collaboration
Transfer KnowledgeTransfer Knowledge• Sharing publications and presentations via SEAri Website• Best Papers: INCOSE Journal 2008 & 2009• Numerous Conference Papers (6 awards in last 3 years)• Teaching MIT Professional Education Courses
seari.mit.edu 4
Teaching MIT Professional Education Courses
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
SEAri Graduates in the Workforce Academic Year 2009/2010Academic Year 2009/2010
• US Air ForceUS M i• US Marines
• US Coast Guard• Consultancy (defense)• JPL• Lincoln Labs• NGANGA • Singapore DSTA• West Point Military Academy
SEAri provides students with a collaborative learning environment focused on real-world problems and collaborating with experts in government and industry
thereby preparing them to contribute to significant systems challenges
seari.mit.edu 5
…thereby preparing them to contribute to significant systems challenges
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Sources of UncertaintySources of Uncertainty
Y
P
X
seari.mit.edu 62010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Where Can Uncertainties Come About?Come About?
• Technology – (e g new type of material)– (e.g., new type of material)
• Policy – (e.g., change in safety standard)
• Economy
Many uncertainties stem from soft factors that are even more difficult to
• Economy – (e.g., economic downturn)
• Resources (e g level of investment)
anticipate
- Demographics
C lt l– (e.g., level of investment)• Markets
– (e.g., new competitor)E d U
- Cultural
- Social factors
• End Uses – (e.g., emergent use of product)
• Environment ( h d l b l i )– (e.g., change due to global warming)
seari.mit.edu 72010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Sources of UncertaintyTechnology and EnvironmentTechnology and Environment
TECHNOLOGY ENVIRONMENT• Innovation in technology
type/purpose• Availability of new
• Degradation of global environment
• Depletion of natural• Availability of new materials
• Miniaturization
• Depletion of natural resources
• Resulting worldview shifts • Interoperability • IP rights
(e.g., “green” starts to influence buying habits)
• Impacts of environmental• Connectivity • Impacts of environmental factors
seari.mit.edu 82010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Sources of UncertaintyEconomics and PolicyEconomics and Policy
ECONOMICS GOVERNANCE/POLICYP li t i t d• Economic conditions in
region/nation• Funding profiles
• Policy constraints and implications
• Intellectual property rights• Funding profiles • Shifts in spending profile• Inter-nation agreements
p p y g• Authority centralization/-
decentralization • Shifts in public/privateg
and sanctions• Investment funding
• Shifts in public/private ownership
• State/regional/national b l• Open/closure of markets governance balance
seari.mit.edu 92010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Sources of Uncertainty Demographics and Social FactorsDemographics and Social Factors
• Population growth/aging trends • Population growth location
Complex Relationships • Population growth location
– Additional growth will be in developing countries*
• Rural vs. urban dwellers
Population growth combined with socio-economics impacts
population average age– By 2032 over 2 billion new city
dwellers*• Distribution of wealth in
Urbanization aggravates environmental pressures and
i l t isocieties• Impact of diseases in nations• Economic inequities within
social tensions
Can indirectly impact many things qregions/ between nations
* Source: Global Environmental Outlook Scenario Framework, Tellus Institute, March 2002
y p y gsuch as acquisition related polices
and threat environment
seari.mit.edu 102010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Sources of Uncertainty Soft FactorsSoft Factors
Many uncertainties rooted in soft factors that are evensoft factors that are even more difficult to anticipate…
Examples:• Demographic
influences/power shiftsinfluences/power shifts• Worldview shifts
– (e.g., “green” starts to influence b i h bit )buying habits)
• Trust profiles
seari.mit.edu 11
Source: Pew Research Center 2008
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Sources of Uncertainty Impact of Disruptive EventsImpact of Disruptive Events
What if Category 5 Hurricane Hits New York?Hits New York?
Based on scenario sketched by Risk Management Solutions (RMS*)
New York the world's second• New York the world's second-most-expensive hurricane target, after Miami, with an estimated cost of a Cat 5 direct Source: Fisher and Helman, If you think the Oil Spill is Bad,estimated cost of a Cat 5 direct hit of $320 billion*
• Cost escalates to $2.2 trillion by
Source: Fisher and Helman, If you think the Oil Spill is Bad, Forbes, June 28, 2010
2070 if sea levels rise as expected
seari.mit.edu 122010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
D i St t iDynamic Strategies
What strategies can help us anticipate and consider the impacts of uncertainties?consider the impacts of uncertainties?
seari.mit.edu 132010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Motivations for Dynamic Strategies
STAKEHOLDER NEEDS CHANGE AS PERCEPTION OF SYSTEM
AND VALUE DELIVEREDEngineering complex
i t h i lAND VALUE DELIVERED EVOLVES
SYSTEMS EXIST IN DYNAMIC CULTURAL POLITICAL
socio-technical systems in a dynamic world
CULTURAL, POLITICAL, FINANCIAL, MARKET
ENVIRONMENTS
HIGHLY COMPLEX AND
NASA
requires multi-faceted methods that evolve over time and throughHIGHLY COMPLEX AND
INTERCONNECTED SYSTEMS WITH CHANGING TECHNOLOGY
OVER LONG LIFESPANSDeere & Company
over time and through synergies of individual research contributions
The engineering of systems has always considered a multitude of dimensions …. and increasingly requires formal methods and
seari.mit.edu 14
enabling technologies to respond to uncertain futures
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Creating Anticipatory Capacity
“Designers” do an adequate job of understanding value perceptions in the
Anticipatory Capacity is the capacity to continuously develop and apply value perceptions in the
short run…but to do so in the long run requires:
• effectively anticipating
y p pp yknowledge acquired through a structured approach to anticipate:(1) changing scenarios as
stakeholder needs and systems context change over• effectively anticipating
what the future will bring• incorporating this
knowledge into
systems context change over time;
(2) to consider their consequences; and
(3) t f l t d i d i iknowledge into present decisions
(3) to formulate design decisions in response.
Rhodes and Ross 2008
SEAri research acknowledges that we can not predict the future in its entirety… but we can anticipate possible and probable scenarios for the
f d di i l d i f h i
seari.mit.edu 15
future, and predict sequential orderings for these scenarios in order to design value robust systems
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Churchill War Rooms: Historical Example of Creating Anticipatory Capacity g p y p y
Three important rooms in complex: Cabinet War Room, Map Room, and Winston Churchill's room
R l ti d i i ki i t t tReal-time decision-making environment at most senior levels/inner sanctum of British government
Source: Kozak-Holland, M., Information Management Special Report, 2007Map Room acted as an p pMap Room acted as an
executive dashboard in providing real-time synthesized information and key performance indicators
Churchill had to transform his organization to the modern-day equivalent of an Adaptive Enterprise …he did this using the emerging technologies of the day…
pe o a ce d ca o s
seari.mit.edu 16
Kozak-Holland, Churchill's Adaptive Enterprise: Lessons for Business Today
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Three Enablers for Anticipatory CapacityAnticipatory Capacity
MINDSET/SKILLS METHODS ENVIRONMENTS
Ability to think deeply about Decades of anticipatory Bring together decision y p y‘systems in context’
Enhanced ability to think about ‘systems in time’
Decades of anticipatory methods but…limited to high level strategies or graphical/narrative scenarios
g gmakers and analysts
Provide computing power/toolsets to enact methods
Situational Leadership –make decisions at multiple system levels and across time periods
scenarios
Model-based approach provides ability to parametrically derive
enact methods
Enables effective display of complex data sets and analyses to facilitate dialoguepossible ‘futures’ and
run simulationsdialogue
seari.mit.edu 17
Rhodes, D.H. and Ross, A.M., "Anticipatory Capacity: Leveraging Model-Based Approaches to Design Systems for Dynamic Futures," 2nd Annual Conference on Model-based Systems, Haiffa, Israel, March 2009
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Five Aspects Taxonomya useful focusing framework for inquirya use u ocus g a e o o qu y
STRUCTURALrelated to form of system components and
their interrelationshipstheir interrelationships
BEHAVIORALrelated to function/performance, operations,
and reactions to stimuli
CONTEXTUALrelated to circumstances in which the
system or enterprise existsf
TEMPORALrelated to the dimensions and properties of
systems over timerelated to stakeholder preferences
PERCEPTUALrelated to stakeholder preferences,
perceptions and cognitive biasesD. Rhodes, Managing Complexity in Aerospace Systems Engineering and Design , Solutions for Complexity Panel, DARPA Workshop on Complexity, September 22, 2009, Rosslyn, VA.
seari.mit.edu 18
p y p y
D. Rhodes, and A. Ross, Five Aspects of Engineering Complex Systems: Emerging Constructs and Methods, IEEE Systems Conference, April 2010
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Five AspectsExample Constructs and ConsiderationsExample Constructs and Considerations
STRUCTURAL• heterogeneous components and constituent systems• elaborate networks, loose and tight couplings• layers vertical/horizontal structures multiplicity of scales• layers, vertical/horizontal structures, multiplicity of scales
BEHAVIORAL• complex variance in response to stimuli • unpredictable behavior of technological connections
t i l t k b h i• emergent social network behavior
CONTEXTUAL• many complexities and uncertainties in system context • political, economic, environmental, threat, market factors • stakeholder needs profile and overall worldview
TEMPORAL• decoupled acquisition phases and context shifts • systems with long lifespan and changing characteristics • time-based system properties (flexibility, survivability, etc.)
PERCEPTUAL• many stakeholder preferences to consider • perception of value shifts changes with context shifts • cognitive constraints and biases
seari.mit.edu 192010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Five Aspects of Complex Systemsd i t t i id t t ti ti hiftdynamic strategies consider context, time, perception shifts
STRUCTURALSTRUCTURAL Addressed via “state of the practice” systems architecting and model-based
systems engineering BEHAVIORAL syste s e g ee g
CONTEXTUALEmerging “state of art”
Epoch Modeling p gMulti-Epoch Analysis Epoch-Era Analysis
Multi-Dimensional Tradespace ExplorationTEMPORAL u t e s o a adespace p o at oMulti-Stakeholder Negotiations
Comprehension of Complex DatasetsCognition-based studies of Decision Makers
TEMPORAL
PERCEPTUAL Cognition based studies of Decision Makersand more….
PERCEPTUAL
seari.mit.edu 202010 SEAri Research Summit
© 2010 Massachusetts Institute of Technology
Contextual Aspect Model-based ApproachModel based Approach
comptroller SI&E
SRS Enterprise Boundary
National Security Strategy/PolicyNational Security Strategy/Policy
DNIUSD(I)
ExtendedSRS
Enterprise
Which SRS Architecture? Definition of Epoch
Time period with a fixed context and needs; characterized by static
Category Variable Name Definition Range
Satellite Radar SystemProgram Manager
Nation
Capital(non‐fungible assets)
Capital(non‐fungible assets)
Resources(fungible assets)
Resources(fungible assets)
RadarProductRadarProduct
NGAJ2
Military
SRS Context
OMBCongress
R&DR&D Comm/GrndComm/Grnd
Infra‐
Struct.
Time period with a fixed context and needs; characterized by static constraints, concepts, available technologies, and articulated expectations
E
Capital
Technology Level
Includes constants for spacecraft (ex. radar and bus) available technology
Level 1 (Low), equiv. TRL = 9 technologyLevel 2 (Med), equiv. TRL = 6 technologyLevel 3 (High), equiv. TRL = 4 technology
Comm. Level Availability of ground stations and space-based relay options
Level 1 – No Backbone + AFSCN Ground Sites Level 2 – WGS + AFSCN Ground Sites
AISR A il bilit f AISR t Y / N
Epoch Vect
AISR Availability of AISR assets Yes / No
Radar Product
Target list Defines the target areas of interest along with target RCS variations
Op plan 9: area AOp plan 19: area BOp plan 44:Op plan 45:Op plan 49:Op plan 60:Op plan 84: O l 94
648 Future
Contextstor Op plan 94:Op plan 103:
Environment Communications jamming Yes / No
Nat Sec Strat/Policy
Utility SAR v. GMTI
Relative importance of the two stakeholder types of multi-attribute utility
Level 1 – SAR < GMTILevel 2 – SAR = GMTILevel 3 – SAR > GMTI
Resources NA Vary budget constraints Era-level Attributes
seari.mit.edu 21
Epoch variables allow for parameterization of some “context” drivers for system value
Resources NA Vary budget constraints Era-level Attributes
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Contextual Aspect Example:Tradespace Shift Across EpochsTradespace Shift Across Epochs
Epoch “171”Baseline Program Context:
Standalone capability needed, Imaging mission (primary)
Epoch “193”New Program Context:
Cooperative capability needed, Tracking mission (primary)(primary) (primary)
Epoch variables are defined in regard to uncertainties (for example, resources, policy, technology availability, and others). Epochs are computationally generated using the possible permutations of the epoch variable set values. This approach has enabled deeper analysis for assessing performance of concept designs across multiple epochs
seari.mit.edu 22
deeper analysis for assessing performance of concept designs across multiple epochs. A.M. Ross and D.H. Rhodes, “Using Natural Value-centric Time Scales for Conceptualizing System Timelines through Epoch-Era Analysis,”18th INCOSE International Symposium, Utrecht, the Netherlands, June 2008
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Temporal Aspect
• Temporal aspect of systems is critically important, but remains undertreated in engineering practiceundertreated in engineering practice
• Use of system scenarios most typical method but “illustrative”• Time-based (e.g., survivability , adaptability) increasingly important
Value (utility) of designs for cost shown across system era with four epoch shifts (arrow indicates design of interest)( g )
A.M. Ross and D.H. Rhodes, “Using Natural Value-centric Time Scales for Conceptualizing System Timelines through Epoch-Era Analysis,”18th
seari.mit.edu 23
INCOSE International Symposium, Utrecht, the Netherlands, June 2008.
C..J. Roberts, M.G. Richards, A.M. Ross, D.H. Rhodes, and D.E. Hastings, "Scenario Planning in Dynamic Multi-Attribute Tradespace Exploration," 3rd Annual IEEE Systems Conference, Vancouver, Canada, March 2009
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Perceptual Aspect Example: Tradespace Shifts as Perceived Value ShiftsTradespace Shifts as Perceived Value Shifts
Perceptual aspect can relate to need to understand ‘goodness’ of design concepts as a stakeholder’s preferences shift over time. Exogenous factors such as
economic changes, available technology, threats and other factors may influence
Original Attribute Relative Weights
Changed Attribute Relative Weights
economic changes, available technology, threats and other factors may influence relative importance of what a stakeholder values.
Impact of Change in Stakeholder Weighting of Desired System Attributes in
Tradespace showing Utility vs Cost for a Multi-Concept System
seari.mit.edu 24
p g y p y
D. Chattopadhyay, A.M. Ross and D.H. Rhodes," Demonstration of System of Systems Multi-Attribute Tradespace Exploration on a Multi-Concept Surveillance Architecture," 7th Conference on Systems Engineering Research, Loughborough University, UK, April 2009
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Combining Aspects Example: Temporal and PerceptualTemporal and Perceptual
What visual construct can combine:
• temporal aspect• temporal aspect(effective display of time-based impacts)andperceptual aspect• perceptual aspect(ability of decision maker to cognitively process complex tradespacetradespace information)?
Richards (2009): Perceptually understandable display of value for cost of satellite radar designs with time based information on survivability of system as it
Challenge: amount of information and complexities within a data setCognitive limits for processing the visual display must be considered as well as mechanisms
radar designs with time-based information on survivability of system as it experiences possible finite disturbances over its lifespan
seari.mit.edu 25
Cognitive limits for processing the visual display must be considered, as well as mechanisms to compute and display synthesis of temporal analysis (survivability over system life)
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
Multi-Aspect Synthesis Example: Responsive Systems Comparison (RSC)Responsive Systems Comparison (RSC)
Process 1Value-Driving Context Definition
Process 2Value-Driven Design
Formulation RSC consists of
Using Multi-Attribute Tradespace Exploration, Epoch-Era Analysis, and
other approaches a coherentProcess 3
Epoch Characterization
Process 4Design Tradespace
Evaluation
Process 5Multi-Epoch
Analysis
Process 6Era Construction
seven processes:1. Value-Driving Context Definition2. Value-Driven Design Formulation3. Epoch Characterization4. Design Tradespace Evaluation5 Multi Epoch Analysis
other approaches, a coherent set of processes were
developed into the RSC method
Process 7Lifecycle Path
Analysis
Time
5. Multi-Epoch Analysis6. Era Construction7. Lifecycle Path Analysis
Synthesis of multi-aspect methods can be used to develop robust methods for engineering complex systems
Example: RSC seven process method supported by mindset/skills and enabled with venue for collaborative decision making
seari.mit.edu 26
A. M. Ross, H.L. McManus, D.H. Rhodes, D.E. Hastings, and A.M. Long, "Responsive Systems Comparison Method: Dynamic Insights into Designing a Satellite Radar System," AIAA Space 2009, Pasadena, CA, September 2009
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology
SummaryySEAri research seeks to shift the paradigm to accelerate having better knowledge for system and enterprise decision making.
Classic paradigm New paradigm
Our research is motivated by having impact on practice
seari.mit.edu 27
Our research is motivated by having impact on practice…. not just academic thought
2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology