WELCOME [seari.mit.edu]seari.mit.edu/documents/summit/2008/01-SEAriSummit... · view of...
Transcript of WELCOME [seari.mit.edu]seari.mit.edu/documents/summit/2008/01-SEAriSummit... · view of...
WELCOME to the
SEAri Research Summit
October 21, 2008
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 2
Agenda
9:00 Welcome and Introductions Dr. Donna Rhodes, SEAri Director
9:30 SEAri – Overview of the SEAri Research Program Dr. Donna Rhodes, Research Director
10:00 Research Profile: Socio-Technical Decision Making and Designing for Value Robustness
Dr. Adam Ross, Research Scientist
10:45 Break
11:00 Research Report: Designing Systems for Survivability Matt Richards, Doctoral Research Assistant
11:30 Research Report: Real Options in Enterprise Architecture Tsoline Mikaelian, Doctoral Research Assistant
noon LUNCH
1:00 Research Profile – Systems Engineering in the Enterprise Dr. Donna Rhodes, Principal Research Scientist
1:30 Research Report: Leveraging Organizational Culture, Standard Process, and Team Norms to Enable Collaborative Systems Thinking
Caroline Lamb, Doctoral Research Assistant
2:00 Stretch Break
2:10 Research Profile: Systems Engineering Economics Dr. Ricardo Valerdi, Research Associate
2:50 Research Poster Session with Refreshments SEAri Research Assistants
4:15 Participant Feedback and Recommendations for Research SEAri Leadership
4:55 Closing Remarks Dr. Donna Rhodes
5:00 Adjourn
SEAri Research Summit
OVERVIEW of RESEARCH PROGRAM
October 21, 2008
Dr. Donna H. Rhodes
Massachusetts Institute of Technology
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 4
Topics
• About SEAri
• Motivations
• Collaboration
• Research Portfolio
• Transfer to Practice
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 5
About SEAri
and
MIT Engineering Systems Division
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 6
Systems Engineering
Field of Practice
Systems Engineering
Considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.
Engineering
Management SystemsEngineering
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 7
Engineering
Management
Social S
cience
EngineeringSystems
Engineering Systems:
Field of Scholarship
ENGINEERING SYSTEMS
A field of study taking an integrative holistic view of large-scale, complex,
technologically-enabled systems with significant enterprise level
interactions and socio-technical interfaces.
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 8
MIT Engineering Systems Division
Academic Unit for SEAri
Engineering systems is a field of study taking an integrative holistic
view of large-scale, complex technologically enabled systems
with significant enterprise level interactions
and socio-technical interfaces
MIT’s Engineering Systems Division (ESD) is a cross-cutting
academic unit -- engineering, management, and social sciences.
It Broadens engineering practice to include context of challenges
as well as consequences of technological advancement
50 faculty 300 masters students 60 PhD students
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 9
ENGINEERING
SYSTEMS
PoliticalEconomy
Economics,Statistics
Systems Theory
OrganizationalTheory
Operations Research/Systems Analysis
System ArchitectureSystems EngineeringProduct Development
EngineeringManagement
Technology & Policy
Engineering Systems
as a Field of Scholarship
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 10
Engineering Systems Requires
Four Perspectives
1. A very broad interdisciplinary perspective, embracing technology, policy, management science, and social science.
2. An intensified incorporation of system properties (such as sustainability, safety and flexibility) in the design process.
3. Enterprise perspective, acknowledging interconnectedness of product system with enterprise system that develops and sustains it.
4. A complex synthesis of stakeholder perspectives, of which there may be conflicting and competing needs which must be resolved toserve the highest order system (system-of-system) need.
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 11
Systems Engineering Advancement
Research Initiative (SEAri)
Major Sponsors: US Air Force Office of
Scientific Research, MIT Portugal Program,
Singapore Defense Sciences Office, US Air
Force HIS Office, Lean Advancement Initiative,
selected US Government Agencies
(many other engagement partners)
3 Cambridge Center
NE20 – 388/343
Mission
Advance the theories, methods, and effective practice
of systems engineering applied to complex socio-
technical systems through collaborative research
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 12
Motivation
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 13
MOTIVATIONChanging Face of Systems Engineering
TRADITIONAL SE
Transformation of customer requirements to design
Requirements clearly specified, frozen early
Minimizing changes
Design to meet well specified set of requirements
Performance objectives specified at project start
Focus on reliability, maintainability, and availability
ADVANCED SE
Effective transformation of stakeholder needs to fielded (and sustainable) solution
Focus on product families and systems-of-systems
Complex interdependencies of system and enterprise
Importance of systems architecting
Designing for dynamic relevance
Emphasis on expanded set of “ilities” and designing in robustness, flexibility, adaptability in concept phase
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 14
MOTIVATION
Paradigm Shift
Classic paradigm New paradigm
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 15
MOTIVATION
Notional ROI
PRACTICE COMPETENCY CULTURE
Basic training
and processes
Strategy to
establish SE as
competency
Transform culture –
mindset and approaches
Effectiveness
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 16
Collaboration
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 17
Research Predisposed
Toward Application
For most engineering students, the goal of a career in industry
motives their pursuit of advanced study and this will
increasingly be the case on the future. Because of this,
engineering students’ outlook on research is predisposed
toward application in engineering practice
National Academy of Engineering, 2005
Survey of
SEANET
doctoral
students
shows only
25% plan
academic
careers
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 18
Engineering education and research
must be a collaborative endeavor
of government, industry, and academia
Complex engineering research can not take place solely in a laboratory within university walls but rather real world enterprises must be our “learning laboratories”
Expanded view of who an “educator” is --faculty, researchers, practitioners, policy makers, peers
Additionally, we need more cross cutting experiences for educators and practitioners alike
Engineering education and research can not be just a cooperation; must be a true collaboration
SE Research
requires real
world laboratory
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 19
Sponsor Engagement Models
Classical “basic research” sponsors – Targeted topic toward broad scientific goals
Innovation grant sponsors – Higher risk/higher payoff research
Contract research sponsors – Toward solving sponsor problem
Consortium sponsors – Pooled funds for shared research benefits
“Deep engagement” partnerships – Symbiotic relationship
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 20
Research Structure and
Portfolio
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 21
Underlying Research Structure
Prescriptive methods seek to advance state of the practice based on
sound principles and theories, as grounded in real limitations and
constraints
• Normative research: identify principles and theories -- “should be”
• Descriptive research: observe practice and identify limits/constraints
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 22
Research Projects Map to Structure
Theory Development with ModelingMethodology Guidance Document
Codified Successful Practices based on
Empirical Studies
Method Validated in Real World Cases
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 23
Portfolio
RESEARCH PORTFOLIO
1. Socio-Technical Decision Making
2. Designing for Value Robustness
3. Systems Engineering in the Enterprise
4. Systems Engineering Economics
5. Systems Engineering Strategic Guidance
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 24
Research Portfolio (1)
SOCIO-TECHNICAL DECISION MAKING
This area of research is concerned with the context of socio-technical systems. Based on a multi-disciplinary approach, decision making techniques are developed through the exploration of:
– Studies of decision processes and effectiveness of techniques
– Constructs for representing socio-technical systems
– Decision strategies for system of systems
– Visualization of complex trade spaces
– Understanding/mitigating cognitive biases in decision processes
While organizational theorists have well developed theories of how
organizations function and make decisions, this understanding needs to
be integrated into the design phase in a quantifiable way….then it will be
the case that apriori the effect of the enterprise organization on the
engineering system will be predicted rather than being a surprise
Hastings, MIT ESD Symposium, 2004
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 25
Survivability Tradespace
McManus, H., Richards, M., Ross, A., and Hastings, D., “A Framework for Incorporating
“ilities” in Tradespace Studies,” AIAA Space 2007, Long Beach, CA, September 2007.
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 26
Research Portfolio (2)
DESIGNING for VALUE ROBUSTNESS
This area of research seeks to develop methods for concept
exploration, architecting and design using a dynamic perspective
for the purpose of realizing systems, products, and services that
deliver sustained value to stakeholders in a changing world.
– Methods for dynamic multi-attribute trade space exploration
– Architecting principles for designing survivable systems
– Quantification of the changeability of a system design
– Techniques for consideration of unarticulated and latent value
Value robustness is the ability of a system to continue to deliver stakeholder value
in the face of changing contexts and needs. Architecting value robust systems
requires new methods for exploring the concept tradespace, as well as for decision
making. Also needed are architecting principles and strategies, an approach for the
quantification of changeability, and an improved ability for architects and analysts
to classify value for purposes of dialogue and implementation
Ross and Rhodes, 2008
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 27
Linking Tradespaces over Time
Utilit
y (d
imensio
nle
ss)
Time
0 1 2 3 4
1
0
System (Within Epoch) Change Timeline
U
0
Epoch 1 Epoch 2 Epoch 3 Epoch n
…S1,b S1,e S2,b S2,e S3,b S3,e Sn,b Sn,e
T1 T2 T3 Tn
Time
U
0
Epoch 1 Epoch 2 Epoch 3 Epoch n
…S1,b S1,e S2,b S2,e S3,b S3,e Sn,b Sn,e
T1 T2 T3 Tn
Time
System EraPareto Tracing across
Epochs
Rk
ODk
≈Nk
Rk
ODk
≈Nk
Changeability Quantified
as Filtered Outdegree
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 28
Research Portfolio (3)SYSTEMS ENGINEERING in the ENTERPRISE
This research area involves empirical studies and case based researchfor the purpose of understanding how to achieve more effective systems engineering practice in context of the nature of the system being developed, external context, and the characteristics of the associated enterprise.
– Engineering systems thinking in individuals and teams
– Collaborative, distributed systems engineering practices
– Social contexts of enterprise systems engineering
– Alignment of enterprise culture and processes
– Socio-technical systems studies and models
The understanding of the organizational and technical interactions in our
systems, emphatically including the human beings who are a part of them,
is the present-day frontier of both engineering education and practice.
Dr. Michael D. Griffin, Administrator, NASA, 2007 Boeing Lecture, Purdue University
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 29
MIT/MITRE Social Contexts of ESE
(2006 – 2008)
Collaborative Distributed SE (Utter 2007)
Empirical Studies of Systems
Thinking (Davidz 2006)
Collaboration
Situation and
Management
AA BB
Knowledge, Data
and Decision
Management
SE Processes
and Practices
Collaboration
Tool Use
CDSE Social
and Cultural
Environment
CDSE Benefits
and Motivation
Description RecommendationLesson
LearnedIssue or Barrier Success Factor Irrelevant
OR
Tool Training Network Reliability Tool VersionsTool AccessLearning
CurvesClassified Data
Interview Heading Topics
Company
Transcripts
Subtopic
Interviewee Experience
Data Analysis
Example
SE in the Enterprise
Empirical Research
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 30
Research Portfolio (4)
SYSTEMS ENGINEERING ECONOMICS
This research area aims at developing a new paradigm that encompasses an economics view of systems engineering to achieve measurable and predictable outcomes while delivering value to stakeholders.
– Measurement of productivity and quantifying SE ROI
– Advanced methods for reuse, cost modeling, and risk modeling
– Application of real options in systems and enterprises
– Leading indicators for systems engineering effectiveness
In a 2004 Air Force/MIT workshop, Dr. Marvin Sambur, (then) Assistant Secretary
of the AF for Acquisition, noted that the average program is 36% overrun
according to recent studies – disrupting the overall portfolio of programs
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 31
Models, Measures, and Leading
Indicators for Project SuccessThrough Better Execution of Systems Engineering
Cost and schedule modelingProject Risk Assessment
Leading Indicators for Performance Systems Engineering ROI
Person Months Risk
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
55%
60%
65%
70%
75%
80%
85%
90%
95%
100%
0 25000 50000 75000 100000
Person Months
Risk (= Prob. That Actual Person Months
Will Exceed Indicated, X-Axis, Figure)
Person Months Confidence (Cumulative Probability)
0%
5%10%
15%20%25%30%35%40%
45%50%55%60%65%70%
75%80%
85%90%95%
100%
0 25000 50000 75000 100000
Person Months
Cumulative Probability of Person
Months
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 32
Research Portfolio (5)
SYSTEMS ENGINEERING STRATEGIC GUIDANCE
This research area involves synthesis of theory with empirical and case based research for the purpose of developing prescriptive strategic guidance to inform the development of policies and procedures for systems engineering in practice.
– Systems Engineering research guidelines
– Participation in focus groups and pilot-phase reviews
– Position papers on proposed policies
– Recommendations for integrating SE research into curriculum
– Identification of SE research gaps and opportunities
The full impact of systems engineering research can only
be achieved through synthesis of research outcomes
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 33
Guide to SE
Leading IndicatorsJune 2007
Guide to SE Leading Indicators(December 2005)
BETA
AF/DOD
SE Revitalization Policies
AF/LAI Workshop on Systems Engineering
June 2004
SE LI Working Group
With PSM
+
Pilot Programs
(several companies)
Masters Thesis
(1 case study)
Validation Survey
(>100 responses/ one
corporation)
SE LI Working Group
With SEAri and PSM
+
+
V. 1.0
Knowledge
Exchange
Event
Tutorial on SE Leading
Indicators
(many companies)
(1) January 2007
(2) November 2007
Practical Software
& Systems
Measurement
Workshops
(1) July 2005
(2) July 2007
(3) July 2008
Applications
IBM® Rational Method Composer – RUP
Measurement Plug-in
Systems
Engineering
Leading
Indicators
Project
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 34
Transfer to Practice
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 35
Technology Transfer
Publications
• Research Bulletin
• Conference Papers
• Journal Papers (preprint)
• Working Papers (SEAri, ESD, others)
• Conference and Event Briefings
• Theses (with Exec Summaries)
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 36
Publications Highlight
Implications for Practice
Dynamic Multi-Attribute Tradespace Exploration Method
Implications for Systems Engineering Practice1. Ability to explore many design options and prevent too early
focus on single ‘point design’
2. Enables quantitative assessment of factors such as variability
in technical performance and cost, and impacts in markets
3. Suitable to multiple domains and demonstrated to improve
decision making
Vision: designers will have an
enhanced ability to consider concept
alternative in a rigorous way, not only
for present situation but also in
considering futures where needs and
contexts have shifted
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 37
Technology Transfer
Involvement of Students
• Case Studies
• Student Internships
• Student Research Presentations Onsite
• Research Sponsorship Engagement
• Students Hired into Organizations
• SEAri Alumni Network
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 38
Technology Transfer
Professional Education
• SEAri Lectures at MIT
• SEAri Lectures for Sponsors
• SEAri Tutorials (conference, etc.)
• MIT Professional Institute
• SDM Program Certificate Program
• Sponsored Research Deliverables
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 39
Technology Transfer
Strategic Initiatives
• Policy Recommendations
• Innovation Research Initiatives
• Review Teams for Industry Standards and Guides
• Advisory Boards of Selected Organizations
• Leadership of INCOSE Doctoral Network
• Leadership Roles in Professional Societies,
Industry Working Groups, and Study Boards
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 40
Access to
Interim Research Results
SEARI Research Bulletin Published
at End of Each Semester
2008
SEAri Research Summit
October 21
MIT Faculty Club
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 41
SEAri Website
http://seari.mit.edu
Portal to information:
• Publications
• Briefings
• Research Bulletins
• Upcoming Courses
• Other Information
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 42
Indicators of Research
Dissemination and Relevance• 28,000 website visitors in two years
• 22 conference publications last year (30+ year ahead)
– 2008 IEEE Systems Best Paper
– Both Best Paper Awards at INCOSE 2008
• Awarded highly competitive innovation research project from a USgovernment agency
• Multi-national, cross-domain, multi-sector research program
• First MIT Professional Institute Class (13 organizations)
• More than two dozen organizations visited
• First SEAri Summit Held
We seek your input on ways to make research
relevant and available
http://seari.mit.edu © 2008 Massachusetts Institute of Technology 43
Agenda
9:00 Welcome and Introductions Dr. Donna Rhodes, SEAri Director
9:30 SEAri – Overview of the SEAri Research Program Dr. Donna Rhodes, Research Director
10:00 Research Profile: Socio-Technical Decision Making and Designing for Value Robustness
Dr. Adam Ross, Research Scientist
10:45 Break
11:00 Research Report: Designing Systems for Survivability Matt Richards, Doctoral Research Assistant
11:30 Research Report: Real Options in Enterprise Architecture Tsoline Mikaelian, Doctoral Research Assistant
noon LUNCH
1:00 Research Profile – Systems Engineering in the Enterprise Dr. Donna Rhodes, Principal Research Scientist
1:30 Research Report: Leveraging Organizational Culture, Standard Process, and Team Norms to Enable Collaborative Systems Thinking
Caroline Lamb, Doctoral Research Assistant
2:00 Stretch Break
2:10 Research Profile: Systems Engineering Economics Dr. Ricardo Valerdi, Research Associate
2:50 Research Poster Session with Refreshments SEAri Research Assistants
4:15 Participant Feedback and Recommendations for Research SEAri Leadership
4:55 Closing Remarks Dr. Donna Rhodes
5:00 Adjourn