Agile Systems Engineering & Agile at SKA Scale
-
Upload
ska-square-kilometre-array-organisation -
Category
Engineering
-
view
43 -
download
4
Transcript of Agile Systems Engineering & Agile at SKA Scale
Agile Software Engineering & System Engineering at SKA scale Juande Santander-Vela SKA SW Systems Engineer
SQUARE KILOMETRE ARRAYExploring the Universe with the worlds’ largest radio telescope
Astronomical Data AnalysisSoftware & Systems
(on behalf of the SE team)
SKA Key Science Drivers: The history of the Universe
Cosmic Dawn(First Stars and Galaxies)
Galaxy Evolution(Normal Galaxies z~2-3)
Cosmology(Dark Energy, Large Scale Structure)Cosmic Magnetism
(Origin, Evolution)
Cradle of Life(Planets, Molecules, SETI)
Testing General Relativity(Strong Regime, Gravitational Waves)
Exploration of the Unknown
Broadest range of science of any facility, worldwide
SKA Observatory Vision
Exploring the Universe with the worlds’ largest radio telescope
3 sites 2 telescopes 1 observatoryDesign Phase:
~ €200M; 600 scientists+engineers, 80% complete
SKA Phase 1 (SKA1) Construction: 2019 – 2025
Construction cost cap: €674.1M (2016 inflation-adjusted)
Operations cost: (estimate) €89M/yrMeerKAT integrated
Observatory Development Programme (€20M/year planned)
SKA Regional centres out of scope of centrally-funded SKAO.
SKA Phase 2: start mid-2020s ~2000 dishes across 3500km of
Southern AfricaMajor expansion of SKA1-Low
across Western Australia
>50 years lifetime! Drives need for reliability,
and adaptability
SKA Observatory Vision
Exploring the Universe with the worlds’ largest radio telescope
3 sites 2 telescopes 1 observatoryDesign Phase:
~ €200M; 600 scientists+engineers, 80% complete
SKA Phase 1 (SKA1) Construction: 2019 – 2025
Construction cost cap: €674.1M (2016 inflation-adjusted)
Operations cost: (estimate) €89M/yrMeerKAT integrated
Observatory Development Programme (€20M/year planned)
SKA Regional centres out of scope of centrally-funded SKAO.
SKA Phase 2: start mid-2020s ~2000 dishes across 3500km of
Southern AfricaMajor expansion of SKA1-Low
across Western Australia
>50 years lifetime! Drives need for reliability,
and adaptability
SKA Organisation
Exploring the Universe with the worlds’ largest radio telescope
! Australia (DoI&S)" Canada (NRC-HIA)# China (MOST)$ India (DAE)% Italy (INAF)& Netherlands (NWO)' New Zealand (MED)( South Africa (DST)) Sweden (Chalmers)* UK (BEIS/STFC)
In discussion with: + Germany , France - Portugal . Spain / Switzerland 0 Japan 1 South Korea
In the process of becoming an Inter-
Governmental Organisation
SKA1 Design Consortia
Exploring the Universe with the worlds’ largest radio telescope
SKA1 Design Consortia
Exploring the Universe with the worlds’ largest radio telescope
Software
Heavy
Elements
Have
~6-10 × FTE
wrt SKAO
SKA1 Design Consortia
Exploring the Universe with the worlds’ largest radio telescope
1000 km Map data ©2017 Terms of Use
Filter ConsortiaAssembly, Integration & Verification
Central Signal Processor
Dish
Infrastructure Australia
Infrastructure South Africa
Low-Frequency Aperture Array
Mid-Frequency Aperture Array
Science Data Processor
Signal and Data Transport
Telescope Manager
Wideband Single Pixel Feeds
Check/uncheck all
GMT-7
GMT+12
Exploring the Universe with the worlds’ largest radio telescope
19 hour timezone span
SKA1-Lowlocations
1600km
N
➤➤
NDataSIO,NOAA,U.S.Navy,NGA,GEBCODataSIO,NOAA,U.S.Navy,NGA,GEBCO
ImageLandsat/CopernicusImageLandsat/Copernicus
SKA1-Midlocations
500km
N
➤➤
NImageLandsat/CopernicusImageLandsat/Copernicus
DataSIO,NOAA,U.S.Navy,NGA,GEBCODataSIO,NOAA,U.S.Navy,NGA,GEBCO
SKA1 Sites
Exploring the Universe with the worlds’ largest radio telescope
SKA1-LOW: 50 – 350 MHz Phase 1: ~130,000 antennas across 65km
SKA1-Mid: 350 MHz – 24 GHz Phase 1: 200 15-m dishesacross 150 km
SKA1 Telescopes
Exploring the Universe with the worlds’ largest radio telescope
SKA1-LOW: 50 – 350 MHz Phase 1: ~130,000 antennas across 65km
SKA1-Mid: 350 MHz – 24 GHz Phase 1: 200 15-m dishes
across 150 km
SKA1 Notional Data Flow
Exploring the Universe with the worlds’ largest radio telescope
2 Pb/s 7.2 Tb/s
7.2 Tb/s
8.8 Tb/s8.8 Tb/s
~50 PFLOPS
~5 Tb/s
~250 PFLOPS
Long Term Storage
300 PB/year
LFAA Data Network Digital Data Backhaul
Digital Data Backhaul
CSP-SDP Network
SKA1 Notional Data Flow
Exploring the Universe with the worlds’ largest radio telescope
2 Pb/s 7.2 Tb/s
7.2 Tb/s
8.8 Tb/s8.8 Tb/s
~50 PFLOPS
~5 Tb/s
~250 PFLOPS
Long Term Storage
300 PB/year
LFAA Data Network Digital Data Backhaul
Digital Data Backhaul
CSP-SDP Network
SKA1 Notional Data Flow
Exploring the Universe with the worlds’ largest radio telescope
2 Pb/s 7.2 Tb/s
7.2 Tb/s
8.8 Tb/s8.8 Tb/s
~50 PFLOPS
~5 Tb/s
~250 PFLOPS
Long Term Storage
300 PB/year
LFAA Data Network Digital Data Backhaul
Digital Data Backhaul
CSP-SDP Network
SKA1 Notional Data Flow
Exploring the Universe with the worlds’ largest radio telescope
2 Pb/s 7.2 Tb/s
7.2 Tb/s
8.8 Tb/s8.8 Tb/s
~50 PFLOPS
~5 Tb/s
~250 PFLOPS
Long Term Storage
300 PB/year
LFAA Data Network Digital Data Backhaul
Digital Data Backhaul
CSP-SDP Network
Similar to LOFAR
in flexibility.
See O4.2 by
Adriaan Renting
How to develop so much software?
Exploring the Universe with the worlds’ largest radio telescope
Agile
Exploring the Universe with the worlds’ largest radio telescope
Agile, Scaled Agile
Exploring the Universe with the worlds’ largest radio telescope
(Framework)
SAFe for Lean Enterprises Large Solution SAFe®
Exploring the Universe with the worlds’ largest radio telescope
Agile Teams, 5-7 people (including Scrum Master), ±2 people
Agile Release Trains (ARTs) in Definition
https://scaledagileframework.com/
Agile Manifesto
• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan
Exploring the Universe with the worlds’ largest radio telescope
http://agilemanifesto.org
The emphasis is on Quality
Exploring the Universe with the worlds’ largest radio telescope
Nick Rees talk I10.1 (Thursday 26th, 9:00 AM)
What about Systems Engineering?
Exploring the Universe with the worlds’ largest radio telescope
INCOSE, 2004
“Systems engineering [SE] is an interdisciplinary approach and means to enable the realisation of successful systems […] [SE] integrates all the disciplines and specialty groups into a team effort forming a structured development process […] from concept to production to operation. [SE] considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.”
Exploring the Universe with the worlds’ largest radio telescope
INCOSE, 2004
“Systems engineering [SE] is an interdisciplinary approach and means to enable the realisation of successful systems […] [SE] integrates all the disciplines and specialty groups into a team effort forming a structured development process […] from concept to production to operation. [SE] considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.”
Exploring the Universe with the worlds’ largest radio telescope
INCOSE, 2004
“Systems engineering [SE] is an interdisciplinary approach and means to enable the realisation of successful systems […] [SE] integrates all the disciplines and specialty groups into a team effort forming a structured development process […] from concept to production to operation. [SE] considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.”
Exploring the Universe with the worlds’ largest radio telescope
INCOSE, 2004
“Systems engineering [SE] is an interdisciplinary approach and means to enable the realisation of successful systems […] [SE] integrates all the disciplines and specialty groups into a team effort forming a structured development process […] from concept to production to operation. [SE] considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.”
Exploring the Universe with the worlds’ largest radio telescope
INCOSE, 2004
“Systems engineering [SE] is an interdisciplinary approach and means to enable the realisation of successful systems […] [SE] integrates all the disciplines and specialty groups into a team effort forming a structured development process […] from concept to production to operation. [SE] considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.”
Exploring the Universe with the worlds’ largest radio telescope
David Long, CEO Vitech Corp
“The role of Systems Engineers is to encode and translate engineering to all stakeholders.”
Exploring the Universe with the worlds’ largest radio telescope
Nick Rees, SKAO Head of Computing
“Systems Engineers capture and remind us of all the bullshit stakeholders say about
the system.”
Exploring the Universe with the worlds’ largest radio telescope
Exploring the Universe with the worlds’ largest radio telescope
Exploring the Universe with the worlds’ largest radio telescope
Exploring the Universe with the worlds’ largest radio telescope
Juande Santander-Vela, SKAO Software SE
“Systems Engineers have to provide useful engineering artefacts to stakeholders, notice still missing definition, and supply it ASAP”
Exploring the Universe with the worlds’ largest radio telescope
Freshly coined for
#ADASS2017CL!
Juande Santander-Vela, SKAO Software SE
“Systems Engineers have to provide useful engineering artefacts to stakeholders, notice still missing definition, and supply it ASAP”
Exploring the Universe with the worlds’ largest radio telescope
Or assess and record the risk of
not doing it
Freshly coined for
#ADASS2017CL!
INCOSE, 2004
“Systems engineering [SE] is an interdisciplinary approach and means to enable the realisation of successful systems […] [SE] integrates all the disciplines and specialty groups into a team effort forming a structured development process […] from concept to production to operation. [SE] considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.”
Exploring the Universe with the worlds’ largest radio telescope
With an assessment of what are the risks against
quality and user needs
SE must be Agile, or it provides diminished value
Exploring the Universe with the worlds’ largest radio telescope
Agile Manifesto
• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan
Exploring the Universe with the worlds’ largest radio telescope
http://agilemanifesto.org
Agile SE
• Keep communication with all stakeholders involved in the project, with speedy and clear dissemination of agreements
• Continuous integration of the engineering artefacts… and code, preferably self-documented
• Get devs (and SEs) that are invested, not just contracted → Gain‑share frameworks
• Have systems & processes that accommodate system (and code) change and prove requirements are satisfied → don’t do document-centric requirements management
Exploring the Universe with the worlds’ largest radio telescope
Agile SE
• Keep communication with all stakeholders involved in the project, with speedy and clear dissemination of agreements
• Continuous integration of the engineering artefacts… and code, preferably self-documented
• Get devs (and SEs) that are invested, not just contracted → Gain‑share frameworks
• Have systems & processes that accommodate system (and code) change and prove requirements are satisfied → don’t do document-centric requirements management
Exploring the Universe with the worlds’ largest radio telescope
At all phases! Including design,
construction, commissioning, and
operations…
Don’t disband SE!
Agile SE
• Keep communication with all stakeholders involved in the project, with speedy and clear dissemination of agreements
• Continuous integration of the engineering artefacts… and code, preferably self-documented
• Get devs (and SEs) that are invested, not just contracted → Gain‑share frameworks
• Have systems & processes that accommodate system (and code) change and prove requirements are satisfied → don’t do document-centric requirements management
Exploring the Universe with the worlds’ largest radio telescope
At all phases! Including design,
construction, commissioning, and
operations…
Don’t disband SE!
Agile SE
• Keep communication with all stakeholders involved in the project, with speedy and clear dissemination of agreements
• Continuous integration of the engineering artefacts… and code, preferably self-documented
• Get devs (and SEs) that are invested, not just contracted → Gain‑share frameworks
• Have systems & processes that accommodate system (and code) change and prove requirements are satisfied → don’t do document-centric requirements management
Exploring the Universe with the worlds’ largest radio telescope
At all phases! Including design,
construction, commissioning, and
operations…
Don’t disband SE!
Agile SE
• Keep communication with all stakeholders involved in the project, with speedy and clear dissemination of agreements
• Continuous integration of the engineering artefacts… and code, preferably self-documented
• Get devs (and SEs) that are invested, not just contracted → Gain‑share frameworks
• Have systems & processes that accommodate system (and code) change and prove requirements are satisfied → don’t do document-centric requirements management
Exploring the Universe with the worlds’ largest radio telescope
Partners, not
contractors/suppliers
mentioned by
JC Guzman on I2.1
At all phases! Including design,
construction, commissioning, and
operations…
Don’t disband SE!
Agile SE
• Keep communication with all stakeholders involved in the project, with speedy and clear dissemination of agreements
• Continuous integration of the engineering artefacts… and code, preferably self-documented
• Get devs (and SEs) that are invested, not just contracted → Gain‑share frameworks
• Have systems & processes that accommodate system (and code) change and prove requirements are satisfied → don’t do document-centric requirements management
Exploring the Universe with the worlds’ largest radio telescope
Partners, not
contractors/suppliers
mentioned by
JC Guzman on I2.1
At all phases! Including design,
construction, commissioning, and
operations…
Don’t disband SE!
Agile-ish Practices at SKAO• SE planning• We have a full plan, from which the backlog is constructed• Progress reviewed every 2 weeks, with forward look to another 2 weeks
• Systems Modelling• Integrated System Model for SKA1_Common and SKA1_Low in Cameo SysML• Second model for SKA1_Mid in Vitech CORE• Used for ICDs, discussions
• SKA1_Mid and SKA1_Low Block Diagrams• Used for reference, updated in agile way, but low priority
Exploring the Universe with the worlds’ largest radio telescope
Agile-ish Practices at SKA1
• Consortia work-planning• Agile for SDP, TM• LFAA uses agile for software development of AAVS1 prototype• Others want to embrace SAFe for construction, just not now
• Multimodal communication• Email, SMS, Confluence, JIRA, Jama comments, Slack, Vidyo, Skype…
Exploring the Universe with the worlds’ largest radio telescope
Not so Agile-ish Practices at SKAO
• Engineering Change Process (ECPs) and Change Notes (CNs)• Bug, or Feature? Still TBD for ECPs → more time to reflect• Need for formality/signature/buy-in across consortia requires time; we need to
be able to move from as agreed to as designed before as signed… but how?
• Lack of JIRA/backlog culture → starting to improve
• Lack of self-awareness of performance metrics → starting to improve
Exploring the Universe with the worlds’ largest radio telescope
Agile SE is not for free…
• Coordination/communication takes time!
• Self-organisation frees time from management, but needs assistance from monitoring → back to coordination!
• Automation of monitoring and communication can help… but still the effectiveness of the communication needs to be monitored.
Exploring the Universe with the worlds’ largest radio telescope
a lot of
But it makes it easier to
focus on the right things
Conclusions
• Agile SE can be seen as a tailoring of SE; you still need to identify which are the processes that provide value to your system/organisation.
• It is a challenge to create teams that can work together, but making it on a cadence makes it easier to achieve it… through repetition.
• We have to improve a lot at SKAO… but we’re working on it.
• Agile for Agile’s sake is a fad; you become Agile it to build quality, and respond to mistakes, sooner.
Exploring the Universe with the worlds’ largest radio telescope
Thanks!
Exploring the Universe with the worlds’ largest radio telescope
SQUARE KILOMETRE ARRAY
Questions?
Exploring the Universe with the worlds’ largest radio telescope
SQUARE KILOMETRE ARRAY