Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first...
Transcript of Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first...
![Page 1: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/1.jpg)
Hypotheses Engineering: first essential steps of experiment-driven software development
Jorge Melegati, Xiaofeng Wang, Pekka Abrahamsson
RCoSE/DDrEE 2019Joint 5th International Workshop on Rapid Continuous Software Engineering and
1st International Workshop on Data-Driven Decisions, Experimentation and EvolutionMontreal, Canada
![Page 2: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/2.jpg)
Goals
u Propose Hypotheses Engineering
u A discipline to better handle hypotheses in experiment-driven software development
u Present a set of research questions to develop practices for this discipline
![Page 3: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/3.jpg)
Experiment-driven software development
u The use of experiments to build features that the user really wants
u Experiments: testing product assumptions applying scientific methods with the purpose of supporting or refuting these assumptions
u Bosch et al. (2018) identified three approach to software development
u Requirement-driven development
u Outcome/data-driven development (experiment-driven)
u AI driven development
![Page 4: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/4.jpg)
Requirement-driven development
u Requirements Engineering is an important component of the process
u Concerns the identification and documentation of stakeholders needs
u Even in agile methodologies, it is still present
u Done more often and with closer and more frequent contact with the customer
u User stories are an artifact for requirements
![Page 5: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/5.jpg)
Experiment-driven development models
HYPEX (Olsson and Bosch, 2014) RIGHT (Fagerholm et al., 2017)
![Page 6: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/6.jpg)
Experiment-driven development models
u All were influenced by the Build-Measure-Learn loop from the Lean Startup methodology (Ries, 2011)
u A common denominator:
![Page 7: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/7.jpg)
Requirement vs Experiment-driven software development
Identify, specify, andprioritize hypotheses
Design theexperiment Execute Analyze
RequirementsEngineering activities Design the software Code Test
Learning
![Page 8: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/8.jpg)
Hypotheses Engineering
u A discipline to handle hypotheses in a similar way requirements were handled in requirements-driven software development
u The team should be able to:
u Identify hypotheses
u Analyze them regarding meaningfulness and duplication
u Prioritize them in order to minimize waste of time and resources
u Communicate hypotheses to the development team
![Page 9: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/9.jpg)
Hypotheses Engineering
u A discipline to handle hypotheses in a similar way requirements were handled in requirements-driven software development
Generation PrioritizationAnalysis
Hypothesis Engineering
DocumentationLearning fromother experiment
results
Business visionand goals
![Page 10: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/10.jpg)
Hypotheses generation
u Discussed briefly in the literature so far
u HYPEX: first practice is feature backlog generation
u Product management and product development staff “based on their understanding of customer needs and strategic business goals” generate features that may bring value to customers (Olsson and Bosch, 2014)
u RIGHT
u “analysis and product owner work with a data scientist role […] to communicate the assumptions of the roadmap and map the areas of uncertainty which need to be tested” (Fagerholm et al., 2017)
u Common theme: business goals and vision to determine which assumptions have to be experimented
![Page 11: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/11.jpg)
Hypotheses generation
RQ1: How can software development teams systematically define hypotheses based on business goals and vision, and own previously accumulated learning?
![Page 12: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/12.jpg)
Hypotheses documentation
u In traditional requirements—driven development is really important
u Agile
u There is not formal documentation
u Long requirements documents are replaced by user stories
u HYPEX: the team should specify how the feature adds value to the customer and support
u RIGHT: experimentation plans and learning as information artifacts not necessarily formally documented
![Page 13: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/13.jpg)
Hypotheses documentation
RQ2: What artifact could be useful to represent hypotheses and support experiments creation?
RQ3: How could a hypothesis artifact be used to keep experiment useful information?
![Page 14: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/14.jpg)
Hypotheses analysis
u Concerns similar to requirements:
u Hypotheses not well-explained
u Not possible to perform an experiment
u Consistent and not-duplicated
![Page 15: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/15.jpg)
Hypotheses analysis
RQ4: How could teams understand if a hypothesis can be practically tested using an experiment?
RQ5: How could teams understand dependencies among different hypotheses?
RQ6: How do hypotheses evolve over the time?
![Page 16: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/16.jpg)
Hypotheses prioritization
u A common theme among scientific and industry authors
u In startups, it is critical to test the most crucial hypotheses first
u Several techniques:
u Leap-of-faith assumptions (Ries, 2011)
u Prioritization matrix (Gothelf and Seiden, 2013)
u No work has evaluated these techniques
![Page 17: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/17.jpg)
Hypotheses prioritization
RQ7: Are current assumption prioritization techniques effective?
RQ8: Could requirements prioritization techniques be adapted to hypotheses in experiment-driven development?
![Page 18: Hypotheses Engineering: first essential steps of experiment- · Hypotheses Engineering: first essential steps of experiment-driven software development Jorge Melegati, XiaofengWang,](https://reader030.fdocuments.in/reader030/viewer/2022041106/5f07895a7e708231d41d7888/html5/thumbnails/18.jpg)
Conclusions
u In this position paper, we proposed a Hypothesis Engineering discipline
u Tailored to experiment-driven software development
u Contrast to Requirements Engineering
u Notes:
u The steps does not mean that it will follow a linear approach
u Maybe there are different techniques for different stages of product development