Automated Software Engineering
-
Upload
cs-ncstate -
Category
Education
-
view
486 -
download
0
description
Transcript of Automated Software Engineering
Career advice
Learn Modeling
(it’s the next big thing in SE)
2
This kind of modeling? No
3
This kind of modeling? Nope
4
(though it does look pretty cool, heh?)
Executable Software Models
5
Software models: are everywhere
• If you call an ambulance in London or New York,
– those ambulances are controlled by emergency response models.
• If you cross the border Arizona to Mexico,
– A models determines if you are taken away for extra security measures.
• If you default on your car loans,
– A model determines when (or if) someone to repossess your car.
• If the stock market crashes,
– it might be that some model caused the crash.
6
• Karplus and Levitt – 2013 Nobel prize in chemistry– development of multi-scale
models for complex chemical systems
– Explored complex chemical reactions (e.g. split-second changes of photosynthesis).
7
• Models are now a central tool in scientific research.
– in physics, biology and other fields of science
– complex simulations using supercomputers.
• E.g. genomic map required analyzing 80 trillion bytes
• E.g.. Other computational modeling projects– the rise and fall of native
cultures, – subnuclear particles – the Big Bang.
Software models: are everywhere
Question: How to best reason
about models?
Answer: automated software engineering
8
The Human Condition:a balancing act between goals
• The human condition – a constant balancing act between vaguely understood, often
competing, goals.
• For example: – Lets build the software faster...
– ... with few bugs ...
– ... using less budget.
• For the consequences of "better, faster, cheaper” :– NASA's loss of the Mars Polar
Orbiter
– $300 million , wasted9
Example
• Choices explored by satellite designers at NASA's Jet Propulsion Laboratory.
– Dozens of experts in propulsion,communications,
guidance and control etc– Week-long sessions – Thrash out possibilities
for NASA's next deep space mission.
• Groups sitting “here” can make decisions that impact “there”, and they just do not realize.
10
Keeping track of all those decisions:Lightweight requirements
notations• E.g. Martin Feather’s DDP
language– Mitigations …– … that retire risks….– …. That damage
requirements
• Multi-objective optimization– Find the cheapest
mitigations that enable the most requirements
11
Keeping track of all those decisions:Lightweight requirements
notations• My optimizers out-performing
state of the art tools
– Faster
– Simpler policies
12
Data Mining for Very Busy People, Tim Menzies, Ying Hu, IEEE Computer, October 2003
Keeping track of all those decisions:Lightweight requirements
notations• E.g. John
Mylopoulos‘soft goals
• Competingrequirements
• Simulations toexplore allthe what-ifs
13Chiang, Eliza, Menzies, Tim, Simulations for very early lifecycle quality evaluations, Software Process: Improvement and Practice 7(3-4), 2003,
• E.g. Kang’s product lines[Kang’90]
• Add in known constraints – E.g. “if we use a camera
then we need a high resolution screen”.
• Extract products – Find subsets of the
product lines that satisfy constraints.
– If no constraints, linear time
– Otherwise, can defeat state-of-the-art optimizers [Pohl et at, ASE’11] [Sayyad, Menzies ICSE’13].
Cross-TreeConstraints
More lightweight requirements notations
14
A.S. Sayyad, J. Ingram, T. Menzies, and H. Ammar: Scalable Product Line Configuration: A Straw to Break the Camel's Back. In the International Conference on Automated Software Engineering (ASE’13). Palo Alto, USA. November 2013.
Automated software engineering and optimization
• Many SE activities are like optimization problems [Harman,Jones’01].
• Due to computational complexity, exact optimization methods can be impractical for large SBSE problems
• So researchers and practitioners use metaheuristic search to find near optimal or good-enough solutions.– E.g. simulated annealing [Rosenbluth et al.’53]
– E.g. genetic algorithms [Goldberg’79]
– E.g. tabu search [Glover86]
15
• Repeat till happy or exhausted– Selection (cull the herd)
– Cross-over (the rude bit)
– Mutation (stochastic jiggle)
Optimization andevolutionary algorithms
12
3
5
4
6
7
8
9
Pareto frontier-- better on some
criteria, worse on noneSelection:
-- generation[i+1] comes from Pareto frontier of generation[i]
16
Long tradition of software, simulation, and optimization
• Not one “best” solutions– Its all trade-offs– Satisficing = satisfy + sacrifice
• Problems have to be explored via computersimulations
– Try it out and see– And approach pioneered
by John Von Neumann,in the 1950s
17
Explosive growth of SE + optimization
Q: Why?
A: Thanks to Big Data, more access to more cpu.18
Applications of Optimization in SE1. Requirements Menzies, Feather, Bagnall, Mansouri, Zhang
2. Transformation Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams
3.Effort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd
4. Management Alba, Antoniol, Chicano, Di Pentam Greer, Ruhe
5. Heap allocation Cohen, Kooi, Srisa-an
6. Regression test Li, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer
7. SOA Canfora, Di Penta, Esposito, Villani
8. Refactoring Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt
9. Test Generation Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen, Gutjahr, Harrold, Holcombe, Jones,
Korel, Pargass, Reformat, Roper, McMinn, Michael, Sthamer, Tracy, Tonella,Xanthakis, Xiao,
Wegener, Wilkins
10. Maintenance Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell, Swift
11. Model checking Alba, Chicano, Godefroid
12. Probing Cohen, Elbaum
13. UIOs Derderian, Guo, Hierons
14. Comprehension Gold, Li, Mahdavi
15. Protocols Alba, Clark, Jacob, Troya
16. Component sel Baker, Skaliotis, Steinhofel, Yoo
17. Agent Oriented Haas, Peysakhov, Sinclair, Shami, Mancoridis 19
Example 1(of four)
Automated program repair
20
Multi-objective Optimization, in the 21st century, automated repair
A Systematic Study Of Automated Program Repair: Fixing 55 Out Of 105 Bugs For $8 Each : Claire Le Goues ; Michael Dewey-vogt ;Stephanie Forrest ; Westley Weimer, ICSE’12
21
A Systematic Study Of Automated Program Repair: Fixing 55 Out Of 105 Bugs For $8 Each : Claire Le Goues ; Michael Dewey-vogt ;Stephanie Forrest ; Westley Weimer, ICSE’12
22
A Systematic Study Of Automated Program Repair: Fixing 55 Out Of 105 Bugs For $8 Each : Claire Le Goues ; Michael Dewey-vogt ;Stephanie Forrest ; Westley Weimer, ICSE’12
23
•
A Systematic Study Of Automated Program Repair: Fixing 55 Out Of 105 Bugs For $8 Each : Claire Le Goues ; Michael Dewey-vogt ;Stephanie Forrest ; Westley Weimer, ICSE’12
24
Example 2(of four)
Software Product Lines
25
• E.g. Kang’s product lines[Kang’90]
• Add in known constraints – E.g. “if we use a camera
then we need a high resolution screen”.
• Extract products – Find subsets of the
product lines that satisfy constraints.
– If no constraints, linear time
– Otherwise, can defeat state-of-the-art optimizers [Pohl et at, ASE’11] [Sayyad, Menzies ICSE’13].
Cross-TreeConstraints
More lightweight requirements notations
26
A.S. Sayyad, J. Ingram, T. Menzies, and H. Ammar: Scalable Product Line Configuration: A Straw to Break the Camel's Back. In the International Conference on Automated Software Engineering (ASE’13). Palo Alto, USA. November 2013.
Problem: many competing goals2 or 3 or 4 or 5 goals
Software engineering = navigating the user goals:1. Satisfy the most domain constraints (0 ≤ #violations ≤ 100%)2. Offers most features3. Build “stuff” In least time4. That we have used most before5. Using features with least known defects
27
A.S. Sayyad, J. Ingram, T. Menzies, and H. Ammar: Scalable Product Line Configuration: A Straw to Break the Camel's Back. In the International Conference on Automated Software Engineering (ASE’13). Palo Alto, USA. November 2013.
Issues of scale up• This model: 10 features, 8 rules
• [www.splot-research.org]: ESHOP: 290 Features, 421 Rules
• LINUX kernel variability projectLINUX x86 kernel6,888 Features; 344,000 Rules
Cross-Tree Constraints
28
A.S. Sayyad, J. Ingram, T. Menzies, and H. Ammar: Scalable Product Line Configuration: A Straw to Break the Camel's Back. In the International Conference on Automated Software Engineering (ASE’13). Palo Alto, USA. November 2013.
State of the Art (as of Nov’13)Features
9
290
544
6888
SPLO
TLi
nu
x (L
VA
T)
Pohl ‘11 Lopez-Herrejon
‘11
Henard‘12
Sayyad,Menzies’
13a
Velazco ‘13
Sayyad, Menzies’13b
Johansen ‘11
Benavides ‘05
White ‘07, ‘08, 09a, 09b, Shi ‘10, Guo ‘11
Objectives
Multi-goalSingle-goal
300,000+
clauses
29A.S. Sayyad, J. Ingram, T. Menzies, and H. Ammar: Scalable Product Line Configuration: A Straw to Break the Camel's Back. In the International Conference on Automated Software Engineering (ASE’13). Palo Alto, USA. November 2013.
Example 3(of four)
Space ships!
30
Skip re-entry• My optimizers vs state of the art numeric optimizers
• My tools: ran 40 times faster
– Generated better solutions
• Powerful succinct explanation tool
31Automatically Finding the Control Variables for Complex System Behavior Gregory Gay, Tim Menzies, Misty Davies, and Karen Gundy-Burlet Journal - Automated Software Engineering, 2010 [PDF]
Example 4 (of four)
Avionics Requirements Engineering
32
33Better Model-Based Analysis of Human Factors for Safe Aircraft Approach, Joseph Krall, Tim Menzies, Misty Davies, IEEE Trans Human Factors, to appear
WMC: GIT’s Work Models that Compute [Kim’11]
• Cognitive models of the agents (both pilots and computers) – Late descent,
– Unpredicted rerouting,
– Different tailwind conditions
• Goal: validate operations procedures (are they safe?)
• NASA’s analysts want to explore 7000 scenarios.– With current tools (NSGA-II)
– 300 weeks to complete
• Limited access to hardware– Queue of researchers wanting
hardware access
– Hardware pulled away if in-flight incidents for manned space missions
Asiana AirlinesFlight 214
34Better Model-Based Analysis of Human Factors for Safe Aircraft Approach, Joseph Krall, Tim Menzies, Misty Davies, IEEE Trans Human Factors, to appear
Active Learning: Skip similar examples,focus on the most different
4 mins (GALE) vs 7 hours (rest)
Better Model-Based Analysis of Human Factors for Safe Aircraft Approach, Joseph Krall, Tim Menzies, Misty Davies, IEEE Trans Human Factors, to appear
Career advice
Learn Modeling
(it’s the next big thing in SE)36
Question: How to reason about models?
Answer: automated software engineering
Cs791, fall 2015
37