20041221 gui testing survey
description
Transcript of 20041221 gui testing survey
![Page 1: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/1.jpg)
GUI Testing GUI Testing ApproachesApproaches
Will ShenWill Shen
2004/12/212004/12/21
![Page 2: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/2.jpg)
ReferencesReferences
Hierarchical GUI Test Case Hierarchical GUI Test Case Generation Using Automated Generation Using Automated PlanningPlanning Atif M. MemonAtif M. Memon, Student Member, IEEE, , Student Member, IEEE,
Martha E. PollackMartha E. Pollack, and , and Mary Lou SoffaMary Lou Soffa, , Member, IEEEMember, IEEE
Plan Generation for GUI TestingPlan Generation for GUI Testing Atif M. MemonAtif M. Memon and and Martha E. PollackMartha E. Pollack and and
Mary Lou SoffaMary Lou Soffa Dept. of Computer Science Dept. of Computer Science University of Pittsburgh Pittsburgh, PA 15260 University of Pittsburgh Pittsburgh, PA 15260 USA fatif, pollack, [email protected] USA fatif, pollack, [email protected]
![Page 3: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/3.jpg)
Coverage Criteria for GUI Testing Atif M. Memon Dept. of Computer
Science University of Pittsburgh Pittsburgh, PA 15260 [email protected]
Mary Lou Soffa Dept. of Computer Science University of Pittsburgh Pittsburgh, PA 15260 [email protected]
Martha E. Pollack Dept. of Electrical Engineering and Computer Science, University of Michigan Ann Arbor, MI [email protected]
![Page 4: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/4.jpg)
OutlineOutline
1.1. IntroductionIntroduction
2.2. Overview of PATHSOverview of PATHS
3.3. Coverage Criteria for GUI TestingCoverage Criteria for GUI Testing
4.4. ConclusionsConclusions
![Page 5: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/5.jpg)
1. Introduction1. Introduction
Testing GUI is difficult:Testing GUI is difficult: The space of possible interactions with a The space of possible interactions with a
GUI is enormous.GUI is enormous. In that each sequence of GUI commands can In that each sequence of GUI commands can
result in a different stateresult in a different state A GUI command may need to be evaluatedion A GUI command may need to be evaluatedion
all of these states.all of these states. Determining the coverage of a set of test Determining the coverage of a set of test
casescases.. No only how much the code is tested, but in No only how much the code is tested, but in
how many different possible states of the how many different possible states of the software each piece of code is tested.software each piece of code is tested.
![Page 6: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/6.jpg)
An important aspect of GUI testing is An important aspect of GUI testing is verificationverification of its state at each step of of its state at each step of test case execution.test case execution. The execution of the test case must be The execution of the test case must be
terminated as soon as an error is detected.terminated as soon as an error is detected. Regression testing presents special Regression testing presents special
challenges for GUIs.challenges for GUIs. The input-output mapping does not remain The input-output mapping does not remain
constant across successive versions of the constant across successive versions of the software.software.
![Page 7: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/7.jpg)
2. Overview of PATHS2. Overview of PATHS
PPlanning lanning AAssited ssited TTester for grapester for grapHHical ical user interface user interface SSystems.ystems.
A new approach to automatic testing of A new approach to automatic testing of GUIs that builds on AI planning GUIs that builds on AI planning techniques.techniques.
Given a specification of initial and goal Given a specification of initial and goal states for a GUI, a planner is used to states for a GUI, a planner is used to generate sequences of GUI actions that generate sequences of GUI actions that lead from the initial state to the goal lead from the initial state to the goal state.state.
![Page 8: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/8.jpg)
2.1 The Example GUI2.1 The Example GUI
Microsoft Microsoft WordPadWordPad
![Page 9: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/9.jpg)
(a) the Initial States
(b) The Goal State
![Page 10: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/10.jpg)
2.2 Test case generation 2.2 Test case generation processprocess
Two phasesTwo phases Setup phaseSetup phase
1.1. PATHS creates a hierarchical model of the PATHS creates a hierarchical model of the GUI and returns a list of operators from GUI and returns a list of operators from the model to the test designerthe model to the test designer
2.2. By using knowledge of the GUI, the test By using knowledge of the GUI, the test designer then defines the preconditions designer then defines the preconditions and effects of the operators in a simple and effects of the operators in a simple language provided by the planning system.language provided by the planning system.
![Page 11: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/11.jpg)
Plan-generationPlan-generation1.1. The test designer describes scenarios The test designer describes scenarios
(tasks) by defining a set of initial and goal (tasks) by defining a set of initial and goal states for test case generation.states for test case generation.
2.2. PATHS generates a test suite for the PATHS generates a test suite for the scenarios.scenarios.
![Page 12: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/12.jpg)
![Page 13: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/13.jpg)
2.3 Deriving GUI 2.3 Deriving GUI operatorsoperators
2.3.1 Events2.3.1 Events 2.3.2 Operators2.3.2 Operators 2.3.3 Operator-Event mapping2.3.3 Operator-Event mapping 2.3.4 The Step1 of example GUI2.3.4 The Step1 of example GUI 2.3.5 The Step2 of EDIT_CUT2.3.5 The Step2 of EDIT_CUT 2.3.6 The Step3 initial state and goal 2.3.6 The Step3 initial state and goal
statestate 2.3.7 The Step4 generate test case2.3.7 The Step4 generate test case
![Page 14: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/14.jpg)
2.3.1 Events2.3.1 Events
Three classes of GUI eventThree classes of GUI event Unrestricted-focus eventsUnrestricted-focus events open GUI open GUI
windows that do not restrict the userwindows that do not restrict the user’’s focuss focus Restricted-focus eventsRestricted-focus events open GUI windows open GUI windows
that have the special property that once that have the special property that once invoked, they monopolize the GUI invoked, they monopolize the GUI interaction.interaction.
System-interaction eventsSystem-interaction events interact with the interact with the underlying software to perform some action.underlying software to perform some action.
![Page 15: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/15.jpg)
2.3.2 Operators2.3.2 Operators
The setup phase starts creating a list The setup phase starts creating a list of a operators to be used during of a operators to be used during planning.planning.
Exploiting the GUI structure to Exploiting the GUI structure to derive hierarchical operators that derive hierarchical operators that are decomposed during planning.are decomposed during planning. System-Interaction OperatorsSystem-Interaction Operators Abstract OperatorsAbstract Operators
![Page 16: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/16.jpg)
System-Interaction System-Interaction OperatorsOperators
To represent sequences of GUI actions To represent sequences of GUI actions that a user might perform to eventually that a user might perform to eventually interact with the underlying software.interact with the underlying software.
A sequence of zero or more A sequence of zero or more unrestricted-focus eventsunrestricted-focus events, followed by , followed by a a system-interaction eventsystem-interaction event..
Example:Example: Edit_Cut = <Edit, Cut>Edit_Cut = <Edit, Cut> Edit_Paste = <Edit, Paste>Edit_Paste = <Edit, Paste>
![Page 17: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/17.jpg)
Abstract OperatorsAbstract Operators
Created from the restricted-focus Created from the restricted-focus events, which contain two parts:events, which contain two parts: The The prefixprefix of an abstract operator is of an abstract operator is
the sequence of the sequence of unrestricted-focus unrestricted-focus eventsevents that lead to that lead to restricted-focus restricted-focus eventevent..
The The suffixsuffix of an abstract operatorof an abstract operator represents the represents the restricted-focus restricted-focus user user interaction.interaction.
![Page 18: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/18.jpg)
2.3.3 Operator-Event 2.3.3 Operator-Event mappingmapping
In order to keep a correspondence In order to keep a correspondence between the original GUI events and between the original GUI events and these high-level operators.these high-level operators.
![Page 19: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/19.jpg)
2.3.4 The Step1 of 2.3.4 The Step1 of example GUIexample GUI
(a) Original GUI Events
(b) Planning operators derived by PATHS
![Page 20: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/20.jpg)
2.3.5 The Step2 of 2.3.5 The Step2 of EDIT_CUTEDIT_CUT
![Page 21: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/21.jpg)
2.3.6 The Step3 initial state 2.3.6 The Step3 initial state and goal stateand goal state
![Page 22: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/22.jpg)
2.3.7 The Step4 generate 2.3.7 The Step4 generate test casetest case
High level plan that must be decomposition
decomposition
decomposition
![Page 23: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/23.jpg)
![Page 24: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/24.jpg)
![Page 25: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/25.jpg)
An alternative An alternative decompositiondecomposition
![Page 26: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/26.jpg)
A new test caseA new test case
![Page 27: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/27.jpg)
3. Coverage Criteria for GUI 3. Coverage Criteria for GUI TestingTesting
3.1 What is Coverage Criteria3.1 What is Coverage Criteria 3.2 Event-flow Graphs3.2 Event-flow Graphs 3.3 Integration Tree3.3 Integration Tree 3.4 Intra-component Coverage 3.4 Intra-component Coverage
CriteriaCriteria 3.5 Inter-component Coverage 3.5 Inter-component Coverage
CriteriaCriteria
![Page 28: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/28.jpg)
3.1 What is Coverage 3.1 What is Coverage CriteriaCriteria
Coverage criteria are sets of rules to Coverage criteria are sets of rules to help determine whether a test suite help determine whether a test suite has adequately tested a program and has adequately tested a program and to guide the testing process.to guide the testing process.
Important rules that provide an Important rules that provide an objective measure of test quality.objective measure of test quality.
![Page 29: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/29.jpg)
This paper define a new class of This paper define a new class of coverage criteria called coverage criteria called event-based event-based coverage criteriacoverage criteria to determine the to determine the adequacy of tested event sequences, adequacy of tested event sequences, focus on GUIs.focus on GUIs.
The key idea is to define the The key idea is to define the coverage of a test suite in terms of coverage of a test suite in terms of GUI events and their interactions.GUI events and their interactions.
![Page 30: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/30.jpg)
A GUI component is represented by A GUI component is represented by a new structure called an a new structure called an event-flow event-flow graphgraph that identifier events within a that identifier events within a component.component.
The interactions among GUI The interactions among GUI components are captured by a components are captured by a representation called the representation called the integration integration treetree..
![Page 31: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/31.jpg)
Intra-component coverage criteriaIntra-component coverage criteria for events within a componentfor events within a component eventevent, , event-interactionevent-interaction and and length-n length-n
event-sequenceevent-sequence coverage. coverage. Inter-component coverage criteriaInter-component coverage criteria
for events among components.for events among components. invocationinvocation, , invocation-terminationinvocation-termination and and
length-n event-sequencelength-n event-sequence coverage coverage
![Page 32: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/32.jpg)
3.2 Event-flow Graphs3.2 Event-flow Graphs
![Page 33: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/33.jpg)
A part of the MainA part of the Main** component of component of MS WordPadMS WordPad
* We assume that all GUIs have a Main component, that is presented to the user when the GUI is first invoked.
III
B
![Page 34: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/34.jpg)
3.3 Integration Tree3.3 Integration Tree
![Page 35: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/35.jpg)
An integration tree for a part of An integration tree for a part of MS WordPadMS WordPad
![Page 36: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/36.jpg)
3.4 Intra-component 3.4 Intra-component Coverage CriteriaCoverage Criteria
3.4.1 Event Coverage3.4.1 Event Coverage 3.4.2 Event-interaction Coverage3.4.2 Event-interaction Coverage 3.4.3 Length-n Event-sequence 3.4.3 Length-n Event-sequence
CoverageCoverage
![Page 37: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/37.jpg)
3.4.1 Event Coverage3.4.1 Event CoverageEvent-
sequence:
![Page 38: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/38.jpg)
3.4.2 Event-interaction 3.4.2 Event-interaction CoverageCoverage
![Page 39: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/39.jpg)
3.4.3 Length-n Event-3.4.3 Length-n Event-sequence Coveragesequence Coverage
![Page 40: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/40.jpg)
3.5 Inter-component 3.5 Inter-component Coverage CriteriaCoverage Criteria
3.5.1 Invocation Coverage3.5.1 Invocation Coverage 3.5.2 Invocation-termination 3.5.2 Invocation-termination
CoverageCoverage 3.5.3 Inter-component Length-n 3.5.3 Inter-component Length-n
Event-sequence CoverageEvent-sequence Coverage
![Page 41: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/41.jpg)
3.5.1 Invocation 3.5.1 Invocation CoverageCoverage
![Page 42: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/42.jpg)
3.5.2 Invocation-3.5.2 Invocation-termination Coveragetermination Coverage
![Page 43: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/43.jpg)
3.5.3 Inter-component 3.5.3 Inter-component Length-n Event-sequence Length-n Event-sequence
CoverageCoverage
![Page 44: 20041221 gui testing survey](https://reader036.fdocuments.in/reader036/viewer/2022062320/558e8f801a28abf62d8b46e7/html5/thumbnails/44.jpg)
4. Conclusion4. Conclusion
Automatic testing of GUIs that builds Automatic testing of GUIs that builds on AI planning techniqueon AI planning technique
Coverage criteria for GUI testingCoverage criteria for GUI testing We also plan to explore the possibility We also plan to explore the possibility
of using the event-based coverage of using the event-based coverage criteria for software other than GUIs.criteria for software other than GUIs. Object-oriented softwareObject-oriented software Networking softwareNetworking software The broader class of reactive softwareThe broader class of reactive software
Any questions?