Using GUI Ripping for Automated Testing of Android Apps

Using GUI Ripping for Using GUI Ripping for Automated Testing of Android Automated Testing of Android

AppsApps

Domenico AmalfitanoDomenico Amalfitano

Salvatore De CarmineSalvatore De Carmine

Anna Rita FasolinoAnna Rita Fasolino

Porfirio Tramontana Porfirio Tramontana

Dipartimento di Informatica e SistemisticaDipartimento di Informatica e Sistemistica

University of Naples Federico II, ItalyUniversity of Naples Federico II, Italy

Atif MemonAtif MemonDepartment of Computer ScienceDepartment of Computer Science

University of MarylandUniversity of Maryland

Porfirio Tramontana – ASE 2012 – Essen – September 6, Porfirio Tramontana – ASE 2012 – Essen – September 6, 20122012

22

MotivationMotivation Android is a very popular operative system and framework for Android is a very popular operative system and framework for

mobile devicesmobile devices Its diffusion is increasing: recently it overtook ioS and Symbian in terms of Its diffusion is increasing: recently it overtook ioS and Symbian in terms of

diffusiondiffusion Over than 500,000 apps are now on the Google Play marketOver than 500,000 apps are now on the Google Play market The most part of the apps are realized by single programmers or little teams The most part of the apps are realized by single programmers or little teams

and lacks in qualityand lacks in quality

Android apps are often developed according to an evolutionary Android apps are often developed according to an evolutionary life cycle, with very frequent releases via Google Playlife cycle, with very frequent releases via Google Play Bugs causing failures and, in particular, crashes are very commonBugs causing failures and, in particular, crashes are very common

There is a strong need for There is a strong need for testing automation testing automation processes processes and and toolstools In this talk, our techniques and tool supporting a process of Android In this talk, our techniques and tool supporting a process of Android

GUI Testing Automation will be presentedGUI Testing Automation will be presented


Android appsAndroid apps

An Android application is composed of several types of Java An Android application is composed of several types of Java components instantiated at run-time:components instantiated at run-time:

Activities, Services, Intents, Broadcast Receivers and Content Providers. Activities, Services, Intents, Broadcast Receivers and Content Providers.

An An ActivityActivity component: component: is responsible for presenting a visual user interface; is responsible for presenting a visual user interface; provides a screen with which the users can interact in order to provides a screen with which the users can interact in order to

do something.do something. An Android application usually includes one or more Activity An Android application usually includes one or more Activity

classes that extend the base Activity class provided by the classes that extend the base Activity class provided by the Android development framework.Android development framework.

Android Applications are Android Applications are Event-Driven SystemsEvent-Driven Systems Three types of events can be triggered:Three types of events can be triggered:

user events (such as Click, MouseOver, etc.);user events (such as Click, MouseOver, etc.); events related to the lifecycle of the Activity classesevents related to the lifecycle of the Activity classes

onStart, on Pause, onResume, onStop, onDestroy, …onStart, on Pause, onResume, onStop, onDestroy, … events that are triggered by other input sourcesevents that are triggered by other input sources

such as GPS receiver, phone, network, etc.such as GPS receiver, phone, network, etc.

33


Existing approaches to Android Testing Existing approaches to Android Testing AutomationAutomation

JUnit TestingJUnit Testing Android Testing FrameworkAndroid Testing Framework

Supports the coding of executable test classesSupports the coding of executable test classes Classes supporting the instrumentation and monitoring of Android apps are provided in the Classes supporting the instrumentation and monitoring of Android apps are provided in the

Android framework itselfAndroid framework itself Libraries such as Libraries such as Robotium Robotium and and MonkeyrunnerMonkeyrunner extends the framework extends the framework

Random TestingRandom Testing MonkeyMonkey (included in the Android framework) supports the randomly (included in the Android framework) supports the randomly

automatic exploration of Android appsautomatic exploration of Android apps

Model Based TestingModel Based Testing TEMA ToolsTEMA Tools (Takala, Katara, Harty, 2011) represent a first tentative of (Takala, Katara, Harty, 2011) represent a first tentative of

approach for the generation of test code from a design model of approach for the generation of test code from a design model of Android appAndroid app

44


Android GUI RippingAndroid GUI Ripping

We propose a technique and a tool for:We propose a technique and a tool for: Automatic exploration of the GUI of an Automatic exploration of the GUI of an

Android application via a systematic Android application via a systematic RippingRipping approachapproach

Similar to approaches adopted for Web crawlingSimilar to approaches adopted for Web crawling Crash testing Crash testing GUI model reverse engineeringGUI model reverse engineering JUnit test case generationJUnit test case generation

55


Android GUI Ripping AlgorithmAndroid GUI Ripping Algorithm

CustomizableCustomizable Selection of types of events to be fired Selection of types of events to be fired Adoption of a GUI traversing strategy (e.g. depth first, breadth Adoption of a GUI traversing strategy (e.g. depth first, breadth

first, etc.)first, etc.) Termination criteria used to stop the GUI traversalTermination criteria used to stop the GUI traversal Setting of specific preconditionsSetting of specific preconditions

Efficient Efficient Provides heuristic solutions to the well known problem of GUI Provides heuristic solutions to the well known problem of GUI

interface states and events explosioninterface states and events explosion GUI exploration criteria based on a Interface Equivalence GUI exploration criteria based on a Interface Equivalence

CriterionCriterion

66


1. Task List Initialization;2. while (Task List Is Not Empty) {3. Extract a Task From The Task List;4. Execute the Task;5. Abstract the Current GUI Abstract

State;6. Update the GUI Tree Model;7. if (GUI Exploration Criterion) then

{8. Define New Tasks ;9. Add New Tasks To The Task

List;10. }

11. }

77

RotateRotate

Press MenuPress MenuClick RefreshClick RefreshClick Click New New PostPost

Click PagesClick Pages

Click AboutClick AboutClick Add Click Add AccountAccount

≡≡ ≡≡

Click Click EditEdit

CrashCrash

……

……

Click SaveClick Save

…………

Ripping Ripping AlgorithmAlgorithm

1. Task List Initialization;2. while (Task List Is Not

Empty) {3. Extract a Task From

The Task List;4. Execute the Task;5. Abstract the Current

GUI Abstract State;6. Update the GUI Tree

Model;7. if (GUI Exploration

Criterion) then {8. Define New Tasks ;9. Add New Tasks To

The Task List;10. }11. }


Crash Testing ProcessCrash Testing Process

Preconconditions are Preconconditions are set by generating a set by generating a Snapshot Image of the Snapshot Image of the device for the emulatordevice for the emulator

In the task execution In the task execution step, the GUI Ripping step, the GUI Ripping algorithm is executedalgorithm is executed

88


A Demonstrational exampleA Demonstrational example

Android for WordpressAndroid for Wordpress A Wordpress client for A Wordpress client for

AndroidAndroid 334 classes, ~10,000 LOCs334 classes, ~10,000 LOCs More than 250,000 downloadsMore than 250,000 downloads

A set of three preconditionsA set of three preconditions No LoginNo Login Single BlogSingle Blog Multiple BlogsMultiple Blogs

Comparison between Comparison between Ripping and Random Testing Ripping and Random Testing with Monkeywith Monkey

99

Measured metricsMeasured metrics EffectivenessEffectiveness

Number of crashes Number of crashes foundfound

Number of Number of different bugs different bugs causing crashescausing crashes

LOCs coverageLOCs coverage

EfficiencyEfficiency Number of test Number of test

casescases Number of eventsNumber of events Ripping time Ripping time


ResultsResults

1010

Preconditions

No Login Single Blog Multiple Blog

Random (Single Blog)

#Crashes 0 6 8 3

#Bugs 0 3 4 1

LOC Cov % 2.65 39.32 37.83 25.27

Time (h) 0.2 4.88 4.58 4.46

#TC 18 166 166 -

#Events 36 517 502 14400


Bugs and CrashesBugs and Crashes

Id Crash Description Java Exception

Ticket and Changeset

B1 The app crashes trying to opening the default post “Hello World”

StringIndexOutOfBoundException

https://android.trac.wordpress.org/ticket/206https://android.trac.wordpress.org/changeset/398

B2 The app crashes when the Stats activity is rapidly opened and closed (via the Back key).

BadTokenException


B3 The app crashes when the Stats activity is open and the Refresh button is clicked while the progress bar widget is still loading.

NullPointerException

https://android.trac.wordpress.org/ticket/212

https://android.trac.wordpress.org/changeset/423

B4 The app crashes when the user opens a post and tries to share it within his blog. The crash occurs when there is a single blog in the app.

NullPointerException


1111

The 14 crashes The 14 crashes found by the found by the Ripping process Ripping process belongs to four belongs to four different bugsdifferent bugs Bugs have been Bugs have been

submitted to the submitted to the bug tracking bug tracking system and system and accepted/fixed by accepted/fixed by the programmerthe programmer

Monkey was able Monkey was able to find only one of to find only one of these bugsthese bugs


1212

Conclusions and Future WorksConclusions and Future Works

Our technique supports the execution of Our technique supports the execution of completely automatic generation and completely automatic generation and execution of test cases for Android execution of test cases for Android applications. applications. In our preliminary demonstration example:In our preliminary demonstration example: Real bugs and crashes have been detected;Real bugs and crashes have been detected; Good level of code coverage has been reached;Good level of code coverage has been reached; Testing time is compatible with smoke testing Testing time is compatible with smoke testing

processes;processes; Reusable test cases has been automatically Reusable test cases has been automatically

generated;generated; Our technique demonstrates its effectiveness with Our technique demonstrates its effectiveness with

respect to standard Random testing tools such as respect to standard Random testing tools such as MonkeyMonkey


1313

Future WorksFuture Works

This presentation covers the first part of a This presentation covers the first part of a larger project on testing automation of Android larger project on testing automation of Android applicationsapplications

Some future worksSome future works Publication of the tool Publication of the tool

As a standalone toolAs a standalone tool As a serviceAs a service

A preliminary demo is available at A preliminary demo is available at http://wpage.unina.it/ptramont/GUIRipperWiki.htmhttp://wpage.unina.it/ptramont/GUIRipperWiki.htm

Further experiments with more applications and Further experiments with more applications and ripping parametersripping parameters

Delay Time, ripping strategy (depth first/breadth first, Delay Time, ripping strategy (depth first/breadth first, activity equivalence criteria, maximum depth), input activity equivalence criteria, maximum depth), input generation strategies, …generation strategies, …

Sensor event testingSensor event testing


1414

Using GUI Ripping for Automated Testing of Android Apps

Technology

Transcript of Using GUI Ripping for Automated Testing of Android Apps