Software Testing of Mobile Applications: Challenges and Future Research Directions

Università degli Studi dell’Aquila

1

Università degli Studi dell’Aquila

Software Testing of Mobile Applications:Challenges and Future Research Directions

@AST 2012, the 7th International Workshop on Automation of Software Test, Zurich, June 02 2012

Henry Muccini, Antonio Di Francesco, Patrizio Esposito Dipartimento di Ingegneria, Scienze dell’Informazione e Matematica,

University of L’Aquila www.henrymuccini.com, @muccinihenry

2

Goa

l & R

Qs

Goal and RQs

Goal of this presentation is to convey two messages to the audience:

testing applications running on mobile devices is an important task

those applications require dedicated testing techniques

3

This work is conducted in the context of a series of activities conducted by my Department on mobile applications:

Testing of (context-aware) applications Engineering of mobile applications A course we launches one year ago on

“applications for mobile devices) A lab we are planning to build on those topics

# of downloads:

29 billions of mobile apps downloaded in 2011. abiresearch

185 billions of downloaded apps by 2014. Gartner

Market:

$ 128 billions (2011).

$ 150 billions (2012).Strategy Analytics

The mobile apps market is growing…

… but failures are growing as well…

http://code.google.com/p/android/issues/list

… and mobile applications are being used in safety/business domains

M-health

Navatar

M-Payments

6

Mobile Computing & Context-Awareness

In mobile computing an application is considered to be mobile if it runs on an electronic device that may move (e.g., mp3 readers, digital camera, mobile phones).

In context-aware computing, an application is aware of the computing environment in which it runs, and adapts/reacts according to its computing, user, physical, or time context [Schilit1994,Chen2000].

7

Native applicationsWeb applicationsHibrid applications

Type of Mobile Apps

Why differentiating Context-aware apps from Apps4Mobile?

From a testing perspective: The type and variaty of inputs are unpredictable Low programming support

From a technological perspective: Sensors-based applications are becoming pervasive

From a market perspective: $ 96 billions in 2015 (gartner) “context-aware mobile applications will change the business world…”

[Beth Schultz] ”creating applications for use in the mobile context will ultimately create

better applications that scale across connected devices” [Eric Schmidt] Projects ongoing

Peculiarities of mobile applications10

Type of Mobile

Application

Peculiarity Implications on Testing

Mobile Applications

MobileApps

Apps4Mobile

Mobile Connectivity Reliability, performance, security, and functional testing through different networks

Limited Resources Performance and functional monitoring

Autonomy Monitoring for energy consumption

User Interface GUI Testing

Context Awareness Context-dependent functional and extra functional testing

Adaptation Adaptation correctness

New programming languages New white box and black-box testing, byte-code analysis

New O.S.s Compatibility and O.S. testing

Diversity of phones and O.S.s Diversity coverage

Touch Screens Usability, and response to screen touch

Peculiarities and Implications on testing11

Type of Mobile

Application


Mobile Applications

MobileApps

Apps4Mobile












Type of Mobile

Application


Mobile Applications

MobileApps

Apps4Mobile











512 MB of RAM,64 GB of disk space, and 1 Ghz dual core

CPU


Type of Mobile

Application


Mobile Applications

MobileApps

Apps4Mobile











iPhone 4S:200 hours autonomy in stand-by …drops to9 hours when Wi-Fi is enabled … drops to 6 hours when a 3G connection is active

New OS releases to increase autonomy


Type of Mobile

Application


Mobile Applications

MobileApps

Apps4Mobile











Huge effort


Type of Mobile

Application


Mobile Applications

MobileApps

Apps4Mobile











Is the app going to work in any environment and under any unpredictable scenarios?


Type of Mobile

Application


Mobile Applications

MobileApps

Apps4Mobile











1,800 different hw/sw configurations

Challenges

Test Selection for context-aware mobile applications

Challenges: unpredictabile and high variable inputs

Potentials: Create higher level libraries for developers Coverage criteria Scenario-based approaches

17

public class SensorActivity extends Activity, implements SensorEventListener { private final SensorManager mSensorManager; private final Sensor mAccelerometer;

public SensorActivity() { mSensorManager = (SensorManager)getSystemService(SENSOR_SERVICE); mAccelerometer = mSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER); }

protected void onResume() { super.onResume(); mSensorManager.registerListener(mAccelerometer, SensorManager.SENSOR_DELAY_NORMAL); }

protected void onPause() { super.onPause(); mSensorManager.unregisterListener(this); }

public void onAccuracyChanged(Sensor sensor, int accuracy) { }

public void onSensorChanged(SensorEvent event) { }}

Context-aware programming

Challenges

Security testingChallenge:

the mobility of the device into networks with different security levels increases security.

a trojan might have access to personal date, private networks, and credit card information

the rich contextual information presents real privacy concerns.

the democratization of mobile application publishers also increases the number of apps store the final user does not know anything about.

20

Challenges

Performance and Reliability testingChallenge:

Performance and reliability of mobile applications strongly depends on the mobile device resources, on the device operational mode, on the connectivity quality and variability, and other contextual information.

Potentials: new techniques to explicitly consider characteristics related

to (changing) contexts and different devices. Run-time analysis techniques to prevent performance

degradation.

21

Challenges

Product LineChallenge:

130 devices running Android, with 1,300 hw/sw configurations

Potentials new systematic combinatorial/product line testing

techniques run-time analysis of instrumented beta versions of the

application testing as a service

22

Conclusions

Test automation to decrease the cost of testing mobile applications must be cheap outsourcing, cloud- and crow-based testing

Testing automation towards all the different layers current bugs are due to interoperability problems that exist

today among the application, application framework, operating system, and hardware (sensoring) layers

25

Software Testing of Mobile Applications: Challenges and Future Research Directions

Education

Transcript of Software Testing of Mobile Applications: Challenges and Future Research Directions