9. April 202319. April 2023

Customer Visual

How to Break your App - Best Practices in Mobile App Testing

Daniel Knott | @dnlkntt

29. April 2023

Who am I ?

• Daniel Knott• Software Test Manager/ Mobile Tester @AOE• Mobile Testing since the end of 2010• Before AOE, Team Lead QA @XING Mobile Team• Blogger and Speaker

http://www.adventuresinqa.com

@dnlkntt

http://www.linkedin.com/pub/daniel-knott/1a/925/993

https://www.xing.com/profile/Daniel_Knott

9. April 202339. April 2023

I Company Profile

49. April 2023

Intro

Current Status

Mobile Challenges

The Customer

Mobile Specific Testing

Mobile PyramidTools

9. April 202359. April 2023

I Current Status

Author: Barry Caruth, https://www.flickr.com/photos/caruth/8401943928/

69. April 2023

Current Status | Some Numbers

2+ millions apps available

100+ billion downloads

80% – 90% of apps are deleted after the first

usage56% of apps that requires a registration gets

deletedAverage user checks device 150 times per

day45% are not well tested

79. April 2023

KIFSU – To Remember the important Things

SimpleUseabl

eFastItKeep

K I F S U

9. April 202389. April 2023

I Mobile Challenges

99. April 2023

Mobile Challenges | All known!?

http://opensignal.com/reports/2014/android-fragmentation/

109. April 2023

Mobile Challenges | All known!?

http://opensignal.com/reports/2014/android-fragmentation/

119. April 2023

Mobile Challenges | How to handle fragmentation

Group your devices, A B C

Test only on those devices

Test on real device

Don’t use simulators or emulators

Know the target group

129. April 2023

Mobile Challenges | How to handle fragmentation

Group 1, Prio A

• High End Devices

• Dual/ Quad Core CPU

• RAM >1024MB• Retina, Full HD

Display >= 4,5“

Group 2, Prio B

• Mid-range devices

• Dual Core CPU• RAM <1024MB• Screen size <=

4“• Software not

older than 1 year

Group 3, Prio C

• Small devices• Single Core• RAM < 512MB• Low screen size

and resolution• Older OS, older

browser

Based on your target group

139. April 2023

Author: Davide Vizzini Limbte, http://www.flickr.com/photos/limbte/4007115435/

Mobile Challenges | More…

Author: Juan de Dios Santander Vela, http://www.flickr.com/photos/juandesant/209098053

Author: Rain Rabbit, http://www.flickr.com/photos/37996583811@N01/8033259053/

9. April 2023149. April 2023

I The Customer

Author: Michael Newman, https://www.flickr.com/photos/mzn37/1226016289/

159. April 2023

The Customer | Better know them

Know the target group

Talk to your customer

Will downsize testing effort

Do usability testing with them

Invite them to a beta testing program (Sonos,

FB, Twitter)

Author: Grzegorz Lobinski, http://www.flickr.com/photos/gregloby/3515990945/

9. April 2023169. April 2023

I Mobile Specific Testing

Author: Theus Falcao, http://www.flickr.com/photos/theusfalcao/9563767145/

179. April 2023

Mobile Specific Testing

Known functional testing is still required

• Boundary values

• Equivalence classes

• Testing against acceptance criteria

Test the app as black box

• Try to break it

• Be creative and think in edge cases

189. April 2023

Mobile Specific Testing

Author: Juan de Dios Santander Vela, http://www.flickr.com/photos/juandesant/209098053

Author: Ian Britton, http://www.flickr.com/photos/freefoto/728651045/

Author: Moyan Brenn, https://www.flickr.com/photos/aigle_dore/5626341059/

Author: @Doug88888, http://www.flickr.com/photos/doug88888/7149187135/

Author: Dan Levy, http://www.flickr.com/photos/danlevy/5607259711/

199. April 2023

Mobile Specific Testing

Author: amateur_photo_bore, http://www.flickr.com/photos/andresv/233667770/ Author: Rain Rabbit, http://www.flickr.com/photos/37996583811@N01/8033259053/

Author, Daniel Sancho, http://www.flickr.com/photos/teclasorg/2852716491/

209. April 2023

Mobile Specific Testing

Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4311105154/ Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4464587302/

Author: Theus Falcao, http://www.flickr.com/photos/theusfalcao/9563767145/

219. April 2023

Mobile Specific Testing

Android

• JouleUnit, to measure battery usage

• Monkey, to stress test the app

iOS

• Energy Usage in Instruments

• UI AutoMonkey, to stress test the app

229. April 2023

Mobile Specific Testing

Author: Molly Holzschlag, http://www.flickr.com/photos/mollyeh11/272434559/ Author: Quinn Dombrowski, http://www.flickr.com/photos/quinnanya/4718635484/

239. April 2023

Mobile Specific Testing

Author: minxlj, http://www.flickr.com/photos/minxlj/3270996158/ Author: Ryan B. BulaPhotography, http://www.flickr.com/photos/nayralub/3763581778/

Author: Sterling Coffey, http://www.flickr.com/photos/n0ssc/7046915929/

9. April 2023249. April 2023

I Tools

Author: OZinOH, http://www.flickr.com/photos/75905404@N00/7126147125/

259. April 2023

Test Automation Pyramid

Unit Tests

Integration Tests

UI Tests

Manual Testing

269. April 2023

Flipped Test Pyramid

Unit Tests

Manual Tests

UI TestsE2E

279. April 2023

Mobile Test Pyramid

Manual Tests

Beta Tests

End2End Tests(UI Tests)

Unit Tests

289. April 2023

Tools

Do test automation!

Start simple and discover different tools

Don’t buy huge test automation tools

No tool is perfect

See which tool fits in your development

environment

299. April 2023

Tools | Android

Robotium (ExtSolo)

Selendroid

Spoon

Calabash for Android

Robolectric

Espresso

Monkey

Great mobile tools provided by the company square: http://square.github.io/

MonkeyTalk

Appium

309. April 2023

Tools | iOS

UIAutomation

MonkeyTalk

Keep It Functional

Calabash for iOS

ios-driver

Kiwi

XCTest

Appium

9. April 2023319. April 2023

I Security

Author: Alexandre Dulaunoy, http://www.flickr.com/photos/adulau/7712545428/

329. April 2023

Security

Author: Anthony Catalano, http://www.flickr.com/photos/badwsky/34164244/ Author: Dennis Wong, http://www.flickr.com/photos/denniswong/2406135310/

Âuthor: orkomedix, http://www.flickr.com/photos/orkomedix/5211437958/

339. April 2023

Security

Security testing is not easy

Get help from experts

Check the OWASP mobile security page

https://www.owasp.org/index.php/OWASP_Mo

bile_Security_Project

Think twice about app permissions

9. April 2023349. April 2023

I Summary

359. April 2023

Mobile Specific Testing | Summary

Combine your in house testing with crowd

testingBe up to date! Know your target group

Keep the mobile features in mind

Keep the mobile test pyramid in mind

Do test automation and know the right tools

Be creative and on the move while testing mobile apps

369. April 2023

Thanks for your attention!

Avai

labl

e so

on

www.handsonmobileapptesting.com

More information about mobile testing can be found in my book

„Hands-On Mobile App Testing“

which will be available very soon.

9. April 2023379. April 2023

AOE GmbH

LuisenForum, Kirchgasse 6

65185 Wiesbaden

Germany

Phone: +49 6122 70 70 7 - 0

Fax: +49 6122 70 70 7 - 199

E-Mail: sales@aoe.com

Web: www.aoe.com

I in Germany

389. April 2023

Links & References

• World Quality Report [http://www.sogeti.com/en-us/looking-for-solutions/Services/Software-Control-Testing/The-World-Quality-Report-2013--2014/]

• Robotium [https://code.google.com/p/robotium/]• Spoon [https://github.com/square/spoon]• Robolectric [http://robolectric.org/]• Calabash for Android [https://github.com/calabash/calabash-android]• Selendroid [http://selendroid.io/]• ExtSolo [

http://docs.testdroid.com/_static/extSolodocs/com/bitbar/recorder/extensions/ExtSolo.html]

• Espresso [https://code.google.com/p/android-test-kit/wiki/Espresso]• Monkey [http://developer.android.com/tools/help/monkey.html]• UI AutoMonkey [https://github.com/jonathanpenn/ui-auto-monkey]

399. April 2023

Links & References

• MonkeyTalk [https://www.cloudmonkeymobile.com/monkeytalk]• UIAutomation[https

://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html]

• Keep It Functional [https://github.com/kif-framework/KIF]• XCTest [

https://developer.apple.com/library/ios/documentation/ToolsLanguages/Conceptual/Xcode_Overview/UnitTestYourApp/UnitTestYourApp.html]

• Calabash for iOS [https://github.com/calabash/calabash-ios]• ios-driver [http://ios-driver.github.io/ios-driver/?page=home]• Kiwi [https://github.com/allending/Kiwi]• Appium [http://appium.io/]• JouleJunit [https://code.google.com/p/jouleunit/]• Energy Usage Instruments [

https://developer.apple.com/library/ios/documentation/AnalysisTools/Reference/Instruments_User_Reference/EnergyUsageInstrument/EnergyUsageInstrument.html]

409. April 2023