1

MOET

https://github.com/eing/moet Mobile End-to-End Testing Mobile Exploratory Testing

Eing Ong

Intuit, Inc. @eingong|eing.ong@intuit.com

1

2

Mobile'automation'technologies1

•  Two categories •  Instrumented technique •  Non-instrumented technique

•  What is instrumentation? •  Tests are compiled with the app •  Tests are installed & launched with the app •  Source code is required and may need to be modified •  Only one app can be executed at a time •  White box approach

3

Advantages'of'both'techniques1

Non$instrumentation- Instrumentation-

•  Device'platform'agnostic1•  Test'code'reuse1•  Test'language'&'test'harness'

autonomy1•  Support'for1

Multi>applications'testing1Custom'composite'UI'elements1Database/server'API'assertions1Use'of'external'libraries1'(e.g.'hardware/image'libraries)1

•  Elements'can'be'accessed1•  Debugging'ease1•  Test'verification'ease1•  Reduce'tools'dependencies1•  Support'for1

Installing'application1Launching'application1Cleanup'(kill'application)1Test'execution'on'device1Code'coverage1

4

Which'technique'should'I'use'?1

Text$based3features-

Image3based3features-

Hardware3interaction-

External3interaction-

Instrumentation Non-instrumentation

5

Mobile'automation'tools1

Mobile3-OS-

Non$instrumentation-

Instrumentation-

Android1 eggPlant,'Sikuli,'MOET,'MonkeyRunner1

Robotium,'Calabash,'MonkeyTalk1

iOS1 eggPlant,'Sikuli,'MOET1 UIAutomation,'KIF,'iCuke,'Frank,'UISpec,'Zucchini,'Bwoken,'Calabash,'MonkeyTalk,'Applecart,'iOS>driver1

6

touch(100,100)

enter(username) touch(100,200)

enter(password)

touch(150, 300)

void enter()

void touch(x,y)

Login(“user1”,”passwd1”) Test

iPhone Sikuli library

iPhone implementation

MOET1

•  Think design o  Interfaces o  Creational pattern

•  Think reuse o  Device independent tests

•  Think One o  IDE o  Test harness and language

7

Anatomy'of''MOET1

iPhone3application3implementation-

Android3MonkeyRunner3Library-

Device3Independent3Tests-

Simulator/Device libraries

Mobile3Application3Interface-

Runtime binding

iPhone3Sikuli3Library-

Android3application3implementation-

8

Top Ten Wish List

a.k.a.

Mobile test automation

Wall of Pain

8

9

Android'Wall'of'Pain1

10.  There’s more to test execution o  Command with report file generation :

adb shell am instrument -w -e reportFile junit-report.xml com.intuit.test/com.zutubi.android.junitreport.JUnitReportTestRunner

o  Date-time fix if using device snapshots : adb shell date -s `date +"%Y%m%d.%H%M%S”`

9.  Where are the test results ? o  Pulling results from device :

adb pull /data/data/com.intuit.test/files/junit-report.xml junit-report.xml

10

Android'Wall'of'Pain1

10.  There’s more to test execution o  Command with report file generation :

adb shell am instrument -w -e reportFile junit-report.xml com.intuit.test/com.zutubi.android.junitreport.JUnitReportTestRunner

o  Date-time fix if using device snapshots : adb shell date -s `date +"%Y%m%d.%H%M%S”`

9.  Where are the test results ? o  Pulling results from device :

adb pull /data/data/com.intuit.test/files/junit-report.xml junit-report.xml

11

iOS'Wall'of'Pain1

8.  iOS simulator…the singleton pattern o  Multiple instances of simulator o  Headless execution

7.  What test harness ? o  Test grouping, test assertions, base test class o  Results needs to be displayable in browser

6.  How about a CI plugin o  Test execution o  Code coverage

12

iOS'Wall'of'Pain1

8.  iOS simulator…the singleton pattern o  Multiple instances of simulator o  Headless execution

7.  What test harness ? o  Test grouping, test assertions, base test class o  Results needs to be displayable in browser

6.  How about a CI plugin o  Test execution o  Code coverage

13

iOS'Wall'of'Pain1

8.  iOS simulator…the singleton pattern o  Headless execution o  Multiple instances of simulator

7.  What test harness ? o  Test grouping, test assertions, base test class o  Results needs to be displayable in browser

6.  How about a CI plugin o  Test execution o  Test results o  Code coverage

14

iOS'Wall'of'Pain1

5.  Where’s the IDE plugin ? o  Code formatting o  API completion o  Breakpoints, debugger o  “javadoc” help o  One click test execution of test method/testsuite

15

All'mobile'platforms1

4.  Problem ? Search ? o  Blogs, stackoverflow, developer groups for devices, etc. o  Mobile testing one-stop tips & tricks (not questions) o  Community driven effort

Tip – Remove authorization prompt

Administrative rights Update /etc/authorization

<key>system.privilege.taskport</key> <dict> <key>allow-root</key> <!-- previous value <false/> --> <true/>

Tip – Clean cache, preferences, SQLite ~/Library/Application Support/iPhone Simulator/{SDK}/Applications/<uuid>

Library: Preferences, Caches Documents: sqlite3 <app>.sqlitedb

16

All'mobile'platforms1

3.  How about a mobile test report ?

o  Mobile test report standard e.g. mobile-test-report.xml •  Screenshots and video recording of failed tests •  Device logs •  Device detection

2.  Wireless?

o  OTA app installation o  Testing via wifi and carrier networks o  Selenium grid-like test infrastructure

17

All'mobile'platforms1

3.  How about a mobile test report?

o  Mobile test report standard e.g. mobile-test-report.xml •  Screenshots and video recording of failed tests •  Device logs •  Device detection

2.  Mobile testing is anything but mobile

o  OTA app installation and execution o  Testing via wifi and carrier networks o  Test tool independent component o  Selenium grid-like test infrastructure

18

All'mobile'platforms1

1.  Yet another proprietary language?

o Open sourced test harness & language for •  Instrumented tests •  Non-instrumented tests •  Hybrid apps tests

19

Next'steps1

Monkey$Runner-

Device3Independent3Tests-

Simulator/Device libraries

Mobile3Application3Interface-

Runtime binding

Sikuli-

Non$instrumented3application3implementation-

Instrumented3application3implementation-

?Android?- ?iOS?3-

Eclipse mobile testing plugin

Test development

Test execution

Test reporting

Continuous integration