Top Practices for Successful Mobile Test Automation
-
Upload
techwellpresentations -
Category
Technology
-
view
193 -
download
0
description
Transcript of Top Practices for Successful Mobile Test Automation
![Page 1: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/1.jpg)
T22
Mobile Testing
5/8/2014 3:00:00 PM
Top Practices for Successful
Mobile Test Automation
Presented by:
Fred Beringer
SOASTA
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
![Page 2: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/2.jpg)
Fred Beringer
SOASTA
Fred Beringer is VP of products at SOASTA where he is helping customers get started with mobile test automation. Previously, as software testing director for Experian, Fred built and led a worldwide organization responsible for testing all products within the Experian Decision Analytics portfolio. He introduced cloud-based testing at Experian while helping the development organization transition to agile. Throughout his career with IBM and Experian, Fred has driven strategic projects around data replication (for Big Data customers), business intelligence, performance testing, cloud computing, cloud testing, and open source. Follow him on Twitter @fredberinger, email him at [email protected], and learn more about Fred at fredberinger.com.
![Page 3: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/3.jpg)
TOP BEST PRACTICES FOR
SUCCESSFUL MOBILE TEST AUTOMATION
@fredberinger
![Page 4: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/4.jpg)
SOMETHING FOR YOU TO TAKE BACK HOME
![Page 5: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/5.jpg)
![Page 6: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/6.jpg)
MOBILE TEST AUTOMATION IS HARD
![Page 7: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/7.jpg)
EXPECTATION
• It is as difficult as developing a good mobile app • It is a software project by itself • It is not optional in mobile • You will probably fail before you succeed
![Page 8: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/8.jpg)
#1 - SET THE RIGHT GOALS
![Page 9: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/9.jpg)
AUTOMATION OBJECTIVES
• Continuous feedback to fix bugs faster • Enable new activities • Increase test coverage • Raise confidence • Better customer support
![Page 10: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/10.jpg)
#2 – TRACK YOUR OBJECTIVES
IF IT IS NOT MEASURED, IT DOESN’T EXIST
![Page 11: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/11.jpg)
SET YOUR OWN METRICS
• Turnaround time for fixes • Customer satisfaction on app store • Customer onboarding time • Lower number of bugs on production • EMTE (Equivalent Manual Test Effort)
![Page 12: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/12.jpg)
ACTIONABLE TEST INTELLIGENCE
• Part of the automation framework • Available and highly visible to everyone • Establish a baseline and track ROI • Take costs into account (Value=Benefit-cost) • Too many metrics can hurt • Rinse and repeat until you get it right
![Page 13: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/13.jpg)
#4
BUT NOT ALL THE TESTS!
![Page 14: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/14.jpg)
MISTER JENKINS IN YOUR FRIEND
• Automate your build • Automate your app deployment • Automate your environment deployment • Automate your tear down • Automate your test execution • Automate your reporting • Automate your metrics tracking • Automate your communication
![Page 15: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/15.jpg)
#5 – AUTOMATE ASAP
BUT BUILD THE FOUNDATION FIRST
![Page 16: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/16.jpg)
Careful planning • Plan your Automation infrastructure
• Cloud, on-premise, devices connectivity, app deployment, data aggregation, reporting, etc.
• Pick your automation tool wisely • Start small and grow • Don’t hesitate to pause tests building activity
![Page 17: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/17.jpg)
#6 – PICK THE RIGHT TESTS TO AUTOMATE
AND I CLEARLY DON’T HAVE A GOOD PICTURE FOR THIS SLIDE
![Page 18: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/18.jpg)
ANALYZE THE RISK OF FAILURE
R(c)=P(c) * I(c) Probability • Code Complexity • Changed Areas • Affected Interfaces • New Technology • Component Maturity
Impact • Financial • Reputation • Legal • Security • Loss of Customers
![Page 19: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/19.jpg)
#7 – WHERE TO RUN TESTS
HINT: 0% USERS RUN YOUR APP ON A SIMULATOR
![Page 20: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/20.jpg)
SIMULATOR VS REAL DEVICES
• Pros • Cheap • Integrated with IDE
• Cons • Not testing on actual platform.
What if the test pass? What’s next?
• Network is different • OS is different (stock) • Can’t simulate real hardware
(CPU, Memory, etc.) Not fit for mobile performance
• Pros • Reproduce real gestures • Real results, no false negative • Can test under OEM
customization • Fit for mobile performance
• Cons • Need to be managed (device
Cloud helps!)
![Page 21: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/21.jpg)
UNDERSTAND YOUR MARKET
74% iOS7
![Page 22: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/22.jpg)
HAVING TROUBLE WITH ANDROID
![Page 23: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/23.jpg)
#8 – BUILD EFFICIENT TESTS
SIZE MATTERS!!
![Page 24: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/24.jpg)
S.F.I.R.S.T.R
• Small – Easier to understand & Fix • Fast – Parallel execution for faster feedback • Independent – Can run any subsets in any order • Repeatable – Tests get the same result every time • Self-Checking – No human checking • Timely – Should be written in parallel with dev • Reusable – To avoid maintenance nightmares
![Page 25: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/25.jpg)
ASK KINDLY …
#8 – BUILD TESTABILITY
![Page 26: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/26.jpg)
WHICH IS BETTER?
• Build more reliable tests with unique IDs • Lower maintenance cost • Test early • A non-optional investment
//TiUIScrollView[@touchTestId='mfaSiteKey-signOn_securityPhraseWindow_ScrollView']/TiUIScrollViewImpl/UIView/TiUIView[.//UILabel[@text='If you don't recognize your personalized security image, don't enter your password.']]/TiUITextField/TiTextField
classname=TiTextField[1]
id=mfaSiteKey-siteKey_Password_Input XPATH
INDEXED OBJECT
UNIQUE ID
text=blah
TEXT
![Page 27: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/27.jpg)
# 9 - DATA IS YOUR FRIEND
![Page 28: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/28.jpg)
BE DATA-DRIVEN
• Increase test coverage FAST • Easy to add, remove and configure tests • Reduce the number of tests to maintain • Separation of Tests and Datas
![Page 29: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/29.jpg)
FORGET ABOUT TIME DELAYS!
#10 – USE RELIABLE WAITS
![Page 30: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/30.jpg)
USE RELIABLE AND INTELLIGENT WAITS
• Use object visibility to manage the flow of the test • Never use time delays to manage back-end variability • Never use time delays to account for device
performance • Spend time finding the right locator to wait on • Run your test 100 times before claiming success
![Page 31: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/31.jpg)
YOU GET THE PICTURE
#11 – LEVERAGE FUNCTIONAL TESTS FOR PERFORMANCE
![Page 32: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/32.jpg)
COVER ALL YOUR BASES
CPU Battery Memory Transaction timing
Native Apps
No backend connection
Stress tests Endurance tests
Load test
Mobile Web Apps App
Server
Database
App Server
Web Server
Cache Load Balancer Web
Server Data traffic HTTP(S)
UDP WebSocket
Shared web & mobile infrastructure
Web browser users
Mobile browser users
Native app users
CPU Battery Memory Transaction timing Response time
![Page 33: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/33.jpg)
#12 – BUILD THE RIGHT TEAM
![Page 34: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/34.jpg)
LEAN AND EFFICIENT • Quality Software Engineers
• Automation Framework ie. CI, Cloud, Pyton, Perl, Web, everything
• Automation Architect • What tests to automate? Identify reusable components,
Metrics, ROI, everything
• Automation Engineer • Build automated tests, everything • Expert in automation and knowledge of mobile apps dev
• Test Engineer • Build tests, everything
![Page 35: Top Practices for Successful Mobile Test Automation](https://reader036.fdocuments.in/reader036/viewer/2022081403/5550ba3fb4c905ff618b4e05/html5/thumbnails/35.jpg)
TOP BEST PRACTICES FOR
SUCCESSFUL MOBILE TEST AUTOMATION
twitter.com/fredberinger
http://www.fredberinger.com
QUESTIONS?