Post on 21-Jan-2018
The Way to Production:Game Testing & Analitics
Lecture 04
2016-2017Mobile Games Development & Promotion, Maksimenkova
Olga, FCS, SSI1
Today
Testing
• What to test in games?
• How to test?
• Features for mobile games?
Statistics
• Metrics
• Tools
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI2
Game testingTesting and QA
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI3
Beta testing
• Closed (internal) beta
• Open (public/external) beta
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI4
What should be tested in a game?
• Balance (balance testing)• Fairness, a gameplay is fair to both the human and AI player alike• Difficulty balance• Levels balance• Maps, weapons,….• The guidelines for balancing a game see in (The many faces of testing: the
game life cycle, Game Development Essentials: Game Testing & QA by L. Levy, J. Novak)
• Hardware compatibility (compatibility testing)• Video subsystem• Audio subsystem• Controllers (input/output devices)• Network… • See a sample checklist at bitbar (http://bitbar.com/mobile-game-testing-
part-4-test-for-connectivity/)
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI5
What should be tested in a game?
• Compliance testing • Meeting standards, specifications, requirements
• Tolerance
• Warnings
• Copyrights, ….
• Internationalization (Localization Testing)
• Funniness (Playtesting)
• Usability (usability testing)• UI/UX
• Tfortesting. Testing make things perfect, Test Cases For Games Apps / Checklist for Games Apps (https://tfortesting.wordpress.com/2012/10/04/test-cases-for-games-apps-checklist-for-games-apps/)
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI6
Testing techniques
• Ad-Hoc• Finding bugs by playing.
• Automated• Let computer do repetitive work for you
• Functionality• Make sure that game functions properly well according to your expectations• Functionality requirements see in (Race to the Finish Line: elite bug hunting,
Game Development Essentials: Game Testing & QA by L. Levy, J. Novak)
• Progression• Seeking possibly interruptions in a gameplay
• Regression• Checking the results of old bugs fixing
• Mobile game testing see at bitbar (http://bitbar.com/mobile-game-testing-the-importance-and-difference-from-app-testing/)
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI7
Study
• Find and compare definitions of crowdsourcing• Please avoid using only Wikipedia
• What is the difference between crowdsourcing and outsourcing?
• What is crowdsourcing testing?• Give a list of features with descriptions and links
• Explore the types of software testing, compare them with game testing.• Give a table of comparison
• Use all above and sources from the References to build testing strategies of your game.• Describe stages, game testing scenarios
• Prepare testing artifacts like checklists, testing requirements document, software testing plan, game testing plan
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI8
Crowdsourced testing
• The Definitive Guide to Crowdsourced Testing (for Testers and Companies) (http://www.softwaretestinghelp.com/guide-to-crowdsourced-testing/)
• Traditional Testing vs. Crowdsource Testing vs. Freelance Testing (https://www.qualitestgroup.com/blog/test-methodologies-and-philosophies/traditional-testing-vs-crowdsource-testing-vs-freelance-testing/)
• Outsource Vs. Crowdsource Testing (http://www.base36.com/2013/05/outsource-vs-crowdsource-testing/)
• Crowdsourced Testing: A New Way to Do Mobile App Testing (https://dzone.com/articles/crowdsourced-testing-a-new-way-to-do-mobile-app-te)
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI9
Game analyticsMetrics and Statistics
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI10
Basic metrics
• Relationship between User and a Game• DAU (Daily Active Users)
• The number of unique users that start at least one session in a game on any day.• DAU may be called a measure of “stickiness” of a game
• Sessions• Sessions number is increased every time user opens a game.• Brings an information as a ratio with DAU
• DAU/MAU• MAU is Mothly Active Users• DAU/MAU ratio may show how well a game retains users
• Retention• Churn
• How many users who downloaded and may be set up a game are no longer playing?
• Money• Conversion Rate
• measures the percentage of unique users that have made a purchase out of the total number of users during that time period.
• ARPDAU (Average Revenue Per Daily Active User)• Allows to measure how a game performs.• Useful if tracked before and after user completes scenario/campaign/level.
• ARPPU (Average Revenue Per Paying User)• ARPDAU only for paid users
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI11
In-Game Metrics
• Source-Sink-Flow• Source means primary and secondary currency
• Sink the ways of spending these types of currency
• Flow defines the balance between gaining and spending currency
• Start-Fail-Complete (progression metrics)• Start is about starting new levels. It measures the number of times a player
starts a new level.
• Fail measures the number of times when user started but did not complete level.
• Complete measures the number of times when user started and complete level
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI12
Reference Study (Metrics)
• Study the following links:• 15 Metrics All Game Developers Should Know by Heart
(http://www.gameanalytics.com/blog/metrics-all-game-developers-should-know.html)
• Glossary of Metrics (https://unity3d.com/ru/learn/tutorials/topics/analytics/glossary-metrics)
• The Game Life Cycle & Game Analytics: What metrics matter when? (http://www.slideshare.net/TomSente/casualconnect2012-honeytracks-game-lifecycle-kpis)
• How to calculate Mobile Game Retention and LTV using Google Analytics (http://measureofluck.com/calculate-retention-ltv-using-google-analytics/)
• Complete the list below with new metrics and their descriptions
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI13
Reference Study (Analytics Tools)
• Study the following links:• What is the best analytics tool for online mobile game?
(https://www.quora.com/What-is-the-best-analytics-tool-for-online-mobile-game)• Analytics that empower you (http://www.gameanalytics.com/features)• Help your players level up their game
(https://www.helpshift.com/gaming/?utm_source=google&utm_medium=cpc&utm_campaign=Mobile+Gaming&utm_term=%2Bmobile%20%2Bgame%20%2Banalytics&mt=b&ad=108252874846&gclid=CjwKEAiA94nCBRDxismumrL83icSJAAeeETQxdtomAPDzvBEfxGEUImyKbnvLFbzY5AFNHvdJ-HFORoC_yzw_wcB)
• The ten best free analytics tools for mobile app developers: from Google Analytics toFabric, Firebase, Flurry and DevMate (http://www.techworld.com/picture-gallery/apps/eight-best-free-analytics-tools-for-mobile-app-developers-3636407/)
• Prepare a list of analytics tools for mobile games, what metrics they allow to measure.
• Select one tool suitable for the genre and monetizing strategy of your game-project.• Study it carefully and justify its application in your project. • Describe scenarios with screenshots of how will you measure your game metrics.• Give the examples of metrics for your game.
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI14
References (Testing)
• L. Levy, J. Novak, Game Development Essentials: Game QA & Testing (2010).
• C.P. Schultz, R.D. Bryant, Game Testing: All in One (2011)
• J. Hoberg, Difference between software testing and game testing (http://www.gamasutra.com/blogs/JohanHoberg/20140721/221444/Differences_between_Software_Testing_and_Game_Testing.php)
• J. Carrera, Test Planning and Execution in a Mobile Development Project using SCRUM (https://www.scribd.com/document/53989828/Test-Planning-and-Execution-in-a-Mobile-Game-Development-Project-using-SCRUM)
• Game testing methodology (https://www.scribd.com/doc/6555758/Game-Testing-Methodology)
• 17 Strategies for End to End Mobile Testing on both iOS and Android (https://www.ymedialabs.com/17-strategies-for-end-to-end-mobile-testing-on-both-ios-and-android/)
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI15
References (Metrics)
• Mobile Game Metrics: Why Focus on a Single KPI Can Be Dangerous (https://sweetpricing.com/blog/2016/08/mobile-game-metrics/)
• Tracking the metrics that matter in social and mobile gaming (http://www.develop-online.net/opinions/tracking-the-metrics-that-matter-in-social-and-mobile-gaming/0196581)
• A comprehensive list of metrics for free-to-play games (http://www.gamesbrief.com/2014/05/a-comprehensive-list-of-metrics-for-free-to-play-games/)
• Free to play game metrics for beginners (http://www.gdcvault.com/play/1015473/Free-to-Play-Game-Metrics)
• 9 Mobile App KPIs to Know (http://mashable.com/2013/09/04/mobile-app-metrics/#OlOMPY9cGkq1)
• Mobile Game Analytics Example (https://calq.io/docs/resources/track/mobile-game)
• New mobile game statistics every game publisher should know in 2016 (https://www.surveymonkey.com/business/intelligence/mobile-game-statistics/)
• Retention and LTV as Core Metrics to Measure Mobile Game Performance (http://www.gamasutra.com/blogs/IgorMatrofailo/20151221/262214/Retention_and_LTV_as_Core_Metrics_to_Measure_Mobile_Game_Performance.php)
2016-2017Mobile Games Development & Promotion,
Maksimenkova Olga, FCS, SSI16