8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 1/21

A General Survey of the ModQA Org

Robb Nicholas Software Development Engineer in TestApptio

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 2/21

Outline

● Part 1: How to do QA

● 1.1 How to test software

● 1.2 How to make sure your app is ready for releasedocumentation)

● Part 2: How to bootstrap your own QA org

● 2.1 How much testing does your product need● 2.2 Finding and Equipping people for success

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 3/21

Why do you need QA

CCP: Eve Online

Dec. 5th, 2007

Boot.ini debacle

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 4/21

Writing TestsOr, how do you test the user experience?

Manual Testing vs. Automated Tes

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 5/21

● Testing 101: If it says I press the player button, then the game should start

● Map the UI to ensure completeness

● Execute Use Cases

Testing FunctionalityOr, Pressing all the buttons

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 6/21

How do you know you’ve tested everything if you don’tmap?

Mapping your UI

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 7/21

● Executing your use case (The Happy Pat

● Go through each decision point in your u

● Generate test cases from each decision p

Use Cases

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 8/21

Testing CodeEnter The SDET

The Prototypical API Testing Example

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 9/21

Testing CodeMapping variables

● Make a map of your variables, then redu

● Only look at the interesting stuff

● Boundaries are interesting● Zeroes are interesting

● Null pointers and Empty strings are inter

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 10/21

Test Types

Test Type Description

Happy Path Verifying it works

Negative Tests Verifying it fails

Integration Tests Do your subsystems communicate

correctly?

Regression Tests Test your previous failures

End-To-End Tests Can you play a complete game?

Boundary Tests Exploring behavior

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 11/21

Testing Algorithms

Things you should consider when testingalgorithms.

● You’ll need data, and lots of it 

● That data needs Ground Truth Metadat

● Gathering data for your algorithm is a

● Living with lots of data is NOT a blast

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 12/21

How to Tell Your Product is R

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 13/21

The Test Plan

Describe how you plan on testing youapplication

Requirements:● What are you going to test

● What are you NOT going to test

● How you’re going to test the various test categor

● How you know your product is ready to ship

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 14/21

The Test Specification

What are the actual tests you need to

Requirements:

● Tests for each test category● Priority – or how interested am I in the results

● What are you expecting to see

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 15/21

Part IISo you wanna make a Test

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 16/21

How big a test org do you ne

● How do you figure out how much testingneed?

● What kinds of people will you need?

● What are the tools they’ll need to do the

● How do you even know what to do with tonce you have them?

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 17/21

How do you figure out how m

testing you’ll need 

What are the manual tests you’re expec

What are the automated tests you’re exp

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 18/21

What kinds of people do you

And how to hire them

QAE – Quality Assurance Engineer

SDET – Software Development Engineer in

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 19/21

What tools might your testers

Utility Name Description

Bug Tracker databases Track bugs, find regressetc.

Test Harnesses Provides test hooks toautomate manual tests

Code Coverage Shows what lines are bhit by test suites

Profilers Identify performancebottlenecks

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 20/21

Effective use of resources

Embedded vs. Team-Based

The Triage Meeting

The WAR Room

8/12/2019 Robb Nicholas Survey Modern QA

http://slidepdf.com/reader/full/robb-nicholas-survey-modern-qa 21/21

Conclusion

Something Something Somethin

Dark Slide