Quality Assurance Testing - helping to ensure the quality of systems How do we know the software works? Claim - Testing Software Development Today - This is the structure Programmer -> Manager (Decision Maker) <- Tester

Test Iteration 1 Programmer - I'm done Tester - This crashes on startup Manager - Ok, calm down. We'll slip the schedule a little. Try again Test Iteration 2 Programmer - I'm done Tester - There are white screens all over the place. Manager - Now remember, we're all in this together. Try again. Test Iteration 3 Programmer - I'm done Tester - It does the wrong thing in half the test cases. Manager - Ok, calm down. We'll slip the schedule a little. Try again Test Iteration 4 Programmer - I'm done Tester - It still fails some of the test cases we agreed upon. Manager - Ok, Try again, but please hurry up! Test Iteration 5 Programmer - I'm done Tester - Yep, it's done Manager - Oops, the world has changed. Here the new spec.

Software Development Today - Why do we have this structure? Key assumptions 1. Independent Development and Testing 2. Specifications must be explicit 3. Specifications must change 4. Resources are finite so Organizations need decision makers - headship principle

1. Why have Independent Testing?

Testing is basic to all engineering - design of drugs, bridges, planes, software? Why?

Because our ability to predict behavior is imperfect

We need to check our work because we will make mistakes 2. Why do we need explicit Specifications? - Software involves multiple people (at least one programmer and one user) - Usually has multiple programmers and multiple testers - Any Team effort requires mutual understanding of the goal - a specification Otherwise, team members will have different goals in mind 3. Why must Specifications change? - Most software systems are truly novel, not one really know what needs to be built - What people what changes as they use the software - The hardware and software components you work with changes Software specifications are usually in prose, imprecise and out-of-date, so they need updating 4. Why do Organizations need decision makers to handle Finite Resources ? - Time and Labor are limited - Not all wishes can be accommodated, so organizations make tradeoffs. - Most use dollars to express relative cost and competing goals Manager says" We have 2 months, 5 programmers and 2 testers, Here is a prioritized list of features. A feature is finished when it passes all the test for that feature. A programmer does not move on to a new feature until all higher priority features are finished or assigned to another programmer. We start now and ship whatever features are finished 60 days from now."

Quality Assurance Testing - helping to ensure the quality of systems Software Development Today - 1. Many proposals for improving software quality 2. But everyone testers. 50% of the cost of software development is testing 3. Testing is important, let's take a closer look at the practice.

Purpose of Testing - Find Bugs - important ones - Clarify the specification. Good Testers - clarify the specification, this is hard work. There is no hope tools

will automate clarification, this will stay hard work. - Help determine what risks are involved in releasing code Manual Testing - Test cases are a list of instructions - a test script - Someone manually executes the script, doing each action, step by step

1. Type URL 2. Enter username and password 3. click OK 4. Manually record the results - check yes on Software Life Cycle System

- Low Tech, simple to implement, very widespread. Why? Because some test can't be automated (usability testing) and some test could be automated, but are not worth the costs of implementation.

Naive Automated Testing - Idea: Record manual test - playback on demand. - This doesn't work as well as expected. Test recording is usually fragile (breaks in environment if anything changes) Automation Tools cannot generalize. They record exactly what happens. - A hidden strength of manual testing - Since people are doing the testing, they can adapt test to slightly

modified situations. When code changes, test break (change a name of a dialog box) - Maintain automated tests is expensive - cost proportional to the number of tests Improved Automated Testing - Recording and playback is too low level, every test case has name of dialog box - Need abstract test to replace dialog box with string variable name x. X is maintained in just one place, when

it changes, only x need to be updated. This is structured programming, just as hard as any system design. Cucumber and other specification to test routine implement this with behavior driven design

- This is state of the art. But does not necessary validate the entire testing surface, but make sure software behavior is invariant under program modification.

The people with whom I’ve worked that promote BDD or ATDD using tools like Cucumber (or Specflow for .NET) all generally agree it’s a design tool and NOT a quality tool. Converting a requirement or a user story into a testable “check” forces all involved to analyze what’s being requested. When it works well, a Cucumber test is a very informative metric letting the developer know when they’ve coded “just enough” to stop. It is not, however, a tool for verifying quality or even the fitness of the requirement under test anymore than a unit test would be a replacement for an integration or performance test. BDD represents an improvement in product development. But it not “testing or QA.” It is a way for product managers (stakeholders),development and even QA/Test to come together to question what would satisfy the stakeholder of the application. It also brings together these groups at the inception of the product something that, many times, never happens.

What is a Tester's Job? - Find important Bugs (only part of this is subject to automation) - Clarify the spec - no automation tool can do this, people must - Communicate well with developers

Software Development Today - How do we know the software works? 1. We need a specification - most use user stories. 2. Compare the code to the specification 3. Testing is important but expensive, to find bugs and clear up the specification so the product is useable by

others.

Quality Assurance Testing - helping to ensure the quality of systems How can Testers Improve? 1. Know your application - if you test without knowledge of the requirement, you will not know if a program is

functioning as designed. You will not be able to tell if required functionality is missing. So reading and getting a clear understanding of the requirements is a must before testing. After than go through all the features of the application in a systematic way.

2. Think like an end user - Don't think like a technical guy. Think like a customers or the end user. Test your application as an end user would. Think how an end user will be using your application and what technical background he has. Personas help with this.

3. Don't make testing a repetitive task - Tester can sometimes get easily bored doing the same things again and again. The testing job involves repetitive things, so take care to distribute the work among team members. Systematically spread out the testing work within a team, this way the team will get knowledge of the whole product. This will help to find more bugs.

4. Take every test case as an opportunity to learn - Without a tester notebook for can't review what you've done and learn from it. You cannot trace what you have tested and what problems occurred. What can you keep in a Tester's Notebook? Testing heuristics and test ideas, collections of observations while testing, logs of testing sessions, meeting notes, one page presentations, mind maps and diagrams, action ideas, reminders and random notes. Some use index cards for bug reports to keep a easily accessible record of bugs they have filed.

What should Testers Learn? 1. Learn to write formal test cases/ test plans - a test case writing guide help keep all test cases in a format

that is easy for all Testers to follow. It list the steps for testing and checks for success to allow test cases to be placed in a Software Life Cycle Tool.

2. Learn session-based exploratory testing - these are strategies for testing that let you systematically work your way through the application. In session-based exploratory testing, a session is the basic testing work unit . What we call a session is an uninterrupted block of reviewable, chartered test effort. By “chartered,” we mean that each session is associated with a mission—what we are testing or what problems we are looking fo

3. Learn new technologies - having a solid foundation in technology will bring new testing ideas. This will help you better understand what the testing surface is.