Making ISTQB Certification Work for YouAs a test manager or an aspiring test

manager

Where are you now?

• You’ve invested in your team• You have a well-educated group– Common knowledge– Common goals– Common language

• They are eager to apply their knowledge• You are eager to get a return on your investment

and leverage the training into high quality products

But….

• But your company doesn’t appreciate testing• Your schedules are too tight• Your budgets are too strict• The code you’re testing is somewhat less than

functional• You must be in the real world!• So, how do you get that return on your

investment?

Bridging Ideal and Reality

• There are 12 main steps that we need to take to leverage good practices and knowledge

• These steps will provide a return where we need it – improved quality and value

• It’s important to set a realistic schedule– “Realistic” depends on your environment and

needs– Remember those pesky projects in progress

How do you make it work?

1. Identify the perceived value

2. Meet the schedules and budgets

3. Improve quality4. Spread the knowledge5. Push testing upstream6. Do the static testing

7. Expand into white box testing

8. Pick, use and improve tools

9. Manage10. Build satisfaction11. Count the numbers12. Show the actual value

1. Identify the Perceived Value

• What is the perceived value of your team?– Do you get told to just work faster, add more people from

the street, work weekends….– Does development get told that?

• The less valued team will get the schedule/budget/manpower cuts

• Look around, ask around, how is your team perceived?

• Understanding the perception tells you what you need to change/fix

Fixing Perceived Value

• Use the test plan to get the stakeholders to understand testing scope and goals

• Engage in the risk analysis and help people understand that there is risk

• Track and publish your metrics– Entry/exit criteria (don’t bend the rules)– Cost of Quality– Defect Detection Percentage– Real numbers from real projects get attention

2. Meet the Schedules and Budgets

• Who’s holding the software when the end dates are missed?

• Who’s charging to the budget when it overruns?

• Who really caused the slip and the overrun?

Fixing Schedules and Budgets

• Track the testing schedule in days allotted, not start/end

• Track the testing budget in money allotted• Let the PM manage the overall project• Track the test coverage (risk / requirements /

code) and map coverage to schedule• Be ready to answer “what could you do if you

had two more weeks to test”

3. Improve Quality

• Goal: Highest quality possible– Obstacles - define “possible”– Time, training, manpower…

• Incoming quality often defines limits of outgoing quality

• “Just do the best you can….” so we can blame you later…

• Time, Time, Time, Time, Time

Fixing Quality

• Incoming quality issues should be dealt with via the entry criteria (test plan)– Unit testing and static analysis are not optional– Objective metrics are used to determine

“testability”• Employ those test strategies your team knows– If time is an issue, use risk-based– Track the residual risk as the time runs out

Fixing Quality

• Break into the arsenal of testing techniques– Reduce tests via equivalence partitioning– Test the combinatorials with your advanced team– Verify code coverage; better code = shorter

testing time– Apply the right techniques to the right situation– Use experience-based to leverage the team’s

experience and knowledge and find gaps

4. Spread the Knowledge

• Time-constrained teams tend to silo• Cross-training isn’t done because there isn’t

time• Faster to use the experts in the area• If someone leaves…. Uh-oh• No time to develop and advance skills

Fixing Knowledge Spreading

• ISTQB training builds a base level of knowledge

• Growing from that common base is easier than starting new

• Find and create opportunities to leverage known techniques

• Build an environment of learning

5. Push Testing Upstream

• Bad code makes for long testing schedules• “Throw it over the fence” is too common• Regardless of lifecycle model, testing has to

start with the requirements• Continuous testing = continuously improved

products• Testing is not “owned” by the test team• The test team may feel like salmon

Fixing Upstream Testing

• Talk to development• Implement code coverage tools to verify unit

testing• Create a plan for real integration testing• Check your entrance/exit criteria– Are they measurable?– Can you make them stick?

6. Do the Static Testing

• Not enough time for reviews• Reviews are not valued • The test team’s input is not valued• Not enough time for reviews• The documents to be reviewed don’t exist…• The test team is busy on other projects• Not enough time for reviews

Fixing Static Testing

• Your team knows how to do high quality reviews• Pick the review type that makes sense• Review the requirements and designs first– Start with a small group if needed– Document the results

• Move into code reviews and static analysis• Be sure to make reviews a standard part of test

documentation as well

7. Expand into White Box Testing

• Does your DDP indicate that you are missing things?

• Do you think your black box coverage is great?• Can’t do white box testing– Not enough time– Not enough skill on the team– Not enough access to the code

Fixing White Box Testing

• Check the defects you are missing – could you have caught them?

• Don’t estimate, use code coverage tools to determine your black box coverage

• Start with targeted white box to address risky areas not covered by black box

• Use your Technical Test Analysts for this

8. Pick, Use and Improve Tools

• Using sub-optimal tools is often a time waster • People get frustrated with poor or awkward

tools• Data can be lost when tools don’t interface• Time spent creating reports could be better

spent• Data dissemination is difficult with some tools

Fixing Tools

• Make sure you have the best tools available using the info you learned on picking tools

• Make sure you are using the tool optimally• Get rid of fields you don’t need and quit annoying

people• Add the critical fields needed for COQ and DDP• Get your TTA’s to code “glueware” to make the

tools work smoothly together• Create and publish reports that prove your value

9. Manage

• Seems obvious, right?• A lot of test managers forget to manage• Managing a team should not be fire fighting• If you are swamped, you need to get your

processes under control• People need to have some of your time• Finding and creating metrics should not be a

significant part of your job

Managing

• Use the test plan to plan projects• Implement the proper control metrics to

monitor and report• Follow up on the escapes and fix the issues• Automate metrics publishing (save time!)• Train your people and create opportunities• Build relationships though risk analysis and

reporting

10. Build Satisfaction

• Is your test team the recipient of comments:– “How did you miss this?”– “Why does testing take so long?”– “The developers should just do the testing”

• Test teams that cannot produce quality products are frustrated and updating resumes

• Hiring and training is a significant investment

Fixing the Satisfaction Problem• Be realistic and let them use their knowledge• Track the metrics so you can explain what was done vs. what

should have been done– Track the risk mitigation– Track the coverage

• Test team’s are happiest when honest metrics are communicated

• Prioritize the testing and test to the priorities– If time is insufficient, the metrics will show it– The team can’t be blamed for what they can’t do

• Deflecting blame is demoralizing; stop the blame

11. Count the Numbers

• Do you have the metrics you need to explain the status of a project?

• Check the defect tracking system – is it getting accurate data?

• Reports and graphs that are too complicated will not be read

• No one will dig for the real numbers – it’s easier to assume they don’t exist

Fixing the Metrics

• Plan to track and use the tracking to plan• Pre-emptive metrics stop the blame game• Your people know why metrics are important– Make sure your tools are tracking the important data

• For DDP, need to know what was found after release as well as what was found in all testing

• For COQ, need to know phase introduced and phase found

– Make reports/dashboards available and readable

12. Show the ACTUAL Value

• The value of testing has to be shown in time and cost savings– Return on investment is required– It’s easy for others to see testing as an expense

with little return• Quality doesn’t just happen, it has to be

planned, implemented and maintained

Fixing the Visible Value

• Use those metrics – Real numbers for real projects show actual value– Risk mitigation– Coverage– Defects found (DDP)– Cost of quality (COQ)

• People should feel good about testing, but should also understand residual risk

• Quality must be embraced by all stakeholders

Long Term

• Continuing education keeps testers current, interested and is a job benefit

• Good testers always want to improve – look to Advanced and Expert levels

• Leverage your team knowledge• Keep your good practices in place – always!– Make sure your processes and tools work for you– Make sure your reporting is consistent and

automated

Goals for the Certified Team

• Improving• Learning• Building• Producing Quality• Performing Consistently• Finding Satisfaction• Maintaining Pride