SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R....

23
SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s most important assets. The tasks of a manager are essentially people oriented. Unless there is some understanding of people, management will be unsuccessful. Software engineering is primarily a cognitive activity. Cognitive limitations effectively limit the software process. Misconception and perception about testing 1. Testing is not technically challenging This argument is true about twenty years ago when most of the testing was manual and the products were somewhere simplistic. Normally development engineers tend to focus on specific modules. But, now testers require a holistic understanding of the entire product rather than just a single module. Suppose if the product is more complex, testing requires deep understanding of multiple domains. In the early 1990s, most people sought to be expert in C, which moved to C++ and then to java. Similarly, interms of platforms, shifted from mainframe to client server networking to network centric computing. About ten years ago, people in the testing functions found very little parallel to these kind of “resume enriching”. In fact, now a day the use of testing tools requires programming in languages remarkably similar to those that developers use. And also the automation tests can prove to be more challenging than even developing the code for the product. Testing is a destructive process (in the sense of finding defects in a product), there are more opportunities for out-of-box thinking. In view of all of the above, testing offers sufficient technical challenges for an interested professional and we can say “there is testing in all development and development in all testing”. 2. Testing does not provide me a career path or growth Testing is not a devil and development is not an angel: opportunities abound equally in testing and development. 3. I am put in testing what is wrong with me? Filling up positions for testing should not be treated as a second string function. People are sometimes made to feel that they are in testing because they could not fit in anywhere else. But, if a person is not suitable for development, for the same or similar reason: he or she may not be suitable for testing. A person should be assigned to testing only when he or she has the right aptitude and attitude for testing.

Transcript of SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R....

Page 1: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 1

4.1. People and organizational issues in Testing

People are an organization’s most important assets. The tasks of a manager are essentially people

oriented. Unless there is some understanding of people, management will be unsuccessful.

Software engineering is primarily a cognitive activity. Cognitive limitations effectively limit the

software process.

Misconception and perception about testing

1. Testing is not technically challenging

This argument is true about twenty years ago when most of the testing was manual and

the products were somewhere simplistic. Normally development engineers tend to focus on

specific modules. But, now testers require a holistic understanding of the entire product rather

than just a single module. Suppose if the product is more complex, testing requires deep

understanding of multiple domains.

In the early 1990s, most people sought to be expert in C, which moved to C++ and then

to java. Similarly, interms of platforms, shifted from mainframe to client server networking to

network centric computing. About ten years ago, people in the testing functions found very little

parallel to these kind of “resume enriching”. In fact, now a day the use of testing tools requires

programming in languages remarkably similar to those that developers use. And also the

automation tests can prove to be more challenging than even developing the code for the product.

Testing is a destructive process (in the sense of finding defects in a product), there are

more opportunities for out-of-box thinking.

In view of all of the above, testing offers sufficient technical challenges for an interested

professional and we can say “there is testing in all development and development in all

testing”.

2. Testing does not provide me a career path or growth

Testing is not a devil and development is not an angel: opportunities abound equally in

testing and development.

3. I am put in testing – what is wrong with me?

Filling up positions for testing should not be treated as a second string function. People

are sometimes made to feel that they are in testing because they could not fit in anywhere else.

But, if a person is not suitable for development, for the same or similar reason: he or she may not

be suitable for testing. A person should be assigned to testing only when he or she has the right

aptitude and attitude for testing.

Page 2: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 2

Appropriate recognition should be given to the Engineers who participate in activities

such as testing, maintenance and documentation.

4. These folk are my adversaries

The main function of testing is to find defects in the products; it is easy for an adversary

attitude between the testing team and the development team. So, testing and developing teams

should reinforce each other.

5. Testing is what I can do in the end if I get time

Testing is not an end activity; it happens throughout and continues even after the release.

6. Testing is only destructive

Test engineers not only say “something doesn’t work”, they also point out what works in

the product. The job of test engineers is not only to find defects but also prevent defects by

proactively reviewing this specifications, design, architecture and code.

Successful people are those who take initiative, volunteer, experiment and share their

learning with others. Testing people should be able to respond quickly to changing priorities,

delayed handover from development.

Role of Education System

The perception and misconceptions cannot be corrected by each organization

individually. Higher level actions are needed and these actions pertain to the entire ecosystem

covering the education system, senior management and the community as a whole. The

education system does not place sufficient emphasis on testing. Here are the facts about what

prevails in most universities.

There are formal core courses on programming but few universities offer core courses

on software testing.

There are “lab courses” for various development tools but none or very few for

common testing tools.

More projects done as a part of the curriculum never ask for neither test plans nor

looking at testing effectiveness. The complete weightage is on coding.

Most courses and projects reward individual performance and present very little

opportunity for team work, which is essential for the success of development and test

engineers.

Role of Senior Management

Page 3: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 3

The senior management of an organization plays a vital role in the development of test

professionals. It is simply not enough to use words like “quality is our top concern” or “people

are our top priority”. This commitment has to be translated into visible actions. It is the

responsibility of the management for recognition and giving reward for the top performers

including testers.

Role of community

Test Engineers need not be just followers of technology – they can take an active part in

shaping and using new technology. Sometimes test engineers are under pressure to learn the

technology and test the product at the same time. Learning the technology and testing the product

at the same time may lead to ineffective testing. This scenario must change. Test Engineers must

understand the new technology areas proactively before they are implemented in a product.

Responsibilities of a senior test engineer

Following the test processes for executing tests, maintaining tests and so on

Filing high quality defects

Categorizing the defects

Adhering to schedules specified

Developing high quality documentation

Generation of metrics related to testing

Responsibilities of a Test Lead

Review of test cases, test design

Planning a test strategy and test plan

Allocating tasks to individuals

Monitoring tasks given to individuals

Making technology and tool choices for module testing

Interacting with development team for debugging

Analysis of metrics

Overall responsibility for test quality at a module level

*****

4.2. Organization structures for Testing Teams

A number of organizational structures are possible, each having its own positives and

negatives. Figure shows a structure often used by small (fewer than 10) project teams. In this

structure, the test group reports into the Development Manager, the person managing the work

of the programmers.

Page 4: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 4

The Development Manager's goal is to have his team develop software. Testers reporting

bugs just hinder that process. Testers doing their job well on one side make the programmers

look bad on the other. If the manager gives more resources and funding to the testers, they'll

probably find more bugs, but the more bugs they find, the more they'll crimp the manager's

goals of making software.

Despite these negatives, this structure can work well if the development manager is very

experienced and realizes that his goal isn't just to create software, but to create quality software.

Such a manager would value the testers as equals to the programmers. This is also a very good

organization for communications flow. There are minimal layers of management and the testers

and programmers can very efficiently work together.

Figure shows another common organizational structure where both the test group and the

development group report to the manager of the project. In this arrangement, the test group

often has its own lead or manager whose interest and attention is focused on the test team and

their work. This independence is a great advantage when critical decisions are made regarding

the software's quality. The test team's voice is equal to the voices of the programmers and other

groups contributing to the product.

Page 5: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 5

• In this structure, the test group that reports to executive management has the most

independence, the most authority and the most responsibility.

• The independence that this group holds allows them to set standards and guidelines,

measure the results, and adopt processes that span multiple projects.

• A corporate quality standard that works well on database software might not work well

when applied to a computer game. To be effective, this independent quality organization

must find ways to work with all the projects they deal with and temper their enthusiasm

for quality with the practicality of releasing software.

• In software development and testing, one size doesn't necessarily fit all, and what works

for one team may not work for another. There are, however, some common metrics that

can be used to measure, and guidelines that can be followed, that have been proven to

work across different projects and teams for improving their quality levels.

*****

4.3. Testing Services

Testing and Deubgging Goals and Polices

A Goal can be described as:

• A statement of an accomplishment that an individual or an organization wants to achieve.

Goal statement – Individual/ group/ organization wants to make improvements.

• At the top level are General organizational goals - Ex: Goal of VCET

• Individual projects have specific goals. It usually reflects organizational goals.

• Personal Goals – Each individual in an organization has a set of goals for self improvement.

So that they can effectively contribute to the project.

• Goal statements can express expectations in quantitative terms or be more general in nature.

Example:

• 100% testing activities are planned. (Quantitative)

Page 6: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 6

• The degree of automation of regression testing is increased from 50% to 75% over the

next 3 years. (Quantitative)

• Testing activities are performed by a dedicated testing group.

• Testing group members have at least a bachelor- level degree and have taken a formal

course in software testing.

Note: Quantitative goals are measurable. It’s useful to evaluate progress towards achieving the

goal.

• In testing domain, goal statement should provide a high level vision of what testing is to

accomplish in the organization with respect to quality of process and product.

• In addition to general testing goal statements, lower level goal statement should be

developed for all levels of testing.

• Policy provides the vision and framework for decision making. It should be properly

documented and it should be available for all interested parties.

• Organization website – Disseminate the goals in website

• A policy statement should be formulated by a team consisting of upper management,

executive personnel and technical staff.

• Testing policy statements reflect, integrate and support achievements of testing goals. –

Which will be useful for increasing the quality and customer satisfaction.

• Test policies also provide high level guidance as to how testing is to be done in the

organization, how its effectiveness will be evaluated, who will be responsible and what

choices of resources are possible.

Example: How to test, what to test and who will test.

• Policies are not written in stone- If organization grows in maturity its policies will

change.

EXAMPLE: Testing policy:

• Our organization (Company Name) realizes that testing is an important component of the

software development process and has a high impact on software quality and the degree

of customer satisfaction.

• Delivering software of the highest quality is our company goal. The presence of defects

has a negative impact on software quality.

• Defect affects the correctness, reliability and usability of a software product, thus

rendering unsatisfactory to the client.

• Testing activities include traditional execution of the developing software, as well as

reviews of the software deliverables produced at all stages of the life cycle.

• The aggregation of all testing activities performed in a systematic manner supported by

organization policies, procedures and standards constitutes the testing process.

• A set of testing standards must be available to all interested parties on an intra

organizational website.

Page 7: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 7

• Standards contain – test related documents, prescribed templates, methods, procedures and

tools.

*****

4.4. Test Planning

A test plan is a detailed document that outlines the test strategy, Testing objectives,

resources (manpower, software, hardware) required for testing, test schedule, Test

Estimation and test deliverables.

The test plan serves as a blueprint to conduct software testing activities as a defined

process which is minutely monitored and controlled by the test manager.

Making Test Plan has multiple benefits

• Test Plan helps us determine the effort needed to validate the quality of the application

under test.

• Help people outside the test team such as developers, business managers,

customers understand the details of testing.

Page 8: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 8

• Test Plan guides our thinking. It is like a rule book, which needs to be followed.

• Important aspects like test estimation, test scope, Test Strategy are documented in Test

Plan, so it can be reviewed by Management Team and re-used for other projects.

Making a Test Plan is the most important task of Test Management Process. Follow the seven

steps below to create a test plan as per IEEE 829

• Analyze the product

• Design the Test Strategy

• Define the Test Objectives

• Define Test Criteria

• Resource Planning

• Plan Test Environment

• Schedule & Estimation

• Determine Test Deliverables

Steps for creating Test Plan

1. Analyze the Product

• How can you test a product without any information about it? The answer

is Impossible. You must learn a product thoroughly before testing it.

• The product under test is Guru99 banking website. You should research clients and the

end users to know their needs and expectations from the application

• Who will use the website?

• What is it used for?

• How will it work?

• What are software/ hardware the product uses?

Page 9: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 9

2. Develop Test Strategy

• Test Strategy is a critical step in making a Test Plan. A Test Strategy document is a high-

level document, which is usually developed by Test Manager. This document defines:

• The project’s testing objectives and the means to achieve them

• Determines testing effort and costs

• Back to your project, you need to develop Test Strategy for testing that banking website.

You should follow steps below:

2.1. Define scope of testing

1. Before the start of any test activity, scope of the testing should be known. You must think

hard about it.

2. The components of the system to be tested (hardware, software, middleware, etc.) are

defined as "in scope"

3. The components of the system that will not be tested also need to be clearly defined as being

"out of scope."

4. Defining the scope of your testing project is very important for all stakeholders. A precise

scope helps you Give everyone a confidence & accurate information of the testing you

are doing.

5. All project members will have a clear understanding about what is tested and what is not?

2.1.1.How do you determine scope your project?

• To determine scope, you must –

• Precise customer requirement

• Project Budget

• Product Specification

• Skills & talent of your test team

• Now should clearly define the "in scope" and "out of scope" of the testing.

Page 10: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 10

• As the software requirement specification, the project Guru99 Bank only focus on testing

all the functions and external interface of website Guru99 Bank (in scope testing)

• Nonfunctional testing such as stress, performance or logical database currently will not

be tested. (out of scope)

2.1.2. Problem Scenario

The customer wants you to test his API. But the project budget does not permit to do so.

In such a case what will you do?

Well, in such case you need to convince the customer that API testing is extra work and will

consume significant resources. Give him data supporting your facts. Tell him if API testing

is included in-scope the budget will increase by XYZ amount.

The customer agrees and accordingly the new scopes, out of scope items are

In-scope items: Functional Testing, API Testing

Out of scope items: Database Testing, hardware & any other external interfaces

2.2. Identifying Testing Type

A Testing Type is a standard test procedure that gives an expected test outcome.

Each testing type is formulated to identify a specific type of product bugs. But, all Testing

Types are aimed at achieving one common goal “Early detection of all the defects before

releasing the product to the customer”

The commonly used testing types are described as following figure

Page 11: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 11

• There are tons of Testing Types for testing software product. Your team cannot

have enough efforts to handle all kind of testing. As Test Manager, you must

set priority of the Testing Types:

• Which Testing Types should be focused for web application testing?

• Which Testing Types should be ignored for saving cost?

2.3. Documents risks and issues

• Risk is future’s uncertain event with a probability of occurrence and a potential for loss.

When the risk actually happens, it becomes the ‘issue’.

Page 12: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 12

2.4.Create Test Logistics

In Test Logistics, the Test Manager should answer the following questions:

• Who will test?

• When will the test occur?

• Who will test?

You may not know exact names of the tester who will test, but the type of tester can be defined.

• To select the right member for specified task, you have to consider if his skill is qualified

for the task or not, also estimate the project budget. Selecting wrong member for the task

may cause the project to fail or delay.

• Person having the following skills is most ideal for performing software testing:

• Ability to understand customers point of view

• Strong desire for quality

• Attention to detail

• Good cooperation

When will the test occur?

• Test activities must be matched with associated development activities.

• You will start to test when you have all required items shown in following figure:

3. Test Objectives

• Test Objective is the overall goal and achievement of the test execution. The objective of

the testing is finding as many software defects as possible; ensure that the software under

test is bug free before release.

• To define the test objectives, you should do the following steps:

1. List all the software features (functionality, performance, GUI…) which may need to

test.

2. Define the target or the goal of the test based on above features.

For Example for banking sector

Page 13: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 13

Based on above features, you can define the Test Objective of the project Guru99 as following:

• Check that whether website Guru99 functionality (Account, Deposit…) is working as

expected without any error or bugs in real business environment

• Check that the external interface of the website such as UI is working as expected and &

meet the customer need.

• Verify the usability of the website. Are those functionalities convenient for user or not?

4. Define Test criteria

• Test Criteria is a standard or rule on which a test procedure or test judgment can be based.

There’re 2 types of test criteria as following

Suspension Criteria

• Specify the critical suspension criteria for a test. If the suspension criteria are met during

testing, the active test cycle will be suspended until the criteria are resolved.

• Example: If your team members report that there are 40% of test cases failed, you

should suspend testing until the development team fixes all the failed cases.

Page 14: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 14

Exit Criteria

• It specifies the criteria that denote a successful completion of a test phase. The exit criteria

are the targeted results of the test and are necessary before proceeding to the next phase

of development. Example: 95% of all critical test cases must pass.

• Some methods of defining exit criteria are by specifying a targeted run rate and pass rate.

• Run rate is ratio between number test cases executed/total test cases of test specification.

For example, the test specification has total 120 TCs, but the tester only executed 100

TCs, So the run rate is 100/120 = 0.83 (83%)

• Pass rate is ratio between numbers test cases passed / test cases executed. For example, in

above 100 TCs executed, there’re 80 TCs that passed, so the pass rate is 80/100 = 0.8

(80%).

• This data can be retrieved in Test Metric documents.

• Run rate is mandatory to be 100% unless a clear reason is given.

• Pass rate is dependent on project scope, but achieving high pass rate is a goal.

• Example: Your Team has already done the test executions. They report the test result to

you, and they want you to confirm the Exit Criteria.

Page 15: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 15

• In above case, the Run rate is mandatory is 100%, but the test team only completed 90%

of test cases. It means the Run rate is not satisfied, so do NOT confirm the Exit Criteria

5. Resource Planning

Resource plan is a detailed summary of all types of resources required to complete

project task. Resource could be human, equipment and materials needed to complete a

project

6. Plan Test Environment

Test Environment

A testing environment is a setup of software and hardware on which the testing team is

going to execute test cases. The test environment consists of real business and user environment,

as well as physical environments, such as server, front end running environment.

7. Schedule & Estimation

Test estimation technique is to estimate the effort to complete the project. In the Test Estimation

phase, the whole project has broken into small tasks and adds the estimation for each task as

below

Task Members Estimate effort

Create the test specification Test Designer 170 man-hour

Perform Test Execution Tester, Test Administrator 80 man-hour

Test Report Tester 10 man-hour

Test Delivery 20 man-hour

Total 280 man-hour

Schedule

8. Test Deliverables

Page 16: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 16

Test Deliverables is a list of all the documents, tools and other components that has to be

developed and maintained in support of the testing effort. There are different test deliverables at

every phase of the software development lifecycle.

Test deliverables are provided before testing phase.

Test plans document.

Test cases documents

Test Design specifications.

Test deliverables are provided during the testing

Test Scripts

Simulators.

Test Data

Test Traceability Matrix

Error logs and execution logs.

Test deliverables are provided after the testing cycles is over.

Test Results/reports Defect Report

Installation/ Test procedures guidelines

Release notes

*****

4.5. TEST REPORTING

Testing requires constant communication between the test team and other teams (like the

development team). There are two types of reports or communication that are required.

1. Test Incident reports 2. Test summary reports

1. Test Incident Reports

A test incident report is a communication that happens through the testing cycle as and

when defects are encountered. Every defect has a unique ID and this is used to identify the

incident. The high impact test incidents (defects) are highlighted in the test summary report.

2. Test cycle Reports

Page 17: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 17

A test cycle entails planning and running certain tests in cycles, each cycle using a

different build of the product. A test cycle report, at the end of each cycle gives

1. A summary of the activities carried out during those cycle.

2. Defects that were uncovered during that cycle, based on their severity and impact.

3. Progress from the previous cycle to the current cycle interms of defects fixed.

4. Any variations observed in effort or schedule.

Test Summary Report

The final step in a test cycle is to recommend the suitability of a product for release. A

report that summarizes the results of a test cycle is the test summary report.

Types

1. Phase wise test summary – which is produced at the end of every phase.

2. Final test summary reports – it is also called release test report

A summary report should present

1. A summary of the activities carried out during the test cycle or phase.

2. Variance of the activities carried out form the activities planned.

The tests that were planned to be run but could not be run ( with reasons)

Modifications to tests

Additional tests that were run

Difference in effort and scheduling

3. Summary of results should include

Test that failed, with any root cause

Severity of impact of the defects

4. Comprehensive assessment and recommendation for release should include

“Fit for release “ assessment and

Recommendation of release

*****

4.6. The Role of the Three Critical Groups in Testing Planning and Test Policy

Development

Three groups were identified as critical players in the testing process

Managers, Developers/Testers, and Users/Clients

In TMM terminology they are called the three critical views (CV)

Each group views the testing process from a different perspective that is related to their

particular goals, needs, and requirements.

The developers/testers work with client/user groups on quality-related activities and tasks

that concern user-oriented needs.

Page 18: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 18

The focus is on soliciting client/user support, consensus, and participation in activities

such as requirements analysis, usability testing, and acceptance test planning.

Upper Management Goals and Policy

1. Provide adequate resources and funding to form the committees

2. Support the recommendations and policies of the committee by:

Distributing testing/debugging goal/policy documents to project managers, developers, and other

interested staff,

Appointing a permanent team to oversee compliance and policy change making.

3. Ensure that the necessary training, education, and tools to carry out defined

testing/debugging goals is made available.

• Assign responsibilities for testing and debugging.

As representatives of the technical staff developers must ensure that the policies reflect best

testing practices, are implementable, receive management support, and support among technical

personnel.

The activities, tasks, and responsibilities for the developers/testers include:

Page 19: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 19

Working with management to develop testing and debugging policies and goals.

Participating in the teams that oversee policy compliance and change management.

Familiarizing themselves with the approved set of testing/debugging goals and policies,

keeping up-to-date with revisions, and making suggestions for changes when appropriate.

When developing test plans, setting testing goals for each project at each level of test that

reflect organizational testing goals and policies.

Carrying out testing activities that are in compliance with organizational policies.

The goals and policies of users and clients reflect the organizations efforts to ensure

customer/client/user satisfaction.

Upper management supports this goal by:

Establishing an organization wide test planning committee with funding.

Ensuring that the testing policy statement and quality standards support test planning with

commitment of resources, tools, templates, and training.

*****

4.7. Test Policy

A Test Policy is a high level document and is at the top of the hierarchy of the Test

Documentation structure.

The purpose of the Test Policy document is to represent the testing philosophy of the

company as a whole and to provide a direction which the testing department should adhere to

and follow. It should apply to both new projects and maintenance work.

Setting an appropriate test policy by senior managers provides a robust framework within

which testing practitioners can then operate. This will help to ensure the maximization of the

strategic value inherent in every project.

Contents of a Test Policy Document

1. Definition of Testing

Organizations need to be clear why they are testing. This will influence the remainder of

the policy document and also the detailed testing techniques that are selected by test managers at

the programme and project level.

Page 20: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 20

From the understanding of why testing is required it is possible to specify what the

purpose of testing is within the organisation. Without this fundamental linkage the test effort is

destined to fail.

Example: “ensuring the software fulfills its requirements”

2. Description of the test process

It is vital to establish a solid view towards the test process. We should address questions

like, which phases and subtasks will the test process include. Which roles will be involved and

the document structure associated with each tasks, as well as what test levels need to be

considered.

Example: “all test plans are written in accordance with company policy”

3. Test Evaluation

How are we going to evaluate the results of testing, what measures will we use to ensure

test effectiveness in the project?

Example: “effect on business of finding a fault after its release”

4. Quality Level to be achieved:

Which quality criteria are going to be tested and which quality level is the system

required achieving prior to its release with regards to these criteria?

Example: “no outstanding high severity faults prior to products release”

5. Approach to Test Process Improvement

How often and when are we going to assess the usefulness of the current processes in

place and what elements need improving and techniques that shall be used to improve the

processes.

Example: “project review meetings to be held after project completion”

*****

4.8. Introducing the test specialist

Page 21: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 21

Software testing specialists help develop the test plan and evaluate the quality of software

components. Their role is to detect major software flaws (e.g. bugs, errors, failures, breakdowns,

risks) in order to fix them and ensure the performance of the developed systems. Testing

specialists have a big responsibility - they inform the testing manager of all the improvements to

be made. The work of an entire team of software designers and developers depends on their

analysis.

The staff members who possess the following necessary skills and motivation are called

test specialists.

Development and application of test-related standards;

• Participating in requirements, design, and code reviews;

• Test planning

• Test design

• Test execution;

• Test measurement;

• Test monitoring (tasks, schedules, and costs);

• Defect tracking, and maintaining the defect repository;

• Acquisition of test tools and equipment;

• Identifying and applying new testing techniques, tools, and methodologies;

• Mentoring and training of new test personnel;

• Test reporting.

4.8.1. Skills Needed by a Test Specialist

Test specialist must have:

• Organizational and planning skills;

• The ability to keep track of, and pay attention to, details;

• The determination to discover and solve problems;

• The ability to work with others and be able to resolve conflicts;

• The ability to mentor and train others;

• The ability to work with users and clients;

• Strong written and oral communication skills;

• The ability to work in a variety of environments;

• The ability to think creatively

The first three skills are necessary because testing is detail and problem oriented. In addition,

testing involves policymaking, knowledge of different types of application areas, planning, and

the ability to organize and monitor information, tasks, and people.

Testing also requires inter actions with many other engineering professionals such as

project managers, developers, analysts, process personal, and software quality assurance staff.

Test professionals often interact with clients to prepare certain types of tests, for example

acceptance tests. Testers also have to prepare test-related documents and make presentations.

Page 22: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 22

Training and mentoring of new hires to the testing group is also a part of the tester’s job. In

addition, test specialists must be creative, imaginative, and experiment oriented.

They need to be able to visualize the many ways that a software item should be tested,

and make hypotheses about the different types of defects that could occur and the different ways

the software could fail.

On the technical level testers need to have:

• An education that includes an understanding of general software engineering principles,

practices, and methodologies.

• Strong coding skills and an understanding of code structure and behavior;

• A good understanding of testing principles and practices;

• A good understanding of basic testing strategies, methods, and techniques;

• The ability and experience to plan, design, and execute test cases and test procedures on

multiple levels (unit, integration, etc.);

• A knowledge of process issues

• Knowledge of how networks, databases, and operating systems are organized and how they

work

• A knowledge of configuration management;

• A knowledge of test-related documents and the role each documents plays in the testing

process;

• The ability to define, collects, and analyzes test-related measurements;

• The ability, training, and motivation to work with testing tools and equipment;

• A knowledge of quality issues.

Testers must have knowledge of both white and black box techniques and methods and

the ability to use them to design test cases. Organizations need to realize that this knowledge is a

necessary prerequisite for tool use and test automation. Testers also need to understand quality

measures such as reliability, maintainability, usability, and how to test for them.

Finally, testers should have some knowledge of the problem domain for which the

software has been written. This knowledge, for example, can help them to understand the domain

vocabulary, domain operations, and domain requirements and constraints.

4.8.2. Building a Testing Group

The major activities required to manage a project and a process are: Organizing, Staffing,

and Directing. These apply to managing the testing process as well.

Staffing activities include filling positions, assimilating new personnel, education and

training, and staff evaluation.

Directing includes providing leadership, building teams, facilitating communication,

motivating personnel, resolving conflicts, and delegating authority.

Page 23: SOFTWARE TESTING – UNIT 4 - WordPress.com · SOFTWARE TESTING – UNIT 4 Prepared by Dr. R. Kavitha Page 1 4.1. People and organizational issues in Testing People are an organization’s

SOFTWARE TESTING – UNIT 4

Prepared by Dr. R. Kavitha Page 23

Organizing includes selecting a organizational structures, creating positions, defining

responsibilities, and delegating authority.

Steps in forming a test group

Step 1: Support from upper management to establish a test group

Step 2: Forming test group

Step 3: Deciding the educational and skill levels required for each testing position.

Step 4: Develop Job description

Step 5: Interview Process for recruiting staff

*****