The Software Development Life Cycle (SDLC) - Zeta Web viewThis section of the document will describe...

Table of ContentsIntroduction to the Software Development Life Cycle (SDLC).............................4

The SDLC Waterfall.........................................................................................4Other SDLC Models..........................................................................................7Spiral Lifecycle................................................................................................7V-Model...........................................................................................................8Verification and Validation..............................................................................9The V & V process...........................................................................................9Verification Vs Validation................................................................................9

Introduction to Software Testing.......................................................................11Different approaches to software Testing......................................................11Levels of Testing...........................................................................................11Types of Testing............................................................................................11

Test Techniques................................................................................................17Black Box Testing..........................................................................................17White Box Testing.........................................................................................18Code Complexity...........................................................................................25

Bug Life Cycle...................................................................................................27Test cases.........................................................................................................29

What’s a Test Case?......................................................................................29What is Quality Writing?................................................................................29Key Points to Remember...............................................................................29Use Cases......................................................................................................30Writing Test Cases........................................................................................31

Software Inspection..........................................................................................33Steps Involved in Inspection..........................................................................33Participants and their Roles in Inspection......................................................36Reviews.........................................................................................................38Walkthroughs................................................................................................39Benefits of Inspection....................................................................................40

Unit Testing /Module Testing.............................................................................41The need for Levels of Testing......................................................................41Objectives of Unit Testing.............................................................................41Unit Test Procedures.....................................................................................44Tools for Unit Testing....................................................................................45

Integration testing............................................................................................46Objective of Integration Testing....................................................................46Advantage of Integration Testing..................................................................46Integration Strategies....................................................................................47

System Testing.................................................................................................55Why System Testing?....................................................................................55Who does System Testing?...........................................................................55Types of System Tests..................................................................................55Exploratory Testing.......................................................................................55Why Exploratory Testing?..............................................................................56

Testing Process.................................................................................................57Fundamental Test Process............................................................................58Test Planning.................................................................................................59Test Design...................................................................................................59Test Execution...............................................................................................59Test Recording..............................................................................................59Checking for Test Completion.......................................................................60What's a 'test plan'?......................................................................................61Test plan identifier........................................................................................61What should be done after a bug is found?...................................................70What is 'configuration management'?...........................................................71

Test Automation................................................................................................73Scope of Automation.....................................................................................73

© 2006 Zeta Cyber Solutions (P) Limited

Tool Support for Life-Cycle testing................................................................73Test Automation Framework.........................................................................75Benefits of Automation..................................................................................77Limitation of Test Automation.......................................................................78Manual vs. Automated...................................................................................79

Testing Terms - Glossary...................................................................................80Points to ponder................................................................................................85References......................................................................................................127

Books..........................................................................................................127Website Links..............................................................................................127


Introduction to the Software Development Life Cycle (SDLC)

This section of the document will describe the Software Development Life Cycle (SDLC) for small to medium database application development efforts. This chapter presents an overview of the SDLC, alternate lifecycle models, and associated references. This chapter also describes the internal processes that are common across all stages of the SDLC and describes the inputs, outputs, and processes of each stage.

The SDLC Waterfall Small to medium database software projects are generally broken down into six stages:

The relationship of each stage to the others can be roughly described as a waterfall, where the outputs from a specific stage serve as the initial inputs for the following stage.To follow the waterfall model, one proceeds from one phase to the next in a purely sequential manner. For example,

After completing the “Project Planning” phase, one will be completing the "requirements definitions" phase.

When and only when the requirements are fully completed, one proceeds to design. This design should be a plan for implementing the requirements given.


When and only when the design is fully completed, an implementation of that design is made by coders. Towards the later stages of this implementation phase, disparate software components produced by different teams are integrated.

After the implementation and integration phases are complete, the software product is tested and debugged; any faults introduced in earlier phases are removed here.

Then the software product is installed, and later maintained to introduce new functionality and remove bugs.

Thus the waterfall model maintains that one should move to a phase only when it’s proceeding phase is completed and perfected. Phases of development in the waterfall model are thus discrete, and there is no jumping back and forth or overlap between them.

The central idea behind the waterfall model - time spent early on making sure that requirements and design are absolutely correct is very useful in economic terms (it will save you much time and effort later). One should also make sure that each phase is 100% complete and absolutely correct before proceeding to the next phase of program creation.

It is argued that the waterfall model in general can be suited to software projects which are stable (especially with unchanging requirements) and where it is possible and likely that designers will be able to fully predict problem areas of the system and produce a correct design before implementation is started.

The waterfall model also requires that implementers follow the well made, complete design accurately, ensuring that the integration of the system proceeds smoothly.

The waterfall model however is argued by many to be a bad idea in practice, mainly because of their belief that it is impossible to get one phase of a software product's lifecycle "perfected" before moving on to the next phases and learning from them (or at least, the belief that this is impossible for any non-trivial program). For example clients may not be aware of exactly what requirements they want before they see a working prototype and can comment upon it - they may change their requirements constantly, and program designers and implementers may have little control over this.


If clients change their requirements after a design is finished, that design must be modified to accommodate the new requirements, invalidating quite a good deal of effort if overly large amounts of time have been invested into the model.

In response to the perceived problems with the "pure" waterfall model, many modified waterfall models have been introduced namely Royce's final model, sashimi model, and other alternative models. These models may address some or all of the criticism of the "pure" waterfall model. There are other alternate SDLC models such as “Spiral” and “V” which have been explained in the later part of this chapter.

After the project is completed, the Primary Developer Representative (PDR) and Primary End-User Representative (PER), in concert with other customer and development team personnel develop a list of recommendations for enhancement of the current software.

Prototypes

The software development team, to clarify requirements and/or design elements, may generate mockups and prototypes of screens, reports, and processes. Although some of the prototypes may appear to be very substantial, they're generally similar to a movie set: everything looks good from the front but there's nothing in the back.

When a prototype is generated, the developer produces the minimum amount of code necessary to clarify the requirements or design elements under consideration. No effort is made to comply with coding standards, provide robust error management or integrate with other database tables or modules. As a result, it is generally more expensive to retrofit a prototype with the necessary elements to produce a production module than it is to develop the module from scratch using the final system design document. For these reasons, prototypes are never intended for business use, and are generally crippled in one way or another to prevent them from being mistakenly used as production modules by end-users.

Allowed Variations

In some cases, additional information is made available to the development team that requires changes in the outputs of previous stages. In this case, the development effort is usually suspended until the changes can be reconciled with


the current design, and the new results are passed down the waterfall until the project reaches the point where it was suspended.

The PER and PDR may, at their discretion, allow the development effort to continue while previous stage deliverables are updated in cases where the impacts are minimal and strictly limited in scope. In this case, the changes must be carefully tracked to make sure all their impacts are appropriately handled.


Other SDLC ModelsApart from the waterfall model other popular SDLC modules are the “Spiral” model and “V” model which have been explained in this section.

Spiral Lifecycle

The spiral model starts with an initial pass through a standard waterfall lifecycle, using a subset of the total requirements to develop a robust prototype. After an evaluation period, the cycle is initiated again, adding new functionality and releasing the next prototype. This process continues with the prototype becoming larger and larger with each iteration, hence the “spiral.”

The Spiral model is used most often in large projects and needs constant review to stay on target. For smaller projects, the concept of agile software development is becoming a viable alternative. Agile software development tends to be rather more extreme in their approach than the spiral model.


http://en.wikipedia.org/wiki/Extreme_programming

http://en.wikipedia.org/wiki/Agile_software_development

The theory is that the set of requirements is hierarchical in nature, with additional functionality building on the first efforts. This is a sound practice for systems where the entire problem is well defined from the start, such as modeling and simulating software. Business-oriented database projects do not enjoy this advantage. Most of the functions in a database solution are essentially independent of one another, although they may make use of common data. As a result, the prototype suffers from the same flaws as the prototyping lifecycle. For this reason, the software development teams usually decide against the use of the spiral lifecycle for database projects.

V-Model

The V-model was originally developed from the waterfall software process model. The four main process phases – requirements, specification, design and Implementation – have a corresponding verification and validation testing phase. Implementation of modules is tested by unit testing, system design is tested by Integration testing, system specifications are tested by system testing and finally Acceptance testing verifies the requirements. The V-model gets its name from the timing of the phases. Starting from the requirements, the system is developed one phase at a time until the lowest phase, the implementation phase, is finished. At this stage testing begins, starting from unit testing and moving up one test level at a time until the acceptance testing phase is completed. During development stage the program will be tested at all levels simultaneously.

V-Model

The different levels in V-Model are unit tests, integration tests, system tests and acceptance test. The unit tests and integration tests ensure that the system design is followed in the code. The system and acceptance tests ensure that the


system does what the customer wants it to do. The test levels are planned so that each level tests different aspects of the program and so that the testing levels are independent of each other. The traditional V-model states that testing at a higher level is started only when the previous test level is completed.

Verification and Validation

Verification: Are we building the product right?The software should conform to its specification. The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase and form formal proof of program correctness.

Validation: Are we building the right product?The software should do what the user really requires. The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements.

The V & V processV & V is a whole life-cycle process and it must be applied at each stage in the software process. It has two principal objectives

The discovery of defects in a system The assessment of whether or not the system is usable in an operational

situation.

Verification Vs Validation

Verification ValidationAm I building the product right Am I building the right productThe review of interim work steps and interim deliverables during a project to ensure they are acceptable. To determine if the system is consistent, adheres to standards, uses reliable techniques and prudent practices, and performs the selected functions in the correct manner.

Determining if the system complies with the requirements and performs functions for which it is intended and meets the organization’s goals and user needs. It is traditional and is performed at the end of the project.

Am I accessing the data right (in the right place; in the right way)

Am I accessing the right data (in terms of the data required to satisfy the requirement)

Low level activity High level activity


Performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards

Performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment

Demonstration of consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle.

Determination of correctness of the final software product by a development project with respect to the user needs and requirements


Introduction to Software Testing

The software testing is a process of devising a set of inputs to a given piece of software that will check if the results produced by the software are in accordance with the expected results. Software testing will identify the correctness, completeness and quality of developed application. Testing can only find defects, but cannot prove that there are none.

Different approaches to software Testing

Positive Approach:

It is a process of establishing confidence that a program/system does what it is supposed to.

Negative Approach:

It is a process of executing a program or system with the intent of finding errors.

Levels of Testing

Types of TestingThe test strategy consists of a series of tests that will fully exercise the product. The primary purpose of these tests is to uncover limitations, defects and measure its full capability. A list of few of the tests and a brief explanation is given below.

Sanity test System test Performance test Security test Functionality/Automated test


Stress and Volume test Recovery test Document test Beta test User Acceptance test Load Volume Usability Reliability Storage Internationalization Configuration Compatibility Installation Scalability …….

Sanity Test

Sanity testing is a cursory testing. It is performed whenever a cursory testing is sufficient to prove that the application is functioning according to specifications. This level of testing is a subset of regression testing. It normally includes a set of core tests of basic GUI functionality to demonstrate connectivity to the database, application servers, printers, etc.

System Test

System tests focus on the behavior of the product. User scenarios will be executed against the system as well as screen mapping and error message testing. Overall, system tests will test the integrated system and verify that it meets the requirements defined in the requirements document.

Performance Test

Performance test will be conducted to ensure that the product’s response time meets user expectations and does not fall outside the specified performance criteria. During these tests, the response time will be measured under simulated heavy stress and/or volume.

Security Test

Security tests will determine how secure the product is. The tests will verify that unauthorized user access to confidential data is prevented. It will also verify data storage security, strength of encryption algorithms, vulnerability to hacking, etc.


Functionality/Automated Test

A suite of automated tests will be developed to test the basic functionality of the product and perform regression testing on all areas of the system to identify and log all errors. The automated testing tool will also assist in executing user scenarios thereby emulating several users simultaneously.

Stress and Volume Test

The product will be subjected to high input conditions and a high volume of data simulating peak load scenarios. The System will be stress tested based on expected load and performance criteria specified by the customer.

Recovery Test

Recovery tests will force the system to fail in various ways (all scenarios created to resemble real-life failures like disk failure, network failure, power failure, etc.) and verify that the system recovers itself based on customer’s requirement.

Documentation Test

Tests are conducted to verify the accuracy of user documentation. These tests will ensure that the documentation is a true and fair reflection of the product in full and can be easily understood by the reader.

Beta test

Beta tests are carried out on a product prior to it is commercially released. This is usually the last set of tests carried out on the system and normally involves sending the product to beta test sites outside the company for real-world exposure.

User Acceptance Test

Once the product is ready for implementation, the Business Analysts and a team that resembles the typical user profile uses the application to verify that it conforms to all expectations they have of the product. These tests are carried out to confirm that the system performs to specifications, ready for operational use and meets user expectations.

Load Testing

Load testing generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the program's services concurrently. As such, this testing is most relevant for multi-user systems; often one built using a client/server model, such as web servers. However, other types of software systems can be load-tested also. For example, a word processor or


graphics editor can be forced to read an extremely large document; or a financial package can be forced to generate a report based on several years' worth of data.

When the load placed on the system is raised beyond normal usage patterns, in order to test the system's response at unusually high or peak loads, it is known as Stress testing. The load is usually so great that error conditions are the expected result, although there is a gray area between the two domains and no clear boundary exists when an activity ceases to be a load test and becomes a stress test.

Volume Testing

Volume Testing, as its name implies, is testing that purposely subjects a system (both hardware and software) to a series of tests where the volume of data being processed is the subject of the test. Such systems can be transactions processing systems capturing real time sales or could be database updates and or data retrieval.

Volume testing will seek to verify the physical and logical limits to a system's capacity and ascertain whether such limits are acceptable to meet the projected capacity of the organization’s business processing.

Usability testing

Usability testing it is carried out pre-release so that any significant issues identified can be addressed. Usability testing can be carried out at various stages of the design process. In the early stages, however, techniques such as walkthroughs are often more appropriate. Usability testing is not a substitute for a human-centered design process. Usability testing comes in many flavors and should occur at different points in the development process.

Explorative testing gathers input from participants in the early stages of site development. Based on the experience and opinions of target users, the development team can decide the appropriate direction for the site's look and feel, navigation, and functionality.

Assessment testing occurs when the site is close to launch. Here you can get feedback on issues that might present huge problems for users but are relatively simple to fix.


http://en.wikipedia.org/wiki/Stress_testing

Evaluation testing can be useful to validate the success of a site subsequent to launch. A site can be scored and compared to competitors, and this scorecard can be used to evaluate the project's success.


Storage testing

It is used to study how memory and space is used by the program, either in resident memory or on disk. If there are limits of these amounts, storage tests attempt to prove that the program will exceed them.

Reliability testing

Verify the probability of failure free operation of a computer program in a specified environment for a specified time.

Reliability of an object is defined as the probability that it will not fail under specified conditions, over a period of time. The specified conditions are usually taken to be fixed, while the time is taken as an independent variable. Thus reliability is often written R (t) as a function of time t, the probability that the object will not fail within time t.

Any computer user would probably agree that most software is flawed, and the evidence for this is that it does fail. All software flaws are designed in; that the software does not break, rather it was always broken. But unless conditions are right to excite the flaw, it will go unnoticed -- the software will appear to work properly.

Internationalization testing

Testing related to handling foreign text and data (for ex: currency) within the program. This would include sorting, importing and exporting test and data, correct handling of currency and date and time formats, string parsing, upper and lower case handling and so forth.

Compatibility Testing

It is the process of determining the ability of two or more systems to exchange information. In a situation where the developed software replaces an already working program, an investigation should be conducted to assess possible comparability problems between the new software and other programs or systems.

Install/uninstall testing

Testing of full, partial, or upgrade install/uninstall processes.


Scalability testing

It is a subtype of performance test where performance requirements for response time, throughput, and/or utilization are tested as load on the SUT is increased over time.

Configuration testing

Quality Monitor runs tests to automatically verify (before deployment) that all installation elements – such as file extensions and shortcuts – are configured correctly and that all key functionality will work in the intended environment.

For example, all of the application shortcuts are automatically listed. Each one can be selected and run. If the shortcut launches the application as expected, a positive comment can be logged and the test will be marked as successful.


Test Techniques

There are mainly two types of test techniques, Functional Testing : Black Box Structural Testing : White Box

Black Box Testing

Black Box Testing is testing a product without having the knowledge of the internal workings of the item being tested. For example, when black box testing is applied to software engineering, the tester would only know the "legal" inputs and what the expected outputs should be, but not how the program actually arrives at those outputs. It is because of this that black box testing can be considered testing with respect to the specifications, no other knowledge of the program is necessary. The opposite of this would be white box (also known as glass box) testing, where test data are derived from direct examination of the code to be tested. For white box testing, the test cases cannot be determined until the code has actually been written. Both of these testing techniques have advantages and disadvantages, but when combined, they help to ensure thorough testing of the product.

Advantages of Black Box Testing more effective on larger units of code than white box testing tester needs no knowledge of implementation, including specific

programming languages tester and programmer are independent of each other tests are done from a user's point of view will help to expose any ambiguities or inconsistencies in the specifications test cases can be designed as soon as the specifications are complete

Disadvantages of Black Box Testing only a small number of possible inputs can actually be tested, to test every

possible input stream would take nearly forever without clear and concise specifications, test cases are hard to design there may be unnecessary repetition of test inputs if the tester is not

informed of test cases the programmer has already tried may leave many program paths untested cannot be directed toward specific segments of code which may be very

complex (and therefore more error prone) most testing related research has been directed towards white box testing


White Box Testing "Bugs lurk in corner and congregate in the boundaries." White box testing is far more likely to uncover them.

White box testing is a testing technique where knowledge of the internal structure and logic of the system is necessary to develop hypothetical test cases. It's sometimes called structural testing or glass box testing. It is also a test case design method that uses the control structure of the procedural design to derive test cases.

If a software development team creates a block of code that will allow the system to process information in a certain way, a test team would verify this structurally by reading the code and given the systems' structure, see if the code could work reasonably. If they felt it could, they would plug the code into the system and run an application to structurally validate the system.

White box tests are derived from the internal design specification or the code. The knowledge needed for white box test design approach often becomes available during the detailed design of development cycle.

Objectives of White Box TestingUsing white box testing methods, software engineer can improve the test cases that will

Guarantee that all independent paths within a module have been executed at least once

Exercise all logical decisions on their true and false sides.

Execute all loops at their boundaries and within their operational bounds

Exercise internal data structures to assure their validity.

Why White Box Testing?A reasonable question that arises in our mind is why should we carry out white box testing on a software product? Answer lies in the following points.

Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed - errors tend to creep into our work when we design and implement functions, conditions and control that are out of the mainstream. Everyday processing tends to be well understood while special case processing tends to fall in cracks.

We often believe that a logical path is not likely to be executed when, in fact, it may be executed on a regular basis - our assumptions about the logical flow of a


program and data may lead us to make design errors that are uncovered only when the path testing commences.

Typographical errors are random - when a program is translated into its programming language source code, it is likely that some typing errors will occur. It is likely that a typo will exist on obscure logical paths on a mainstream path.

Aspects of Code to considerWhite box testing of software is predicated on a close examination of procedural detail. White box test cases exercise specific sets of conditions and / or loops tests logical paths through the software. The "Status of the program" may be examined at various points to determine if the expected or asserted status corresponds to the actual status.

At first glance it would seem that very through white box testing would lead to 100% correct program. All we need to do is to define all logical paths. Develop test cases to exercise them and evaluate their results, i.e. generate test cases to exercise program logic exhaustively. Unfortunately, exhaustive testing presents certain logistical problems. For even small programs, the number of possible logical paths can be very large. For example, the procedural design that might correspond to a 100-line Pascal program with a single loop that may be executed no more than 20 times. There are approximately 10*14 possible paths that may be executed!

To put this number in perspective, we assume that a magic test processor ("magic", because no such processor exists) has been developed for exhaustive testing. The processor can develop a test case, execute it and evaluate the result in one millisecond. Working 24 hours a day, 365 days a year, the processor would work for 3170 years to test the program represented in figure. This would, undeniably cause havoc in most development schedules. Exhaustive testing is impossible for large software systems. White box testing should not, however be dismissed as impractical. Limited number of important logical paths can be selected and exercised. Important data structure can be probed for validity.

Test Adequacy CriteriaWhite box testing is very useful in achieving the test adequacy rather than designing test cases. The test adequacy criteria are based on the "code coverage analysis" which includes coverage of:

Statements in software program.

Branches or multiple condition branches those are present in a code.

Exercising program paths from entry to exit.


Execution of specific path segments derived from data flow combinations definitions and uses of variables.

The code coverage analysis methods mentioned above are discussed in the section below.

The goal for white box testing is to ensure that internal components of a program are working properly. A common focus is on structural elements such as statements and braches. The tester develops test cases that exercise these structural elements to determine if defects exist in the program structure. By exercising all of these structural elements, the tester hopes to improve the chances of detecting defects.

The testers need a framework for deciding which structural elements to select as a focus of testing, for choosing the appropriate test data and for deciding when the testing efforts are adequate enough to terminate the process with confidence that the software is working properly. The criteria can be viewed as representing minimal standards for testing a program.

The application scope of adequacy criteria also includes:

Helping testers to select properties of a program to focus on during tests.

Helping them to select a test data set for the program based on the selected properties.

Supporting testers with the development of quantitative objectives for testing.

Indicating to testers whether or not testing can be stopped for that program.

A program is adequately tested to a given criterion if all the target structural elements have been exercised according to the selected criterion. Using the selected adequacy criterion a tester can terminate testing, when the target structures have been exercised. Adequacy criteria are usually expressed as statements that depict the property or feature of interest, and the conditions under which testing can be stopped.

In addition to statement and branch adequacy criteria, other types of program-based test adequacy criteria in use are the path testing and loop testing wherein, the total paths and all the loops are executed.


Code Coverage MethodsLogic based white box test design and use of test data adequacy/ coverage concepts provide quantitative coverage goals to the tester. As mentioned in the previous section test adequacy criteria and coverage analysis play an important role in testing. The major techniques involved in code coverage analysis are:

Statement Testing

Branch testing

Multiple condition Coverage

Path testing

Modified path testing

Loop testing

Statement testing

In this method every source language statement in the program is executed at least once so that no statements are missed out. Here we need to be concerned about the statements controlled by decisions. The statement testing is usually regarded as the minimum level of coverage in the hierarchy of code coverage and is therefore a weak technique. It is based on - feeling absurd to release a piece of software without having executed every statement.

In statement testing, 100% coverage may be difficult considering the following,

There exists unreachable code

The exception handling in the code is improper

The missing statements also creates issues

Statement coverage can be calculated as:

Statement coverage = #Statements Executed / #Total Statements

Branch testing

Branch testing is used to exercise the true and false outcomes of every decision in a software program or a code. It is a superset of statement testing where the chances of achieving code coverage is more. The disadvantage of branch testing is that it is weak in handling multiple conditions.

Branch coverage can be found out as:

Branch coverage = #Branches Executed / #Total Branches


Multiple condition coverage

Multiple condition coverage is a testing metrics that exercise on all possible combinations of true and false outcomes of conditions within a decision in the control structure of a program. This is a superset of branch testing.

The primary advantage of multiple condition testing is that it tests all feasible combinations of outcomes for each condition with in program.

The drawback of this metrics is that it provides no assistance for choosing data. Another expensive drawback of this coverage is that the decision involving n Boolean conditions has 2n combinations => 2n test cases.

Multiple condition coverage can be calculated as:

Multiple condition coverage = #Conditions Executed I #Total Multiple Conditions

Path testing

Set of test cases by which every path in the program (in control flow graph) is executed at least once.

The figure in page#25 shows the graphical representation of flow of control in a program. The number of paths in this representation can be found out and we can clearly that all paths are distinct. Note that the path shown by 1-2-5-6 is different from that depicted in 1-3-4-6.

The major disadvantage of path testing is that we cannot achieve 100% path testing, as there are potentially infinite paths in a program, especially if the code is large and complex then path testing is not feasible.

Modified path testing

Modified path testing as the name suggests is a modification of existing path testing and is based on number of distinct paths to be executed, which is in turn based on McCabe complexity number. The McCabe number is given as V = E - N + 2, where E is the number of edges in the flow graph and N is the number of nodes.

Testing criterion for modified path testing is as given below:

Every branch is executed at least once

At least V distinct paths must be executed

Choosing the distinct path is not easy and different people may choose different distinct paths.

Path coverage is assessed by:

Path coverage = #Paths Executed / #Total Paths© 2006 Zeta Cyber Solutions (P) Limited

Loop testing

Loop testing strategies focus on detecting common defects associated with loop structures like simple loops, concatenated loops, nested loops, etc. The loop testing makes sure that all the loops in the program have been traversed at least once during testing. Defects in these areas are normally due to poor programming practices or inadequate reviewing.

Why Code Coverage Analysis?Code coverage analysis is necessary to satisfy test adequacy criteria and in practice is also used to set testing goals and, to develop and evaluate test data. In the context of coverage analysis, testers often refer to test adequacy criteria as "coverage criteria."

When coverage related testing goal is expressed as a percentage, it is often called "degree of coverage." The planned degree of coverage is usually specified in the test plan and then measured when the tests are actually executed by a coverage tool. The planned degree of coverage is usually specified as 100% if the tester wants to completely satisfy the commonly applied test adequacy or coverage criteria.

Under some circumstances, the planned degree of coverage may be less than 100% possibly due to the following reasons:

1. The nature of the unit

Some statements or branches may not be reachable.

The unit may be simple.

2. The lack of resources

The time set aside for testing is not adequate to achieve 100% coverage.

There are not enough trained testers to complete coverage for all units.

There might be lack of tools to support complete coverage.

3. Other project related issues such as timing, scheduling, etc.

The concept of coverage is not only associated with white box testing. Testers also use coverage concepts to support black box testing where a testing goal might be to exercise or cover all functional requirements, all equivalence classes or all system features. In contrast to black box approaches, white box based coverage goals have stronger theoretical and practical support.


The application of coverage analysis is associated with the use of control and data models to represent structural elements and data. The logic elements are based on the flow of control in a code. They are:

Program statements

Decisions or branches (these influence the program flow of control)

Conditions (expression that evaluate true/false and do not contain any other true/false valued expression)

Combinations of decisions and conditions

Paths (node sequence in flow graphs)

All structured programs can be built from three basic primes - sequential, decision and iterative. Primes are nothing but the representation of flow of control in a software program which can take anyone form. The figure shows the graphical representation of primes mentioned above.

Using the concept of a prime and the ability to use the combinations of primes to develop structured code, a control flow diagram for the software under test can be developed. The tester to evaluate the code with respect to its testability as well as to develop white box test cases can use the flow graph. The direction of the transfer depends upon the outcome of the condition.


Iteration

1

2

True

1

2

False

Sequence Condition

True1

3

2

FalseFalse1

2 3

4

Code Complexity

Cyclomatic ComplexityThomas McCabe coined the term Cyclomatic complexity. The Cyclomatic complexity is software metric that provides a quantitative measure of the logical complexity of a program. When used in the context of the basis path testing method, the value computed for Cyclomatic complexity defines the number of independent paths in the basis set of a program and provides us with an upper bound for the number of tests that must be conducted to ensure that all statements have been executed at least once. The concept of Cyclomatic complexity can be well explained using a flow graph representation of a program as shown below in figure.

In the diagram above circles denote nodes, and arrows denote edges. Each circle represents a processing task (one or more source code statements) and arrows represent flow of control.

Code Complexity ValueMcCabe defines a software complexity measure that is based on the Cyclomatic complexity of a program graph for a module. Otherwise known as code complexity number, the Cyclomatic complexity is a very useful attribute to the tester. The formula proposed by McCabe can be applied to flow graphs where there are no disconnected components. Cyclomatic Complexity is computed in one of the following ways:

1. The number of regions of the flow graph edges corresponds to the cyclomatic complexity.

2. Cyclomatic complexity, V, for a flow graph is defined as V = E - N + 2, where E is the number of flow graph edges and N is the number of flow graph nodes.

3. Cyclomatic complexity, V is also defined as V = P + 1, where P is the number of predicate nodes contained in the flow graph. A predicate note is the one having


1

4

6

32

5

a condition in it. In the figure the predicate node is the circle 1.

Use of Cyclomatic ComplexityThe code complexity value is useful to programmers and testers in a number ways which are enlisted below:

It provides an approximation on number of test cases that must be developed for branch coverage in a module of structured code.

It provides an approximation of the testability of module.

The tester can use the value of V along with past project data to approximate the testing time and resources required to test a software module.

The complexity value V along with control flow graph give the tester another tool for developing white box test cases using the concept of path.

It is also a measure of number of so called independent or distinct paths in the graph.


Bug Life Cycle

Bug life cycles are similar to software development life cycles. At any time during the software development life cycle errors can be made during the gathering of requirements, requirements analysis, functional design, internal design, documentation planning, document preparation, coding, unit testing, test planning, integration, testing, maintenance, updates, retesting and phase-out.

Bug life cycle begins when a programmer, software developer, or architect makes a mistake, creates an unintentional software defect, i.e. a bug, and ends when the bug is fixed, and the bug is no longer in existence.


What should be done after a bug is found? When a bug is found, it needs to be communicated and assigned to developers that can fix it. After the problem is resolved, fixes should be retested.

Additionally, determinations should be made regarding requirements, software, hardware, safety impact, etc., for regression testing to check the fixes didn't create other problems elsewhere. If a problem-tracking system is in place, it should encapsulate these determinations.

A variety of commercial, problem tracking/management software tools are available which can be used by software test engineers. These tools, with the detailed input of software test engineers, will give the team complete information so developers can understand the bug, get an idea of its severity, reproduce it and fix it.


Test cases

What’s a Test Case?

A test case specifies the pretest state of the IUT and its environment, the test inputs or conditions, and the expected result. The expected result specifies what the IUT should produce from the test inputs. This specification includes messages generated by the IUT, exceptions, returned values, and resultant state of the IUT and its environment. Test cases may also specify initial and resulting conditions for other objects that constitute the IUT and its environment. Some more definitions are listed below.

“Test case is a set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement”

“Test case is a documentation specifying inputs, predicted results, and a set of execution conditions for a test item” (as per IEEE Standard 829-1983 definition)

“Test cases are the specific inputs that you’ll try and the procedures that you’ll follow when you test the software”

What is Quality Writing?Quality writing is clear, concise, well-thought-out, and grammatically and syntactically correct. It conveys the author’s thoughts to the intended audience without additional, unnecessary information. When someone reads a document that has been written well, he or she is able to quickly and easily ascertain the author’s purpose for writing the document and what key points or issues the author covered.

There are two important elements in quality writing—the way it is written and what is written. There are particular benefits from each element on its own, but your writing will be most effective if you pay attention to both.

Key Points to RememberThe ability to write well can provide a number of direct and indirect benefits to your testing efforts. The following things are important to remember each time you write:

better writing leads to better thinking always write in your own voice


always keep your audience in mind use the tools available to you (such as spell-check, grammar-check, and

reference books) use the people available to you (peer reviews, technical writers'

suggestions, and marketing professionals’ suggestions) always review your work, particularly after several days (when possible) writing does get easier with practice, but it never gets easy

(There are no hard-and-fast rules in writing)

Use CasesTo write test cases, it is best if Use Cases were made available. A Use Case describes the functional component in great detail. It forms the specification of the component. A Use Case ill contain information that will form the basis of the tests that need to be carried out for the component as it is the basis on which the software has been written. A Use Case contains the following information:

Name. The name should implicitly express the user's intent or purpose of the use case, such as "Enroll Student in Seminar."

Identifier [Optional]. A unique identifier, such as "UC1701," that can be used in other project artifacts (such as your class model) to refer to the use case.

Description. Several sentences summarizing the use case. Actors [Optional]. The list of actors associated with the use case.

Although this information is contained in the use case itself, it helps to increase the understandability of the use case when the diagram is unavailable.

Status [Optional]. An indication of the status of the use case, typically one of: work in progress, ready for review, passed review, or failed review.

Frequency. How often this use case is invoked by the actor. This is often a free-form answer such as once per each user login or once per month.

Pre-conditions. A list of the conditions, if any, that must be met before a use case may be invoked.

Post-conditions. A list of the conditions, if any, that will be true after the use case finishes successfully.

Extended use case [Optional]. The use case that this use case extends (if any). An extend association is a generalization relationship where an extending use case continues the behavior of a base use case. The extending use case accomplishes this by inserting additional


action sequences into the base use-case sequence. This is modeled using a use-case association with the <<extend>> stereotype.

Included use cases [Optional]. A list of the use cases this one includes. An include association is a generalization relationship denoting the inclusion of the behavior described by a use case within another use case. This is modeled using a use-case association with the <<include>> stereotype. Also known as a uses or a has-a relationship.

Assumptions [Optional]. Any important assumptions about the domain that you have made when writing this use case. At some point you should verify these assumptions and evolve them either into decisions or simply into parts of the basic course or alternate courses of action.

Basic course of action. The main path of logic an actor follows through a use case. Often referred to as the happy path or the main path because it describes how the use case works when everything works as it normally should.

Alternate courses of action. The infrequently used paths of logic in a use case, paths that are the result of an alternate way to work, an exception, or an error condition.

Change history [Optional]. Details about when the use case was modified, why, and by whom.

Issues [Optional]. A list of issues or action items, if any, which are related to the development of this use case.

Decisions: A list of critical decisions, typically made by the SME pertaining to the content of the use case. It is important to record these decisions to maintain a knowledgebase that others can refer to.

Where such use cases are not available, do not attempt to create it as it is very difficult to create accurate and complete Use Cases unless the design documents are available.

Writing Test CasesTo create a test case when Use Cases are not available, the best approach is to start with creating the work-flow of the application and ending up with test cases. The sequence of activities is as follows:

a) Draw the work-flow diagram from the descriptive document that you created when you navigated through the application.

b) For each component in the work-flow, create a list of the screens that are involved.


c) For each screen, create a list of fields that must be checked. For each field, create a list of tests that need to be carried out.

d) Refer to the DB Schema and the DB Structures to test that all integrity checks are in place and that all necessary fields appear on the screen and are active.

e) For each function described, create a test case that will yield only one resulted value each. Where necessary, split the test case into more than one TC to ensure that the total number of steps in each test case does not exceed eight (exceptions are allowed). Ideally, “ONLY ONE RESULTED VALUE IS ALLOWED PER TEST CASE”.

f) The Test Cases must be ideally numbered in the following sequence: 999.999.999.999. The first 3 sets of numbers (3 digits each) refer to the product center and the ID within the application. The fourth three digit number is a serial number.

g) As far as possible, do not batch more than 50 test cases per module. Split the module into two at a convenient place.

h) Ensure that no test case is dependent on the previous test case’s execution to avoid a problem of daisy-chaining them so that even if one test case fails, the others must continue to run without problems.

i) There should not be any ambiguity in the steps nor the resulted valuesj) Test cases must be written to arrive at positive results. When writing test

cases for applications where the use cases are not available, negative result test cases are also needed


Software Inspection

Software inspection is a verification process, which is carried out in a formal, systematic and organized manner. Verification techniques are used during all phases of software development life cycle to ensure that the product generation is happening the "right way". The various verification techniques other than formal inspection are reviews and walkthroughs. These processes work directly on the work product under development. The other verification techniques, which also ensure product quality, are management reviews and audits. We will be focusing more on reviews, walkthroughs and formal inspections.

Steps Involved in InspectionInspection is a team review of a work product by peers of the author of the work product. This is a very effective verification technique. Several steps are involved in the inspection process that is outlined as shown in the figure.

The figure given above reveals the several steps that are carried out during the inspection process. The responsibility for initiating and carrying through the steps


Defect Database

MetricDatabase

Entry Criteria

InspectionPolicies and

plan

Invitation

Preparation

InspectionMeeting

ReportingResults

Rework andFollow-up

Exit

Checklist

belongs to the inspection leader or moderator, who is usually a member of technical staff or the software quality assurance team. The steps in the figure above are as discussed:

Inspection policies and plansThe inspection leader plans for the inspection, sets the date, schedules the meeting, runs the inspection meeting, appoints a recorder to record the results and monitor the follow up period after the review.

Entry criteriaThe inspection process begins when the inspection pre-conditions are met as specified in the inspection policies, procedures and plans. A personal pre-inspection be performed carefully by each team member. Error, problems and items should be noted by each individual for each item on the list. When the actual meeting takes place, the document under review is presented by a reader and is discussed as it is read. Attention is paid to issues related to quality, adherence to standard, testability, traceability and satisfaction of the user's requirements

ChecklistThe checklist varies with the software artifact being inspected. It contains items that the inspection participants should focus their attention on, check and evaluate. The inspection participants address each item on the checklist. The recorder records any discrepancies, misunderstandings, errors and ambiguities or in general any problem associated with an item. The competed checklist is part of the review summary document.

Reviewers use organization's standard checklist for all work-products to look for common errors. Specific checklist is also prepared for all work-products to increase review effectiveness before going for actual review/inspection. Checklist is dynamic and improves over time in the organization by analyzing root causes for the common issues found during inspection process.

InvitationThe inspection leader invites each member participating in the meeting and distributes the documents that are essential for the conduct of the meeting.

PreparationThe key item that the inspection leader prepares is the check list of items that serves as agenda for the review which helps in determining the area of focus, its objectives and tactics to be used.


Inspection meetingThe inspection leader announces the inspection meeting, and distributes the items to be inspected, the checklist and any other auxiliary materials to the participants usually a day or two before the scheduled meeting. Participants must do their homework and study the items and the checklist. The group as a whole addresses all the items in the checklist and the problems is recorded. The recorder documents all the findings and the measurements.

Reporting resultsWhen the inspection meeting has been completed, (all agenda items covered), the inspectors are usually asked to sign a written document that is sometimes called as summary report. The defect report is stored in the defect database. Inspection metrics are also recorded in the metric database.

Rework and follow-upThe inspection process requires a formal follow-up process. Rework session should be scheduled as needed and monitored to ensure that all problems identified at the inspection meeting have been addressed and resolved. This is the responsibility of the inspection leader. Only when all problems have been resolved and the item is either re-inspected by the group or the moderator is the inspection process completed.


Participants and their Roles in InspectionThe organization of the inspection team is critical for the success and efficiency of the inspection process. The following roles are defined for an inspection process. The roles of participants in the inspection process are as shown below.

Moderator/Inspection leader

Recorder/scribe

Author

Inspectors


Planning, check entry and exit criteriaCo-ordinate meeting Preparing checklists Distributing review documentManaging the review meetingIssuing review reports Follow-up oversight

Recording and documenting problems, defects, findings, and recommendations

Owner of the document, Present review item Perform any needed rework on the reviewed item

Attend review-training sessions, Prepare for reviews, Participate in meetings; Evaluate reviewed item, Perform rework where appropriate

Focus Areas of Software InspectionThe focus area of inspections depends on the work product under review. The following section discusses the focus area of inspection process.

Requirements specification documentIn case of a requirements specification document, the following attributes can be the focus areas:

o Usability: Effort required in learning, operating, preparing, inputting and interpreting the output

o Reliability: Extent to which the work product can be expected to perform its required functions

o Serviceability or maintainability: Effort required locating and fixing an error in the work product

o Traceability to a customer baseline: Ability to trace a work product or a software component back to its requirements

o Performance: Measure of efficiency of a work product.

o Stability: Ability of the work product to remain stable when a change occurs.

o Portability: Ability of the work product to adapt to new working environments.

o Understandability:

o Modifiability: Ability of a software work product to sustain the modification made without affecting its other parts.

Functional design documentIn this document we look how successfully the user requirements were incorporated in the system or the software product. An end user point of view is adopted to check for reviewing the functional design document. This also helps to trace the incompleteness of the document.

Internal design document inspectionBy inspecting the internal design document correctness of data structure, clarity on data flow, clarity on algorithms/logic used, traceability of the document and clarity on interfaces between modules can be verified.

Code inspectionWhen a software code is inspected, one can check the code's adherence to design


specification and the constraints it is supposed to handle. We can also check for its language and correspondence of the terms in code with the data dictionary.

ReviewsReview involves a meeting of a group of people, whose intention is to evaluate a software related item. Reviews are a type of static testing technique that can be used to evaluate the quality of software artifacts such as requirements document, a test plan, a design document or a code component.

The composition of a review group may consist if managers, clients, developers, testers and other personnel, depending on the type of artifact under review.

Review objectivesThe general goals for the reviewers are to

- Identify problem components or components in the software artifacts that need improvement.

- Identify problem components in the software artifact that do not need improvement.

- Identify specific errors or defects in the software artifact.

- Ensure that the artifact conforms to organizational standards.

Other review goals are informational, communicational and educational where by review participants learn about the contents of the developing software artifacts, to help them understand the role of their own work and to pan for the future stages of development.

Benefits of reviewingReviews often represent project milestones and support the establishment of a baseline for a software artifact. Review data can also have an influence on test planning which can help test planners, select effective classes of test and may also have an influence on testing goals. In some cases, clients attend the review meetings and give feedback to development team so reviews are also a means for client communication. To summarize, the benefits of review programs are

- The quality of the software is higher.

- Productivity (shorter rework time) is increased.

- There is a closer adherence to project schedules (improved process control).

- Awareness of software quality issues can be increased.

- It provides opportunity to identify reusable software artifacts.


- Maintenance costs can be reduced.

- More effective test planning can be done.

- Customer satisfaction is higher.

- A more professional attitude is developed on the part of development staff.

Reviews are characterized by its less formal nature compared to inspections. The reviews are more like a peer group discussion with no specific focus on defect identification, data collection and analysis. Reviews do not require elaborate preparation.

WalkthroughsWalkthroughs are a type of technical review where the producer of the review material servers as the review leader and actually guides the progress of the review. They are normally applied on the following documents.

- The design document

- The software code

- The data description

- The reference manuals

- The software specification documents

In the case of detailed design and code walkthroughs, test inputs may be selected and review participants walkthrough the design or code with the set of inputs in a line by line manner. The reader can compare this process to manual execution of the code. If the presenter gives a skilled presentation of the material, the walkthrough participants are able to build a comprehensive mental model of the detailed design or code and are able to evaluate its quality and detect defects.

The primary intent of a walkthrough is to make a team familiarized with a work product. Walkthroughs are useful when one wants to impart a complex work product to a team. For example, a walkthrough for a complex design document can be done by the lead designer to the coding team.


Advantages of walkthroughsThe following are the advantages of a walkthrough that makes it attractive for reviewing less critical artifacts

- One can eliminate the checklist and preparation step for a walkthrough.

- There is usually no mandatory requirement for a walkthrough, neither a defect list.

- There is also no formal requirement for a follow-up step.

Benefits of InspectionThe benefits of inspection can be classified as direct and indirect benefits which are as discussed below:

Direct benefits:- By using various inspection techniques we can improve the development

productivity by 25% to 30%.

- There will be a considerable reduction in the development timescale by 35% to 50%.

- We can reduce the testing cost and in turn the time required for testing by 50% to 90%.

- It helps in achieving a toward-zero-defect state.

Indirect benefits:The indirect benefits include the following:

Management benefits:- The software inspection process gives visibility to the relevant facts and

figures about their software engineering environment earlier in the software life cycle.

- It helps in improving and impressing the quality of audit functioning.

Deadline benefits:- Inspection will give early danger signals so that one can act appropriately and

reduce the impact to the deadline of the project.

Organization and people benefits:- The quality of the work is improved to a greater extent and it becomes more

maintainable when inspection is carried out.

- Also the software professionals can expect to live under less intense deadline


pressure.


Unit Testing /Module Testing

The need for Levels of TestingExecution based software testing, especially for large systems, is usually carried out at different levels. Each of these may consist of one or more sublevels or phases. At each level there are specific testing goals.

A unit test is a level of testing where a single component or a unit is tested. Unit testing is also called module testing. A unit is the smallest possible testable software component which can be a function or a group of functions, class or collection of classes, a screen or collection of screens or a report.. It can be characterized in several ways. For example, a unit in a typical procedure oriented software system:

- Performs a single cohesive function

- Can be compiled separately

- Is a task In a work breakdown structure (from the manager's point of view)

- Contains code that can fit on a single page or screen

A unit is traditionally viewed as a function or procedure implemented in a procedural programming language.

Objectives of Unit TestingA principal goal is to detect functional and structural defects in the unit. Or in other words unit testing is done to insure that each individual software unit is functioning according to the specification. Since the software component being tested is relatively small in size and simple in function, it is easier to design, execute, record and analyze test results. If a defect is revealed by the tests it is easier to locate and repair since only one unit is under consideration.

Developers perform unit test in many cases soon after the module is completed. Some developers also perform an informal review of the unit.

Faults detected by Unit testing

The tests that occur as a part of unit testing are as discussed.

Interface and implementation issues

The module interface is tested to ensure that information properly flows into and out off the program unit which is under test. Tests of dataflow across the module interface are required before any other test is initiated because; if data does not enter and exit properly all other tests are debatable.


Local data structures

The local data structures (groups of related data items) are examined to ensure that the data stored temporarily maintains its integrity during all steps in an algorithms execution.

Data handling errors

These include handling data types and boundary conditions which help in ensuring that the data types like characters, integers etc are properly checked along with boundary value checks.

Exception handling errors

The exceptions that arise during the running of .the program need to be anticipated and handled so that the program recovers and continues to function properly. An example is when a network disconnect occurs, the program should inform the user of the same and allow him to restart is activities when the network is restored.

Memory related errors

These errors include reading or writing beyond array bounds and not releasing allocated memory (results in resource leaks during running).

Display format errors

These include all errors related to displaying and formatting data. For example, name getting truncated and displayed incorrectly on the screen or using positive number format for negative values while using string formatted output functions like 'printf' function in C language.

Independent paths

Unit tests need to unearth non-reachable code segments. All independent paths through the control structure need to be exercised to ensure that all statements in the module are executed at least once.

Unit Test considerations

Among the more common errors, that are detected using unit tests, some are listed below.

- Some errors may be misunderstood or incorrect arithmetic precedence that occur in a code or module.


- There are mixed mode operations that can give rise to errors in the program.

- Sometimes incorrect initialization too leads to issues.

- Precision inaccuracy is another issue that can be encountered during unit testing.

- The incorrect symbolic representation of an expression also causes defects.

Selective testing or execution paths are essential tasks during the unit tests. Test cases should be designed to uncover erroneous computations, incorrect comparisons and improper control flow. Path and loop testing are effective techniques to uncover broad array of path errors. They should also uncover potential defects in

- Comparison of different data types in the work product.

- Incorrect logical operators or precedence.

- Expectation of equality when precision error makes equality unlikely.

- Incorrect comparison of variables.

- Improper of non-existent loop termination.

- Failure to exit when divergent iteration is encountered.

- Improperly modified loop variables.

Be sure that we design tests to execute every error handling path. If we do not do that the path may fail when it is invoked. Good design dictates that error conditions be anticipated and error handling paths set reroute or cleanly terminate processing when an error occurs.

Among the potential errors that should be tested when error handling is evaluated are:

- When error description is unintelligible in a software work product.

- When error noted in unit code does not correspond to error encountered.

- If the error condition causes system intervention prior to error handling.

- Exception condition processing is incorrect in a module.

- When an error description does not provide enough information to assist in the location of the cause of the error.


Unit Test Procedures Unit testing is normally considered as an adjunct to the coding step. After source level code has been developed, reviewed and verified for correspondence to component level, unit test cases begin.

The unit test environment is illustrated in Figure above. Since the component is not a standalone program, driver and/or stub software must be developed for each unit test. In most applications the driver is nothing other than a "main program" that accepts test case data, passes such data to the component to be tested, and prints the relevant results. Stubs serve to replace modules that are subordinate to (called by) the component under test. A stub or "dummy subprogram" uses the subordinate module's interface, may do minimal data manipulation, prints verification of entry, and returns control to the module under testing.

Drivers and stubs present overhead, as they are not part of the final deliverable. Judicious decision must be taken whether to write too many stubs and drivers of postpone the some of the testing to integration phase.

Unit testing is simplified when a component with high cohesion is designed. Hence only one function is addressed by a component, the number of test cases is reduced and errors can be more easily predicted and uncovered.


Interface Local data structures Boundary conditions Independent paths Error handling paths

Driver

Module To be tested

Stub Test Cases

RESULTS

Stub

Tools for Unit Testing The following tools are available for unit testing. Most of them are open source.

Coverage analyzersThese are tools that help in coverage analysis of the software work product. Some of them are: JCover, PureCoverage, and McCabe Toolset

Test frameworkTest frameworks provide one with a standard set of functionality that will take care of common tasks specific to a domain, so that the tester can quickly develop application specific test cases using facilities provided by the framework. Frameworks contain specialized APls, services, and tools. This will allow the tester to develop test tools without having to know the underlying technologies, thereby saving time. Some of the test frameworks available are JVerify, JUnit, CppUnit, Rational Realtime, Cantata, and JTest

Static AnalyzersStatic analyzers carry out analysis of the program without executing it. Static analysis can be used to find the quality of the software by doing objective measurements on attributes such as cyclomatic complexity, nesting levels etc. Some of the commonly used static analyzers are JStyle, QualityAnalyzer, JTest

Memory leak detectorsThe tools mentioned below detects whether any memory leak is affecting the software program. Memory leaks happen when resources allocated dynamically during testing are not freed after use, which will result in the program utilizing all the available resources over a period of time, making the system unusable. Some of the tools that can be used for memory leak detection are BoundsChecker, Purify, Insure++


Integration testingThere is a risk that Data can be lost at the interface. Similarly one module can have inadvertent affect on another. When several sub-functions are combined the desired major function may not be produced and individually acceptable imprecision may be magnified to unacceptable levels. The global data structures can also present problems. So in order to rectify similar issues we need to carry out integration testing.Integration testing is a systematic testing technique for constructing the program structure while at the same time conducting tests to uncover errors associated with the interfaces.

Objective of Integration TestingThe objective of unit testing is to take unit tested modules and build a program structure that has been dictated by the design.The two major goals of integration testing are

o To detect the defects that occurs on the interface of the units.o To assemble the individual unit into working subsystems and finally a

complete system that is ready for system test.The interfaces are more adequately tested when the units are finally connected to a full and working implementation of those units it calls and those that call it. As a consequence of this integration process software subsystems are put together to form a complete system and integration testing is carried out.With a few minor exceptions, integration test should be performed on units that have been reviewed and successfully passed unit testing. A tester might believe erroneously that since a unit has already been tested during unit test with a driver and stub, it does not need to be retested in combination with other units during integration tests. However, a unit tested in isolation may not have been tested adequately for the situation where it is combined with other modules.One unit at a time is integrated into a set of previously integrated modules which have passed a set of integration tests. The interfaces and the functionality of the new unit in combination with the previously integrated units are tested.

Advantage of Integration TestingIntegrating one unit at a time helpful in several waysIt keeps the number of new interfaces to be examined ‘small’, so tests can focus on these interfaces only as experienced testers know that many defects occur at the module interfaces.Massive failures that occur when multiple units are integrated at once can be avoided. For developers it allows the defect search and repair to be confined to a small known number of components and interfaces.


Independent subsystems can be integrated in parallel as long as required units are available.


Integration Strategies

Non Incremental ApproachThis approach is also called big bang approach. A representation of modules in big bang approach is as shown in the Figure.

In this, all the modules are put together and combined randomly into integrated system and then tested as a whole. Here all modules are combined in advance and the entire program is tested. The result is a usually a chaos because a set of errors are encountered. Correction is difficult because isolation of the causes is complicated by the vast expansion of the program. Once these errors are corrected one can find that new defects pop up and this becomes a seemingly endless loop. Therefore to solve this problem we come up with another strategy, which is called incremental approach.

Incremental Approach As the name suggests here we have assemble different units one by one in a systematic or incremental manner. Each module will be tested before it is integrated with another unit. And this approach is taken care when all the units are integrated into one system. We can classify the incremental approach as Top down

approach Bottom up

approach Sandwich

approach


M3

M7

M4M2

M5 M6

M8

M1

Top down ApproachIn this approach, modules are integrated by moving downward through the control hierarchy beginning with the main control module or the main program and gradually coming down one by one. Modules subordinate and ultimately subordinate to the main control module and are incorporated into the structure either in depth first or breadth first manner.Referring to the figure below which represents the depth first integration, it would integrate all the modules on a major control path of the structure. Selection of the major path is somewhat arbitrary and depends on application-specific characteristics. For example, selecting the left hand path with components M1, M2, M8 would be integrated first (They are shown by the dotted line)

Advantage of top-down integrationThe top down integration strategy verifies the major control or decision- points early in test process. In a well-factored program, structure decision-making occurs at upper level and if a major control problem exists then it needs to be recognized early.If depth first approach is used then a complete function of the software may be implemented and demonstrated. Early demonstration of functional capability is a confidence builder.

Disadvantage of top-down IntegrationHere logistical problems can arise. It occurs commonly when processing at low levels in the beginning of top down approach therefore no significant data can flow upward in the hierarchy is required to test the upper levels adequately. Stubs replace the low level modules at the program structure. Now the testers are left


M1

M5

5

M7

M4M2 M3

M6

M8

with three choices:1. Delay many tests until stubs are replaced with actual

models.This delay in tests causes us to loose some control over the correspondence between specific tasks and incorporation of specific modules. This can lead to the difficulty in determining the cause of errors and tends to violate the highly constrained nature of the top-down approach.

2. Develop stubs that perform limited functions that stimulate the actual module. This approach develops stubs to perform limited functions is workable but can lead to significant overhead as stubs become more and more complex.

3. Integrate the software form bottom of the hierarchy upward.This approach called the bottom-up approach is discussed in the next

section

Bottom-UP INTEGRATIONAs its name implies, in case of bottom-up integration, testing begins with construction and testing with atomic modules i.e., components at the lower levels in the program structure. Since components are integrated from the bottom up, processing required for the components subordinate to a given level is always available and the need for stubs is eliminated.

Bottom-up integration is a technique in integration testing where modules are added to the growing subsystem starting from the bottom or lower level. Bottom-up integration of modules begins with testing the lowest-level modules. These modules do not call other modules. Drivers are needed to test these modules. The next step is to integrate modules on the next upper level of the structure. In the process for bottom-up integration after a module has been tested, its driver is replaced by an actual module (the next one to be integrated. This next module to be integrated may also need a driver and this will be the case until we reach the highest level of the structure. Bottom-up integration follows the pattern illustrated in the figure below:


Mc cccccccc1ccc

Mxvxvxvxvxv

Mb

D111111111111111111111111

D2 D3

Cluster 3

Cluster1

Cluster 2Cluster consist of classes that are related and that may work together to support a required functionality for the complete system. In the figure 3 components are combined to form clusters 1, 2, 3. Each of these dusters is tested using a driver shown as a dashed box. Components in clusters in 1 and 2 are subordinate to Ma. Drivers D1 and D2 are removed and the clusters are interfaced directly to Ma. Similarly driver D3 for cluster 3 is removed prior to integration with module Mb. This process is followed for other modules also,

Advantage of bottom-up integrationAs integration moves upward the need for separate test drivers lessen. In fact if the top two levels of the program structure are integrated top down then the number of drivers can be substantially reduced and the integration of clusters is greatly simplified. Such an approach is called sandwich integration testing.Bottom-up integration has the advantage that the lower-level modules are usually well tested early in the integration process. This is important if these modules are candidates to be reused.Disadvantage of bottom-up integrationThe major disadvantage of this strategy is that the program as an entity does not exist until the last module has been added to it. This drawback is tempered by easier test case design and a lack of stubs.The upper-levels modules in the bottom-up integration are tested later in the integration process and consequently may not be tested well. If they are critical decision makers then this would be risky.

Sandwich IntegrationIt is a strategy where both top-down and bottom-up integration strategies are used to integration test a program. This is a combined approach, which is useful when the program structure is very complex and frequent use of drivers and stubs becomes unavoidable. Because using sandwich approach at some portions of the code we can eliminate the excess use of drivers and stubs thereby simplifying the integration testing process. It is an approach that uses top down approach for modules in the upper level of the hierarchy and bottom up approach


for lower levels.


The figure below shows the top down and bottom approach incorporated in a sandwich model:

M1

M2 M3 M4

M5 M6 M7 M8

In the figure bottom up approach is applied to the modules M1, M2 and M3. Similarly a top down approach is applied to M1, M4 and M8. The sandwich model has been widely used since it uses both top down and bottom up approach.

Regression TestingThey are tests that are run every time a change is made to the software so that we can find that the change has not broken or altered any other part of the software program. Regression testing is an important strategy for reducing side effects.Each time a new module is added as part of integration testing, the software changes. New data flow paths are established, new I/O may occur and new control logic is invoked. These changes may cause problems with functions that previously worked flawlessly. In the context of an integration test strategy, the regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects.

In a broader context, successful tests results in the discovery of errors and errors must be corrected. Whenever software is corrected some aspect of software configuration like the program, its documentation, etc. is changed. Regression testing is the activity that helps to ensure that changes do not introduce unintended behavior or additional errors.

As the integration testing proceeds the number of regression tests can also grow quite large. Regression testing may be conducted manually or by using automated tools. It is impractical and inefficient to re-execute every test for every program function once change has occurred. Therefore, regression test suite


should be designed to include only those tests that address one or more classes of errors in each of the major program functions.


A regression test suite is the subset of tests to be executed. It contains three different classes of test cases:

A representative sample of tests that will exercise all software function is one of the classes

Additional tests that focus on software functions that are likely to be affected by the change

Tests that focus on the software components that have been changed constitute the third type of class

Smoke Testing

Smoke testing is an integration testing strategy that is commonly used when "shrink-wrapped" software products are being developed. It is designed as a pacing mechanism for time-critical projects, allowing the software team to assess its projects on a frequent basis. The smoke testing approach encompasses the following activities:

Software components that have been translated into code are integrated into a "build." A build includes all data field, libraries, reusable modules and engineered components that are required to implement one or more product components.

A series of tests is designed to expose errors that will keep the build from properly performing its function. The intent should be to uncover "show stopper" errors that have the highest likelihood of throwing the software project behind schedule.

The build is integrated with other builds and the entire product in its current form is smoke tested daily. The integration approach may be top down or bottom up.

Treat the daily build as the heartbeat of the project. If there is no heartbeat then the project is dead. The smoke test should exercise the entire system from end to end. It does not have to be exhaustive but it should be capable of exposing major problems. The smoke test should be thorough enough that if the build passes, you can assume that it is stable enough to be tested more thoroughly.

Benefits of smoke testingSmoke testing provides a number of benefits when it is applied on complex time critical software engineering projects. Some of them are as discussed:

Integration risk is minimized - Since smoke tests are conducted daily; incompatibilities and other show stopper errors are uncovered early, thereby reducing the likelihood of serious schedule impact when errors are


uncovered. The quality of the end product is improved - Smoke testing is likely to

uncover both functional errors as well as architectural and component level design defects, which will in turn improve the quality of the product

Error diagnosis and Correction are simplified - Software that has just been added to the builds is a probable cause of a newly discovered defect Smoke testing makes that defect identification easier.

Progress is easier to assess - With each build being added to the previous one the progress can be tracked easily.

Validating Integration Testing

During integration testing it becomes important that for every integrated module, necessary validation be carried out so that faults or defects are not injected and carried over to other parts of the program. While integrating we need to check out several areas where the issues are likely to gather.As integration testing is conducted, tester should identify issues in the following areas:Critical modules:Critical modules are those that have the following characteristics:

Addresses several software requirements Has a high level control (resides relatively high in the program

structure) A module which is complex or error prone (cyclomatic complexity can

be used as an indicator) Modules that have definite performance requirements Modules that use critical system resources like CPU, memory, I/O

devices, etc.

Critical modules should be tested as early as possible. In addition regression tests should focus on critical module function.Interface integrity:Internal and external interfaces are tested as each module or duster is incorporated into the structure.Functional validity:Tests designed to uncover functional errors associated with the system are conducted to track the faultsInformation content:Tests designed to uncover errors associated with local or global data structures are conducted.


Non-functional issues:Performance - Tests designed to verify performance bounds established during software design are conducted. Performance issues such as time requirements for a transaction should also be subjected to tests.Resource hogging issues - When modules are integrated, then one module may interfere with the resources used by another module or might require resources for its functions. This usage of additional resources like CPU, memory, external devices, etc can cause interference and performance problems which may res1Tict the system function. This contributes to resource hogging.Scalability issues - The requirement of additional resources by various integrated modules can cause scalability issues.

Integration Test Planning

Integration test must be planned. Planning can begin when high-level design is complete so that the system architecture is defined. Other documents relevant to integration test planning are requirements document, the user manual, and usage scenarios. These documents contain structure char3, state char3, data dictionaries, cross-reference tables, module interface descriptions, data flow descriptions, messages and event descriptions, all necessary to plan integration tests.

The strategy for integration should be defined. For procedural-oriented system the order of integration of the units of the units should be defined. This depends on the strategy selected. Consider the fact that the testing objectives are to assemble components into subsystems and to demonstrate that the sub system functions properly with the integration test cases. For object-oriented systems a working definition of a cluster or similar construct must be described, and relevant test cases must be specified. In addition, testing resources and schedules for integration should be included in the test plan.

As stated earlier in the section, one of the integration tests is to build working subsystems, and then combine theses into the system as a whole. When planning for integration test the planner selects subsystems to build based upon the requirements and user needs. Very often subsystems selected for integration are prioritized. Those that represent key features, critical features, and/or user-oriented functions may be given the highest priority. Developers may want to show the clients that certain key subsystems have been assembled and are minimally functional.


System Testing

Why System Testing?The system testing requires a large amount of resources. The goal is to ensure that the system performs according to its requirements. The system test evaluates both functional behavior as well as quality requirements such as performance, reliability, usability, security etc. This phase of testing is especially useful for detecting external hardware and software interface defects for example those causing race conditions, deadlocks, problems with interrupts and exception handling, and ineffective memory usage. After system test, the software will be turned over to the users for evaluation during acceptance testing, alpha/beta tests. The organization would like to ensure that the quality of the software has been measured and evaluated before users or clients are invited to use the system.

Who does System Testing?Since system test often require many resources, special laboratory equipments and long testing time, a team of system testers usually performs it. The best scenario is for the team to be part of an independent testing group. The team must do their best to find any weak areas in the software, therefore it is best that no developers are directly involved.

Types of System TestsAs the system has been assembled from its component parts, many of the types of tests that are implemented on the component parts and subsystems are basically non-functional in nature. In fact when a system is subjected to system testing, we check for the non-functional requirements of that system.

Exploratory Testing

How it differs from Scripted Testing?

The plainest definition of exploratory testing is test design and test execution at the same time. This is the opposite of scripted testing (predefined test procedures, whether manual or automated). Exploratory tests, unlike scripted tests, are not defined in advance and carried out precisely according to plan. This may sound like a straightforward distinction, but in practice it's murky. That's because "defined" is a spectrum. Even an otherwise elaborately defined test procedure will leave many interesting details (such as how quickly to type on the keyboard, or what kinds of behavior to recognize as a failure) to the discretion of the tester. Likewise, even a free-form exploratory test session will involve tacit constraints or mandates about what parts of the product to test, or what


strategies to use. A good exploratory tester will write down test ideas and use them in later test cycles. Such notes sometimes look a lot like test scripts, even if they aren't. Exploratory testing is sometimes confused with "ad hoc" testing. Ad hoc testing normally refers to a process of improvised, impromptu bug searching. By definition, anyone can do ad hoc testing. The term "exploratory testing"--coined by Cem Kaner, in Testing

Balancing Exploratory Testing with Scripted Testing

To the extent that the next test we do is influenced by the result of the last test we did, we are doing exploratory testing. We become more exploratory when we can't tell what tests should be run, in advance of the test cycle, or when we haven't yet had the opportunity to create those tests. If we are running scripted tests, and new information comes to light that suggests a better test strategy, we may switch to an exploratory mode (as in the case of discovering a new failure that requires investigation). Conversely, we take a more scripted approach when there is little uncertainty about how we want to test, new tests are relatively unimportant, the need for efficiency and reliability in executing those tests is worth the effort of scripting, and when we are prepared to pay the cost of documenting and maintaining tests. The results of exploratory testing aren't necessarily radically different than those of scripted testing, and the two approaches to testing are fully compatible.

Why Exploratory Testing?

Recurring themes in the management of an effective exploratory test cycle are tester, test strategy, test reporting and test mission. The scripted approach to testing, attempts to mechanize the test process by taking test ideas out of a test designer's head and putting them on paper. There's a lot of value in that way of testing. But exploratory testers take the view that writing down test scripts and following them tends to disrupt the intellectual processes that make testers able to find important problems quickly. The more we can make testing intellectually rich and fluid, the more likely we will hit upon the right tests at the right time. That's where the power of exploratory testing comes in: the richness of this process is only limited by the breadth and depth of our imagination and our emerging insights into the nature of the product under test.

Scripting has its place. We can imagine testing situations where efficiency and repeatability are so important that we should script or automate them. For example, in the case where a test platform is only intermittently available, such as a client-server project where there are only a few configured servers available


and they must be shared by testing and development. The logistics of such a situation may dictate that we script tests carefully in advance to get the most out of every second of limited test execution time. Exploratory testing is especially useful in complex testing situations, when little is known about the product, or as part of preparing a set of scripted tests. The basic rule is this: exploratory testing is called for any time the next test you should perform is not obvious, or when you want to go beyond the obvious.

Testing Process

The diagram above outlines the Test Process approach that will be followed.

a. Organize Project involves creating a System Test Plan, Schedule & Test Approach, and requesting/assigning resources.

b. Design/Build System Test involves identifying Test Cycles, Test Cases, Entrance & Exit Criteria, Expected Results, etc. In general, test conditions/expected results will be identified by the Test Team in conjunction with the Project Business Analyst or Business Expert. The Test Team will then identify Test Cases and the Data required. The Test conditions are derived from the Business Design and the Transaction Requirements Documents

c. Design/Build Test Procedures includes setting up procedures such as Error Management systems and Status reporting, and setting up the data tables for the Automated Testing Tool.

d. Build Test Environment includes requesting/building hardware, software and data set-ups.

e. Execute Project Integration Testf. Execute Operations Acceptance Testg. Signoff - Signoff happens when all pre-defined exit criteria have been

achieved.


Fundamental Test ProcessTesting must be planned. Good testing requires thinking out an overall approach, designing tests and establishing expected results for each of the test cases we choose. The fundamental test process comprises planning, specification, execution, recording and checking for completion. The Planning process consists of five main stages as figure below depicts.

Test Phases


Test PlanningTest Planning involves producing document that describes your overall approach and test objectives. Completion or exit criteria must be specified so that you know when testing (at any stage) is complete. Plan your test.

A tactical test plan must be developed to describe when and how testing will occur. The test plan should provide background information on the software being tested, on the test objectives and risks, as well as on the business functions to be tested and the specific tests to be performed.

Entrance Criteria Hardware has been be acquired and installed Software has been acquired and installed Test cases and test data have been identified and are available Updated test cases Complete functional testing database Automated test cases (scripts) for regression testing The Software Requirements Specification (SRS) and Test Plan have been

signed off Technical specifications for the system are available Acceptance tests have been completed, with a pass rate of not less than

80%

Resumption CriteriaIn the event that system testing is suspended resumption criteria will be specified and testing will not re-commence until the software reaches these criteria.

Test DesignThis involves designing test conditions and test cases using many of the automated tools out in the market today. You produce a document that describes the tests that you will carry out. It is important to determine the expected results prior to test execution.

Test Execution This involves actually running the specified tests on a computer system either manually or by using an automated Test tool like WinRunner, Rational Robot or SilkTest.


Test Recording This involves keeping good records of the test activities that you have carried out. Versions of the software you have tested and the test design are recorded along with the actual results of each test.

Checking for Test CompletionThis involves looking at the previously specified test completion criteria to see if they have been met. If not, some tests may need to be rerun and, in some instances, it may be appropriate to design some new test cases to meet a particular coverage target.

Exit Criteria The system has the ability to recover gracefully after failure. All test cases have been executed successfully. The system meets all of the requirements described in the Software

Requirements Specification. All the first and second severity bugs found during QA testing have been

resolved. At least all high exposure minor and insignificant bugs have been fixed and

a resolution has been identified and a plan is in place to address the remaining bugs.

100% of test cases have been executed at least once. All high-risk test cases (high-risk functions) have been successful

executed. The system can be successfully installed in the pre-live and production

environments and has the ability to recover gracefully from installation problems.

The system must be successfully configured and administered from the GUI.

The system must co-exist with other production applications software. The system must successfully migrate from the prior version. The system must meet all stated security requirements. The system and

data resources must be protected against accidental and/or intentional modifications or misuse.

Should errors/bugs be encountered, fixes will be applied and included in a scheduled release cycle dependent upon the priority level of the error. This process will continue until an acceptable level of stability and test coverage is achieved.

The system displays maintainability.


What's a 'test plan'?

A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project:

Test plan identifier

Some type of unique company-generated number to identify this test plan, its level and the level of software that it is related to. Preferably the test plan level will be the same as the related software level. The number may also identify whether the test plan is a Master plan, a Level plan, an integration plan or whichever plan level it represents. This is to assist in coordinating software and test ware versions within configuration management.

1. References 2. Introduction 3. Test Items 4. Software Risk Issues 5. Features to be Tested 6. Features not to be Tested 7. Approach 8. Item Pass/Fail Criteria 9. Suspension Criteria and Resumption Requirements 10. Test Deliverables 11. Remaining Test Tasks 12. Environmental Needs 13. Staffing and Training Needs 14. Responsibilities 15. Schedule 16. Planning Risks and Contingencies 17. Approvals 18. Glossary


References

List all documents that support this test plan. Refer to the actual version/release number of the document as stored in the configuration management system. Do not duplicate the text from other documents as this will reduce the viability of this document and increase the maintenance effort. Documents that can be referenced include:

Project Plan Requirements specifications High Level design document Detail design document Development and Test process standards Methodology guidelines and examples Corporate standards and guidelines

Introduction

State the purpose of the Plan, possibly identifying the level of the plan (master etc.). This is essentially the executive summary part of the plan.

You may want to include any references to other plans, documents or items that contain information relevant to this project/process. If preferable, you can create a references section to contain all reference documents.

Identify the Scope of the plan in relation to the Software Project plan that it relates to. Other items may include, resource and budget constraints, scope of the testing effort, how testing relates to other evaluation activities (Analysis & Reviews), and possibly the process to be used for change control and communication and coordination of key activities.

As this is the "Executive Summary" keep information brief and to the point.

Test items (functions)

These are things you intend to test within the scope of this test plan. Essentially, something you will test, a list of what is to be tested. This can be developed from the software application inventories as well as other sources of documentation and information.

This can be controlled on a local Configuration Management (CM) process if you have one. This information includes version numbers, configuration requirements


where needed, (especially if multiple versions of the product are supported). It may also include key delivery schedule issues for critical elements.

Remember, what you are testing is what you intend to deliver to the Client.

This section can be oriented to the level of the test plan. For higher levels it may be by application or functional area, for lower levels it may be by program, unit, module or build.

Software risk issues

Identify what software is to be tested and what the critical areas are, such as:

1. Delivery of a third party product. 2. New version of interfacing software 3. Ability to use and understand a new package/tool, etc. 4. Extremely complex functions 5. Modifications to components with a past history of failure 6. Poorly documented modules or change requests

There are some inherent software risks such as complexity; these need to be identified.

1. Safety 2. Multiple interfaces 3. Impacts on Client 4. Government regulations and rules

Another key area of risk is a misunderstanding of the original requirements. This can occur at the management, user and developer levels. Be aware of vague or unclear requirements and requirements that cannot be tested.

The past history of defects (bugs) discovered during Unit testing will help identify potential areas within the software that are risky. If the unit testing discovered a large number of defects or a tendency towards defects in a particular area of the software, this is an indication of potential future problems. It is the nature of defects to cluster and clump together. If it was defect ridden earlier, it will most likely continue to be defect prone.

One good approach to define where the risks are is to have several brainstorming sessions.


Features to be tested

This is a listing of what is to be tested from the user's viewpoint of what the system does. This is not a technical description of the software, but a USERS view of the functions.

Set the level of risk for each feature. Use a simple rating scale such as (H, M, L): High, Medium and Low. These types of levels are understandable to a User. You should be prepared to discuss why a particular level was chosen.

Features not to be tested

This is a listing of what is 'not to be tested from both the user's viewpoint of what the system does and a configuration management/version control view. This is not a technical description of the software, but a user's view of the functions.

Identify why the feature is not to be tested, there can be any number of reasons.

Not to be included in this release of the Software. Low risk, has been used before and is considered stable. Will be released but not tested or documented as a functional part of the

release of this version of the software.

Approach (strategy)

This is your overall test strategy for this test plan; it should be appropriate to the level of the plan (master, acceptance, etc.) and should be in agreement with all higher and lower levels of plans. Overall rules and processes should be identified.

Are any special tools to be used and what are they? Will the tool require special training? What metrics will be collected? Which level is each metric to be collected at? How is Configuration Management to be handled? How many different configurations will be tested? Hardware Software Combinations of HW, SW and other vendor packages What levels of regression testing will be done and how much at each test

level? Will regression testing be based on severity of defects detected? How will elements in the requirements and design that do not make sense

or are un-testable be processed? © 2006 Zeta Cyber Solutions (P) Limited

If this is a master test plan the overall project testing approach and coverage requirements must also be identified.

Specify if there are special requirements for the testing.

Only the full component will be tested. A specified segment of grouping of features/components must be tested

together.

Other information that may be useful in setting the approach are:

MTBF, Mean Time between Failures - if this is a valid measurement for the test involved and if the data is available.

SRE, Software Reliability Engineering - if this methodology is in use and if the information is available.

Item pass/fail criteria

What are the Completion criteria for this plan? This is a critical aspect of any test plan and should be appropriate to the level of the plan.

At the Unit test level this could be items such as: o All test cases completed. o A specified percentage of cases completed with a percentage

containing some number of minor defects. o Code coverage tool indicates all code covered.

At the Master test plan level this could be items such as: o All lower level plans completed. o A specified number of plans completed without errors and a

percentage with minor defects.

This could be an individual test case level criterion or a unit level plan or it can be general functional requirements for higher level plans.

What is the number and severity of defects located?

Is it possible to compare this to the total number of defects? This may be impossible, as some defects are never detected.

o A defect is something that may cause a failure, and may be acceptable to leave in the application.

o A failure is the result of a defect as seen by the User, the system crashes, etc.


Suspension criteria and resumption requirements

Know when to pause in a series of tests.

If the number or type of defects reaches a point where the follow on testing has no value, it makes no sense to continue the test; you are just wasting resources.

Specify what constitutes stoppage for a test or series of tests and what is the acceptable level of defects that will allow the testing to proceed past the defects.

Testing after a truly fatal error will generate conditions that may be identified as defects but are in fact ghost errors caused by the earlier defects that were ignored.

Test deliverables

What is to be delivered as part of this plan?

Test plan document. Test cases. Test design specifications. Tools and their outputs. Simulators. Static and dynamic generators. Error logs and execution logs. Problem reports and corrective actions.

One thing that is not a test deliverable is the software itself that is listed under test items and is delivered by development.

Remaining test tasks

If this is a multi-phase process or if the application is to be released in increments there may be parts of the application that this plan does not address. These areas need to be identified to avoid any confusion should defects be reported back on those future functions. This will also allow the users and testers to avoid incomplete functions and prevent waste of resources chasing non-defects.

If the project is being developed as a multi-party process, this plan may only cover a portion of the total functions/features. This status needs to be identified so that those other areas have plans developed for them and to avoid wasting resources tracking defects that do not relate to this plan.


When a third party is developing the software, this section may contain descriptions of those test tasks belonging to both the internal groups and the external groups.

Environmental needs

Are there any special requirements for this test plan, such as:

Special hardware such as simulators, static generators etc. How will test data be provided? Are there special collection requirements

or specific ranges of data that must be provided? How much testing will be done on each component of a multi-part feature? Special power requirements. Specific versions of other supporting software. Restricted use of the system during testing.

Staffing and training needs

Training on the application/system

Training for any test tools to be used

What is to be tested and who is responsible for the testing and training.

Responsibilities

Who is in charge?

This issue includes all areas of the plan. Here are some examples:

Setting risks. Selecting features to be tested and not tested. Setting overall strategy for this level of plan. Ensuring all required elements are in place for testing. Providing for resolution of scheduling conflicts, especially, if testing is done

on the production system. Who provides the required training? Who makes the critical go/no go decisions for items not covered in the test

plans?

Schedule

A schedule should be based on realistic and validated estimates. If the estimates for the development of the application are inaccurate, the entire project plan will slip and the testing is part of the overall project plan.


As we all know, the first area of a project plan to get cut when it comes to crunch time at the end of a project is the testing. It usually comes down to the decision, ‘Let’s put something out even if it does not really work all that well’. And, as we all know, this is usually the worst possible decision.

How slippage in the schedule will to be handled should also be addressed here.

If the users know in advance that a slippage in the development will cause a slippage in the test and the overall delivery of the system, they just may be a little more tolerant, if they know it’s in their interest to get a better tested application.

By spelling out the effects here you have a chance to discuss them in advance of their actual occurrence. You may even get the users to agree to a few defects in advance, if the schedule slips.

At this point, all relevant milestones should be identified with their relationship to the development process identified. This will also help in identifying and tracking potential slippage in the schedule caused by the test process.

It is always best to tie all test dates directly to their related development activity dates. This prevents the test team from being perceived as the cause of a delay. For example, if system testing is to begin after delivery of the final build, then system testing begins the day after delivery. If the delivery is late, system testing starts from the day of delivery, not on a specific date. This is called dependent or relative dating.

Planning risks and contingencies

What are the overall risks to the project with an emphasis on the testing process?

Lack of personnel resources when testing is to begin. Lack of availability of required hardware, software, data or tools. Late delivery of the software, hardware or tools. Delays in training on the application and/or tools. Changes to the original requirements or designs.

Specify what will be done for various events, for example:

Requirements definition will be complete by January 1, 20XX, and, if the requirements change after that date, the following actions will be taken:


The test schedule and development schedule will move out an appropriate number of days. This rarely occurs, as most projects tend to have fixed delivery dates.

The number of tests performed will be reduced. The number of acceptable defects will be increased. These two items could lower the overall quality of the delivered product. Resources will be added to the test team. The test team will work overtime (this could affect team morale). The scope of the plan may be changed. There may be some optimization of resources. This should be avoided, if

possible, for obvious reasons.

Management is usually reluctant to accept scenarios such as the one above even though they have seen it happen in the past.

The important thing to remember is that, if you do nothing at all, the usual result is that testing is cut back or omitted completely, neither of which should be an acceptable option.

Approvals

Who can approve the process as complete and allow the project to proceed to the next level (depending on the level of the plan)?

At the master test plan level, this may be all involved parties.

When determining the approval process, keep in mind who the audience is:

The audience for a unit test level plan is different than that of an integration, system or master level plan.

The levels and type of knowledge at the various levels will be different as well.

Programmers are very technical but may not have a clear understanding of the overall business process driving the project.

Users may have varying levels of business acumen and very little technical skills.

Always be wary of users who claim high levels of technical skills and programmers that claim to fully understand the business process. These types of individuals can cause more harm than good if they do not have the skills they believe they possess.


Glossary

Used to define terms and acronyms used in the document, and testing in general, to eliminate confusion and promote consistent communications.


What should be done after a bug is found?

The bug needs to be communicated and assigned to developers who can fix it. After the problem is resolved, fixes should be re-tested, and determinations made regarding requirements for regression testing to check that fixes didn't create problems elsewhere. If a problem-tracking system is in place, it should encapsulate these processes. A variety of commercial problem-tracking/management software tools are available.

The following are items to consider in the bug tracking process:

Complete information such that developers can understand the bug, get an idea of its severity, and reproduce it if necessary.

Bug identifier (number, ID, etc.) Current bug status (e.g., 'Released for Retest', 'new', etc.) The application name or identifier and version The function, module, feature, object, screen, etc. where the bug occurred Environment specifics, system, platform, relevant hardware specifics Test case name/number/identifier One-line bug description Full bug description Description of steps needed to reproduce the bug if not covered by a test

case or if the developer doesn't have easy access to the test case/test script/test tool

Names and/or descriptions of file/data/messages/etc. used in test File excerpts/error messages/log file excerpts/screen shots/test tool logs

that would be helpful in finding the cause of the problem Severity estimate (a 5-level range such as 1-5 or 'critical'-to-'low' is

common) Was the bug reproducible? Tester name Test date Bug reporting date Name of developer/group/organization the problem is assigned to Description of problem cause Description of fix Code section/file/module/class/method that was fixed Date of fix Application version that contains the fix Tester responsible for retest


Retest date Retest results Regression testing requirements Tester responsible for regression tests Regression testing results

A reporting or tracking process should enable notification of appropriate personnel at various stages. For instance, testers need to know when retesting is needed, developers need to know when bugs are found and how to get the needed information, and reporting/summary capabilities are needed for managers.

What is 'configuration management'?

Configuration management covers the processes used to control, coordinate, and track: code, requirements, documentation, problems, change requests, designs, tools/compilers/libraries/patches, changes made to them, and who makes the changes.

How can it be known when to stop testing?

This can be difficult to determine. Many modern software applications are so complex, and run in such an interdependent environment, that complete testing can never be done. Common factors in deciding when to stop are:

Deadlines (release deadlines, testing deadlines, etc.) Test cases completed with certain percentage passed Test budget depleted Coverage of code/functionality/requirements reaches a specified point Bug rate falls below a certain level Beta or alpha testing period ends

What if there isn't enough time for thorough testing?

Use risk analysis to determine where testing should be focused.Since it's rarely possible to test every possible aspect of an application, every possible combination of events, every dependency, or everything that could go wrong, risk analysis is appropriate to most software development projects. This requires judgment skills, common sense, and experience. Considerations can include:

Which functionality is most important to the project's intended purpose?


Which functionality is most visible to the user? Which functionality has the largest safety impact? Which functionality has the largest financial impact on users? Which aspects of the application are most important to the customer? Which aspects of the application can be tested early in the development

cycle? Which parts of the code are most complex, and thus most subject to

errors? Which parts of the application were developed in rush or panic mode? Which aspects of similar/related previous projects caused problems? Which aspects of similar/related previous projects had large maintenance

expenses? Which parts of the requirements and design are unclear or poorly thought

out? What do the developers think are the highest-risk aspects of the

application? What kinds of problems would cause the worst publicity? What kinds of problems would cause the most customer service

complaints? What kinds of tests could easily cover multiple functionalities? Which tests will have the best high-risk-coverage to time-required ratio?


Test Automation

Scope of Automation Software must be tested to have confidence that it will work as it should in its intended environment. Software testing needs to be effective at finding any defects which are there, but it should also be efficient, performing the tests as quickly as possible.Automating software testing can significantly reduce the effort required for adequate testing or significantly increase the testing that can be done in limited time. Tests can be run in minutes that would take hours to run manually. Savings as high as 80% of manual testing effort have been achieved using automation.At first glance it seems that automating testing is an easy task. Just buy one of the popular test execution tools, record the manual tests, and play them back whenever you want them to. Unfortunately, it doesn't work like that in practice. Just as there is more to software design than knowing a programming language, there is more to automating testing than knowing a testing tool.

Tool Support for Life-Cycle testingThe figure shows the availability of the tools support for testing in every stage of the software development life cycle. The different types of tools and their positions are as shown:


Acceptance test

System test

Requirement Specification

Architectural design

Detailed design

Integration test

Unit test

CodeManagement tools

Test execution and comparison

Coverage tools

Static analysis tools

Performance stimulator tools

Debugging tools

Dynamic analysis tools

Test design tools:Logical design tools:Physical designtools:

Test design tools:They help to derive test inputs or test data.

Management tools:These include tools that assist in test planning, keeping track of what tests have been run, etc.

Static tools:They analyze code without execution.

Coverage tools:They help in assessing how much of the software under test has been exercised by a set of tests.

Debugging tools:These are not essentially testing tools as debugging is not a part of testing. However they are used in testing when trying to isolate a low-level defect. It is of more use to a developer.

Dynamic analysis tools:They assess the system while the software is running. They are of great use for detecting memory leaks. The memory leaks occur if the program does not release blocks of memory when it should, so the block has leaked out of the pool of memory blocks available to the program.

Performance Simulator tools:They are tools that enable parts of a system to be tested in ways which would not be otherwise possible in the real world.


Test Automation Framework

The various components of test automation framework are as mentioned below:

DriverDrivers are tools used to control and operate the software being tested.

StubsStubs are essentially the opposite of drivers and they receive or respond to the data that the software sends. Stubs are frequently used when software needs to communicate with external devices.

SimulatorSimulators are used in place of actual systems and behave like the actual system. Simulators are excellent way of test automation when the actual system that the software interfaces is not available.

EmulatorEmulator is used to describe a device that is a plug-in replacement for the real device. A PC acting as a printer, understanding the printer codes and responding to the software as though it were a printer is an emulator. The difference between an emulator and stub is that the stub also provides a means for a tester to view and interpret the data send to it.


Driver

Stub(s) Simulator(s) Emulator(s)

Setup OracleExecute Cleanup

Test Data TestResult

Test Log

SetupThe task required to be done before a test or a set of tests can be executed. The setups maybe build for a single test, a test set or a test suite as per the need.

ExecuteDuring this phase, the test cases are run.

OracleTest oracle is a mechanism to produce the predicted outcomes to compare with the actual outcomes of the software under test.

Test dataData that exists (for example in a database) before it a test is executed and that affects by the software under test

CleanupThis task is done after This task is done after a test or set of tests has finished or stopped in order to leave the system in a clean state for the next test or set of tests. It is particularly important where a test has failed to complete.


Benefits of Automation

Test automation can enable some testing tasks to be performed far more efficiently that could ever be done by testing manually. Some of the benefits are included below.

Run existing tests on a new version of the Program

This is the most obvious task, particularly in an environment where many programs are frequently modified. Given that the test already exist and have been automated to run on an earlier version of the program it should be possible to select the tests and initiate their execution with just a few minutes of manual effort.

Run more tests more oftenA clear benefit of automation is the ability to run more tests in less time and therefore, to make it possible to run those more often. This will lead to a greater confidence in the system.

Performs tests, which should be difficult or impossible to do manuallyWhen testing manually, expected outcome typically include the obvious things that are visible to the tester. However, there are attributes that should be tested which are not easy to verify manually. For example a GUI object may trigger some event that does not produce an immediate output. A test execution tool maybe able to check that the event has been triggered which would otherwise not be possible to check without using a tool.

Better use of resourcesAutomating menial and boring tasks such as repeatedly inputting the same test inputs gives greater accuracy as well as improved staff morale and frees skilled testers to put more effort in designing better test cases to be run.There will also be some test which is best done manually. The testers can do a better job of manual testing, if there are fewer tests to be run manually. Machines which would otherwise lay idle overnight or weekends can be used to run automated tests.

Consistency and repeatability of testsTests that are repeated automatically will be repeated exactly every time (at least the inputs will be, the output may differ due to timing). This gives a level of consistency to the tests, which is very difficult to achieve manually.


The same tests can be executed on different configurations using different operating systems or databases giving consistency of cross platform quality for multi-platform products.

Reuse of TestsThe effort put into deciding what to test, designing the tests and building the tests can be distributed over many executions of those tests. Test which will be reused are worth spending time on to make sure that they are reliable.

Increased confidenceKnowing that an extensive set of automated tests have run successfully, there can be greater confidence when the system is released provided that tests being run are good tests.

The other benefits of test automation are It helps in reducing risks It is fast Helps in providing better estimation of the test effort It boosts the

morale of the tester It enhances volume testing It helps in efficient regression testing

Limitation of Test Automation Automated testing is not a panacea and there are limitations to what it can accomplish. Some of the limitations of test automation are

Does not replace manual testingIt is not possible and some times not desirable to automate all testing activities or all tests. Tests that should probably not automated include

Tests that are run only rarely When the software is very volatile Tests where the results is easily verified by humans, for example,

the suitability of a color scheme

Manual tests find more defects than automated testsA test is most likely to reveal a defect when it is run for the first time. If a test case needs to be automated, it first needs to be tested to make sure it is correct. There is little point in automating defective tests. If the software being tested has


defects that the test cases reveal, it will be revealed when run manually.

Greater reliance on quality of testsA tool can only identify the differences between the actual and expected outcomes. That is, it helps in making a comparison. When tests are run, the tool will tell you that whether the test has passed or failed, when in fact they have only matched your expected outcomes. It is therefore more important to be confident of the quality of the tests that are to be automated.

Test automation does not improve effectivenessAutomating a set of tests does not make them more effective that those same tests run manually. Automation can eventually improve the efficiency of tests i.e. how much they cost to run and how long they take to run. It also affects the resolvability of the tests.

Tools have no imagination Tool is only software, which follows instructions to execute a set of test cases. But a human tester will perform the same tasks differently, effectively and creatively. When unexpected event happen that are not part of the planned sequence of test cases, human tester can identify it easily.

Manual vs. Automated

Some writers believe that test automation is so expensive relative to its value that it should be used sparingly. Others, such as advocates of agile development, recommend automating 100% of all tests. A challenge with automation is that automated testing requires automated test oracles (an oracle is a mechanism or principle by which a problem in the software can be recognized). Such tools have value in load testing software (by signing on to an application with hundreds or thousands of instances simultaneously), or in checking for intermittent errors in software. The success of automated software testing depends on complete and comprehensive test planning. Software development strategies such as test-driven development are highly compatible with the idea of devoting a large part of an organization's testing resources to automated testing. Many large software organizations perform automated testing. Some have developed their own automated testing environments specifically for internal development, and not for resale. The debate is still on…….


Testing Terms - Glossary

Acceptance test: Formal tests conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept a system. This particular kind of testing is performed with the STW/Regression suite of tools. Back-to-back testing: For software subject to parallel implementation, back-to-back testing is the execution of a test on the similar implementations and comparing the results. Basis paths: The set of non-iterative paths. Black Box testing: A test method where the tester views the program as a black box, that is the test is completely unconcerned about the internal behavior and structure of the program. Rather the tester is only interested in finding circumstances in which the program does not behave according to its specifications. Test data are derived solely from the specifications without taking advantage of knowledge of the internal structure of the program. Black-box testing is performed with the STW/Regression suite of tools. Bottom-up testing: Testing starts with lower level units. Driver units must be created for units not yet completed, each time a new higher level unit is added to those already tested. Again a set of units may be added to the software system at the time, and for enhancements the software system may be complete before the bottom up tests starts. The test plan must reflect the approach, though. The STW/Coverage suite of tools supports this type of testing. Built-in testing: Any hardware or software device which is part of an equipment, subsystem of system and which is used for the purpose of testing that equipment, subsystem or system. Byte mask: A differencing mask used by EXDIFF that specifies to disregard differences based on byte counts. C0 coverage: The percentage of the total number of statements in a module that are exercised, divided by the total number of statements present in the module. C1 coverage: The percentage of logical branches exercised in a test as compared with the total number of logical branches known in a program. Call graph: The function call tree capability of S-TCAT. This utility shows caller-callee relationship of a program. It helps the user to determine which function calls need to be tested further. Call pair: A connection between two functions in which one function "calls" (references) another function.


Complexity: A relative measurement of the ``degree of internal complexity'' of a software system, expressed possibly in terms of some algorithmic complexity measure. Component: A part of a software system smaller than the entire system but larger than an element. Control statement: A statement that involves some predicate operation. For example: an if statement or a while statement. Correctness proof: A mathematical process which demonstrates the consistency between a set of assertions about a program and the properties of the program, when executed in a known environment. Coverage testing: Coverage testing is concerned with the degree to which test cases exercise or cover the logic (source code) of the software module or unit. It is also a measure of coverage of code lines, code branches and code branch combinations. Cyclomatic number: A number which assesses program complexity according to a program's flow of control. A program's flow of control is based on the number and arrangement of decision statements within the code. The cyclomatic number of a flow graph can be calculated as follows Decision statement: A decision statement in a module is one in which an evaluation of some predicate is made, which (potentially) affects the subsequent execution behavior of the module. Defect: Any difference between program specifications and actual program behavior of any kind, whether critical or not. What is reported as causing any kind of software problem?Directed graph: A directed graph consists of a set of nodes which are interconnected with oriented arcs. An arbitrary directed graph may have many entry nodes and many exit nodes. A program directed graph has only one entry and one exit node. End-to-end testing: Test activity aimed at proving the correct implementation of a required function at a level where the entire hardware/software chain involved in the execution of the function is available. Error: A difference between program behavior and specification that renders the program results unacceptable. See Defect. Flow graph: The oriented diagram composed with nodes and edges with arrows, the shows the flow of control in a program. It is also called a flow chart or a directed graph. Functional specifications: Set of behavioral and performance requirements which in aggregate determine the functional properties of a software system. Functional test cases: A set of test case data sets for software which are derived from structural test cases.


Infeasible path: A logical branch sequence is logically impossible if there is no collection of setting of the input space relative to the first branch in the sequence, which permits the sequence to execute. Inspection/review: A process of systematically studying and inspecting programs in order to identify certain types of errors, usually accomplished by human rather than mechanical means. Instrumentation: The first step in analyzing test coverage is to instrument the source code. Instrumentation modifies the source code so that special markers are positioned at every logical branch or call-pair or path. Later, during program execution of the instrumented Source code, these markers will be tracked and counted to provide data for coverage reports. Integration Testing: Exposes faults during the process of integration of software components or software units and it is specifically aimed at exposing faults in their interactions. The integration approach could be bottom-up (using drivers), top-down (using stubs) or a mixture of the two. The bottom up is the recommended approach. Interface: The informational boundary between two software systems, software system components, elements, or modules. Module: A module is a separately invocable element of a software system. Similar terms are procedure, function, or program. Regression Testing: Testing which is performed after making a functional improvement or repair of the software. Its purpose is to determine if the change has regressed other aspects of the software. As a general principle, software unit tests are fully repeated if a module is modified, and additional tests which expose the fault removed, are added to the test set. The software unit will then be re-integrated and integration testing repeated. S0 coverage: The percentage of modules that are invoked at least once during a test or during a set of tests. S1 coverage: The percentage of call-pairs exercised in a test as compared with the total number of call-pairs known in a program. By definition the S1 value for a module which has no call pairs is 100% if the module has been called at least once and 0% otherwise. Software sub-system: A part of a software system, but one which includes many modules. Intermediate between module and system. Software system: A collection of modules, possibly organized into components and subsystems, which solves some problem or performs some task. Spaghetti code: A program whose control structure is so entangled by a surfeit of GOTO's that is flow graph resembles a bowl of spaghetti.


Statement complexity: A complexity value assigned to each statement which is based on (1) the statement type, and (2) the total length of postfix representations of expressions within the statement (if any). The statement complexity values are intended to represent an approximation to potential execution time. Static analysis: The process of analyzing a program without executing it. This may involve wide range of analyses. The STW/Advisor suite of tools performs static analyses. System Testing: Verifies that the total software system satisfies all of its functional, quality attribute and operational requirements in simulated or real hardware environment. It primarily demonstrates that the software system does fulfill requirements specified in the requirements specification during exposure to the anticipated environmental conditions. All testing objectives relevant to specific requirements should be included during the software system testing. Software system testing is mainly based on black-box methods. The STW/Coverage suite of tools supports this type of testing. Test: A [unit] test of a single module consists of (1) a collection of settings for the inputs of the module, and (2) exactly one invocation of the module. A unit test may or may not include the effect of other modules which are invoked by the undergoing testing. Test Bed: See Test Harness. Test coverage measure: A measure of the testing coverage achieved as the result of one unit test usually expressed as a percentage of the number logical branches within a module traversed in the test. Test data set: A specific set of values for variables in the communication space of a module which are used in a test. Test harness: A tool that supports automated testing of a module or small group of modules. Test object, object under test: The central object on which testing attention is focused. Test path: A test path is a specific (sequence) set of logical branches which is traversed as the result of a unit test operation on a set of test case data. A module can have many test paths. Test purpose: The free-text description indicating the objective of a test, which is usually specified in the source clause of a SMARTS ATS file. Test stub: A testing stub is a module which simulates the operations of a module which is invoked within a test. The testing stub can replace the real module for testing purposes. Test target: The current module (system testing) or the current logical branch (unit testing) upon which testing effort is focused.


Test target selector: A function which identifies a recommended next testing target. Testability: A design characteristic which allows the status (operable, inoperable, or degrade) of a system of any of its sub-system to be confidently determined in a timely fashion. Testability attempts to qualify those attributes of system designs which facilitate detection and isolation of faults that affect system performance. Testability can be defined as the characteristic of a design which allows the status of a system of any of its subsystems to be confidently determined in a timely fashion. Testing: Testing is the execution of a system in a real or simulated environment with the intent of finding faults. Testing Techniques: Can be used in order to obtain a structured and efficient testing which covers the testing objectives during the different phases in the software life cycle. Top-Down Testing: The testing starts with the main program. The main program becomes the test harness and the subordinated units are added as they are completed and testing continues. Stubs must be created for units not yet completed. This type of testing results in retesting of higher level units when lower level units are added. The adding of new units one by one should not be taken too literary. Sometimes a collection of units will be included simultaneously, and the whole set of units will serve as test harness for each unit test. Each unit is tested according to a unit test plan, with a top-down strategy. Unit Testing: Unit testing is meant to expose faults on each software unit as soon as this is available regardless of its interaction with other units. The unit is exercised against its detailed design and by ensuring that a defined logic coverage is performed. Informal tests on module level which will be done by the software development team and are informal tests which are necessary to check that the coded software modules reflect the requirements and design for that module. White-box oriented testing in combination with at least one black box method is used. Unreachability: A statement (or logical branch) is unreachable if there is no logically obtainable set of input-space settings which can cause the statement (or logical branch) to be traversed. Validation: The process of evaluation software at the end of the software development process to ensure compliance with software requirements. The techniques for validation are testing, inspection and reviewing. Verification: The process of determining whether or not the products of a given phase of the software development cycle meet the implementation steps and can be traced to the incoming objectives established during the previous phase. The techniques for verification are testing, inspection and reviewing.


White-box testing: A test method where the tester views the internal behavior and structure of the program. The testing strategy permits one to examine the internal structure of the program. In using this strategy, the tester derives test data from an examination of the program's logic without neglecting the requirements in the specification. The goal of this test method is to achieve a high-test coverage, which is examination of as much of the statements, branches, paths as possible.


Points to ponder

1. What testing approaches can you tell me about?A: Each of the followings represents a different testing approach: black box testing, white box testing, unit testing, incremental testing, integration testing, functional testing, system testing, end-to-end testing, sanity testing, regression testing, acceptance testing, load testing, performance testing, usability testing, install/uninstall testing, recovery testing, security testing, compatibility testing, exploratory testing, ad-hoc testing, user acceptance testing, comparison testing, alpha testing, beta testing, and mutation testing.

2. What is stress testing?A: Stress testing is testing that investigates the behavior of software (and hardware) under extraordinary operating conditions. For example, when a web server is stress tested, testing aims to find out how many users can be on-line, at the same time, without crashing the server. Stress testing tests the stability of a given system or entity. Stress testing tests something beyond its normal operational capacity, in order to observe any negative results.

3. What is load testing?A: Load testing simulates the expected usage of a software program, by simulating multiple users that access the program's services concurrently. Load testing is most useful and most relevant for multi-user systems, client/server models, including web servers. For example, the load placed on the system is increased above normal usage patterns, in order to test the system's response at peak loads.

4. What is the difference between stress testing and load testing?A: Load testing generally stops short of stress testing. During stress testing, the load is so great that the expected results are errors, though there is gray area in between stress testing and load testing. Load testing is a blanket term that is used in many different ways across the professional software testing community. The term, load testing, is often used synonymously with stress testing, performance testing, reliability testing, and volume testing.

5. What is the difference between performance testing and load testing?A: Load testing is a blanket term that is used in many different ways across the professional software testing community. The term, load testing, is often used synonymously with stress testing, performance testing, reliability testing, and


volume testing. Load testing generally stops short of stress testing. During stress testing, the load is so great that errors are the expected results, though there is gray area in between stress testing and load testing.

6. What is the difference between reliability testing and load testing?A: Load testing is a blanket term that is used in many different ways across the professional software testing community. The term, load testing, is often used synonymously with stress testing, performance testing, reliability testing, and volume testing. Load testing generally stops short of stress testing. During stress testing, the load is so great that errors are the expected results, though there is gray area in between stress testing and load testing.

7. What is automated testing?A: Automated testing is a formally specified and controlled method of formal testing approach.

8. What is the difference between volume testing and load testing?A: Load testing is a blanket term that is used in many different ways across the professional software testing community. The term, load testing, is often used synonymously with stress testing, performance testing, reliability testing, and volume testing. Load testing generally stops short of stress testing. During stress testing, the load is so great that errors are the expected results, though there is gray area in between stress testing and load testing.

9. What is incremental testing?A: Incremental testing is partial testing of an incomplete product. The goal of incremental testing is to provide an early feedback to software developers.

10. What is software testing?A: Software testing is a process that identifies the correctness, completeness, and quality of software. Actually, testing cannot establish the correctness of software. It can find defects, but cannot prove there are no defects.

11. What is alpha testing?A: Alpha testing is final testing before the software is released to the general public. First, (and this is called the first phase of alpha testing), the software is tested by in-house developers. They use either debugger software, or hardware assisted debuggers. The goal is to catch bugs quickly. Then, (and this is called second stage of alpha testing), the software is handed over to software QA staff for additional testing in an environment that is similar to the intended use.


12. What is beta testing?A: Following alpha testing, "beta versions" of the software are released to a group of people, and limited public tests are performed, so that further testing can ensure the product has few bugs. Other times, beta versions are made available to the general public, in order to receive as much feedback as possible. The goal is to benefit the maximum number of future users.

13. What is the difference between alpha and beta testing?A: Alpha testing is performed by in-house developers and software QA personnel. Beta testing is performed by the public, a few select prospective customers, or the general public.

14. What is gamma testing?A: Gamma testing is testing of software that has all the required features, but it did not go through all the in-house quality checks. Cynics tend to refer to software releases as "gamma testing".

15. What is boundary value analysis?A: Boundary value analysis is a technique for test data selection. A test engineer chooses values that lie along data extremes. Boundary values include maximum, minimum, just inside boundaries, just outside boundaries, typical values, and error values. The expectation is that, if a systems works correctly for these extreme or special values, then it will work correctly for all values in between. An effective way to test code is to exercise it at its natural boundaries.

16. What is ad hoc testing?A: Ad hoc testing is a testing approach; it is the least formal testing approach.

17. What is clear box testing?A: Clear box testing is the same as white box testing. It is a testing approach that examines the application's program structure, and derives test cases from the application's program logic.

18. What is glass box testing?A: Glass box testing is the same as white box testing. It is a testing approach that examines the application's program structure, and derives test cases from the application's program logic.


19. What is open box testing?A: Open box testing is the same as white box testing. It is a testing approach that examines the application's program structure, and derives test cases from the application's program logic.

20. What is black box testing?A: Black box testing a type of testing that considers only externally visible behavior. Black box testing considers neither the code itself, nor the "inner workings" of the software.

21. What is functional testing?A: Functional testing is the same as black box testing. Black box testing a type of testing that considers only externally visible behavior. Black box testing considers neither the code itself, nor the "inner workings" of the software.

22. What is closed box testing?A: Closed box testing is the same as black box testing. Black box testing a type of testing that considers only externally visible behavior. Black box testing considers neither the code itself, nor the "inner workings" of the software.

23. What is bottom-up testing?A: Bottom-up testing is a technique for integration testing. A test engineer creates and uses test drivers for components that have not yet been developed, because, with bottom-up testing, low-level components are tested first. The objective of bottom-up testing is to call low-level components, for testing purposes.

24. What is software quality?A: The quality of the software does vary widely from system to system. Some common quality attributes are stability, usability, reliability, portability, and maintainability. See quality standard ISO 9126 for more information on this subject.

25. What is software fault?A: A software fault is a hidden programming error. A software fault is an error in the correctness of the semantics of a computer program.

26. What is software failure?A: Software failure occurs when the software does not do what the user expects to see.


27. What is the difference between a software fault and software failure?A: A software failure occurs when the software does not do what the user expects to see. Software faults, on the other hand, are hidden programming errors. Software faults become software failures only when the exact computation conditions are met, and the faulty portion of the code is executed on the CPU. This can occur during normal usage. Other times it occurs when the software is ported to a different hardware platform, or, when the software is ported to a different complier, or, when the software gets extended.

28. Who is a test engineer?A: We, test engineers, are engineers who specialize in testing. We create test cases, procedures, scripts and generate data. We execute test procedures and scripts, analyze standards of measurements, and evaluate results of system/integration/regression testing.

29. Who is a QA engineer?A: QA engineers are test engineer, but they do more than just testing. Good QA engineers understand the entire software development process and how it fits into the business approach and the goals of the organization. Communication skills and the ability to understand various sides of issues are important. A QA engineer is successful if people listen to him, if people use his tests, if people think that he's useful, and if he's happy doing his work. I would love to see QA departments staffed with experienced software developers who coach development teams to write better code. But I've never seen it. Instead of coaching, QA engineers tend to be process people.

30. How do test case templates look like?A: Software test cases are documents that describe inputs, actions, or events and their expected results, in order to determine if all features of an application are working correctly. A software test case template is, for example, a 6-column table, where column 1 is the "Test case ID number", column 2 is the "Test case name", column 3 is the "Test objective", column 4 is the "Test conditions/setup", column 5 is the "Input data requirements/steps", and column 6 is the "Expected results". All documents should be written to a certain standard and template. Standards and templates maintain document uniformity. It also helps in learning where information is located, making it easier for a user to find what they want. Lastly, with standards and templates, information will not be accidentally omitted from a document


31. What is the role of the test engineer?A: We, test engineers, speed up the work of the development staff, and reduce the risk of your company's legal liability. We also give your company the evidence that the software is correct and operates properly. We, test engineers, improve problem tracking and reporting, maximize the value of the software, and the value of the devices that use it. We, test engineers, assure the successful launch of the product by discovering bugs and design flaws, before users get discouraged, before shareholders loose their cool and before employees get bogged down. We, test engineers, help the work of the software development staff, so the development team can devote its time to build up the product. We, test engineers, promote continual improvement. We provide documentation required by FDA, FAA, other regulatory agencies, and your customers. We, test engineers, save your company Helping companies solve and avoid FDA problems and get FDA approvals money by discovering defects EARLY in the design process, before failures occur in production, or in the field. We save the reputation of your company by discovering bugs and design flaws, before bugs and design flaws damage the reputation of your company.

32. What are the QA engineer's responsibilities?A: Let's say, an engineer is hired for a small software company's QA role, and there is no QA team. Should he take responsibility to set up a QA infrastructure/process, testing and quality of the entire product? No, because taking this responsibility is a classic trap that QA people get caught in. Why? Because QA engineers cannot assure quality and QA departments cannot create quality. What we CAN do is to detect lack of quality, and prevent low-quality products from going out the door. What is the solution? We need to drop the QA label, and tell the developers that they are responsible for the quality of their own work. The problem is, sometimes, as soon as the developers learn that there is a test department, they will slack off on their testing. We need to offer to help with quality assessment, only.

33. What metrics can be used in for software development?A: Metrics refer to statistical process control. The idea of statistical process control is a great one, but it has only a limited use in software development. On the negative side, statistical process control works only with processes that are sufficiently well defined AND unvaried, so that they can be analyzed in terms of statistics. The problem is, most software development projects are NOT sufficiently well defined and NOT sufficiently unvaried. On the positive side, one CAN use statistics. Statistics are excellent tools that project managers can use. Statistics can be used, for example, to determine when to stop testing, i.e. test


cases completed with certain percentage passed, or when bug rate falls below a certain level. But, if these are project management tools, why should we label them quality assurance tools?

34. What is role of the QA engineer?A: The QA Engineer's function is to use the system much like real users would, find all the bugs, find ways to replicate the bugs, submit bug reports to the developers, and to provide feedback to the developers, i.e. tell them if they've achieved the desired level of quality.

35. What metrics can be used for bug tracking?A: Metrics that can be used for bug tracking include the total number of bugs, total number of bugs that have been fixed, number of new bugs per week, and number of fixes per week. Other metrics in quality assurance include...McCabe metrics: cyclomatic complexity metric (v(G)), actual complexity metric (AC), module design complexity metric (iv(G)), essential complexity metric (ev (G)), pathological complexity metric (pv (G)), design complexity metric (S0), integration complexity metric (S1), object integration complexity metric (OS1), global data complexity metric (gdv(G)), data complexity etric (DV), tested data complexity metric (TDV), data reference metric (DR), tested data reference etric (TDR), maintenance severity metric (maint_severity), data reference severity metric (DR_severity), data complexity severity metric (DV_severity), global data severity metric (gdv_severity).

36. What metrics can be used for bug tracking? (Cont'd...)McCabe object-oriented software metrics: encapsulation percent public data (PCTPUB), access to public data (PUBDATA), polymorphism percent of unoverloaded calls (PCTCALL), number of roots (ROOTCNT), fan-in (FANIN), quality maximum v(G) (MAXV), maximum ev(G) (MAXEV), and hierarchy quality (QUAL). Other object-oriented software metrics: depth (DEPTH), lack of cohesion of methods (LOCM), number of children (NOC), response for a class (RFC), weighted methods per class (WMC), Halstead software metrics program length, program volume, program level and program difficulty, intelligent content, programming effort, error estimate, and programming time.Line count software metrics: lines of code, lines of comment, lines of mixed code and comments, and lines left blank.

37. How do you perform integration testing?A: First, unit testing has to be completed. Upon completion of unit testing, integration testing begins. Integration testing is black box testing. The purpose of


integration testing is to ensure distinct components of the application still work in accordance to customer requirements. Test cases are developed with the express purpose of exercising the interfaces between the components. This activity is carried out by the test team. Integration testing is considered complete, when actual results and expected results are either in line or differences are explainable/acceptable based on client input.

38. What metrics are used for test report generation?A: Metrics that can be used for test report generation include...McCabe metrics: Cyclomatic complexity metric (v(G)), Actual complexity metric (AC), Module design complexity metric (iv(G)), Essential complexity metric (ev(G)), Pathological complexity metric (pv(G)), design complexity metric (S0), Integration complexity metric (S1), Object integration complexity metric (OS1), Global data complexity metric (gdv(G)), Data complexity metric (DV), Tested data complexity metric (TDV), Data reference metric (DR), Tested data reference metric (TDR), Maintenance severity metric (maint_severity), Data reference severity metric (DR_severity), Data complexity severity metric (DV_severity), Global data severity metric (gdv_severity).McCabe object oriented software metrics: Encapsulation percent public data (PCTPUB), and Access to public data (PUBDATA), Polymorphism percent of unoverloaded calls (PCTCALL), Number of roots (ROOTCNT), Fan-in (FANIN), quality maximum v(G) (MAXV), Maximum ev(G) (MAXEV), and Hierarchy quality(QUAL). Other object oriented software metrics: Depth (DEPTH), Lack of cohesion of methods (LOCM), Number of children (NOC), Response for a class (RFC), Weighted methods per class (WMC), Halstead software metrics program length, Program volume, Program level and program difficulty, Intelligent content, Programming effort, Error estimate, and Programming time. Line count software metrics: Lines of code, Lines of comment, Lines of mixed code and comments, and Lines left blank.

39. What is the "bug life cycle"?A: Bug life cycles are similar to software development life cycles. At any time during the software development life cycle errors can be made during the gathering of requirements, requirements analysis, functional design, internal design, documentation planning, document preparation, coding, unit testing, test planning, integration, testing, maintenance, updates, retesting and phase-out. Bug life cycle begins when a programmer, software developer, or architect makes a mistake, creates an unintentional software defect, i.e. a bug, and ends when the bug is fixed, and the bug is no longer in existence. What should be done after a


bug is found? When a bug is found, it needs to be communicated and assigned to developers that can fix it. After the problem is resolved, fixes should be retested. Additionally, determinations should be made regarding requirements, software, hardware, safety impact, etc., for regression testing to check the fixes didn't create other problems elsewhere. If a problem-tracking system is in place, it should encapsulate these determinations. A variety of commercial, problem tracking/management software tools are available. These tools, with the detailed input of software test engineers, will give the team complete information so developers can understand the bug, get an idea of its severity, reproduce it and fix it.


40. What is integration testing?A: Integration testing is black box testing. The purpose of integration testing is to ensure distinct components of the application still work in accordance to customer requirements. Test cases are developed with the express purpose of exercising the interfaces between the components. This activity is carried out by the test team. Integration testing is considered complete, when actual results and expected results are either in line or differences are explainable / acceptable, based on client input.

41. How do test plan templates look like?A: The test plan document template describes the objectives, scope, approach and focus of a software testing effort. Test document templates are often in the form of documents that are divided into sections and subsections. One example of this template is a 4-section document, where section 1 is the "Test Objective", section 2 is the "Scope of Testing", and section 3 is the "Test Approach", and section 4 is the "Focus of the Testing Effort". All documents should be written to a certain standard and template. Standards and templates maintain document uniformity. It also helps in learning where information is located, making it easier for a user to find what they want. With standards and templates, information will not be accidentally omitted from a document. Once this document is reviewed for standards it will be recommended for improvements and/or additions.

42. What is a software project test plan?A: A software project test plan is a document that describes the objectives, scope, approach and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the why and how of product validation. It should be thorough enough to be useful, but not so thorough that no one outside the test group will be able to read it.

43. When do you choose automated testing?A: For larger projects, or ongoing long-term projects, automated testing can be valuable. But for small projects, the time needed to learn and implement the automated testing tools is usually not worthwhile. Automated testing tools sometimes do not make testing easier. One problem with automated testing tools is that if there are continual changes to the product being tested, the recordings have to be changed so often, that it becomes a very time-consuming task to continuously update the scripts. Another problem with such tools is the


interpretation of the results (screens, data, logs, etc.) that can be a time consuming task.


44. What's the ratio between developers and testers?A: This ratio is not a fixed one, but depends on what phase of the software development life cycle the project is in. When a product is first conceived, organized, and developed, this ratio tends to be 10:1, 5:1, or 3:1, i.e. heavily in favor of developers. In sharp contrast, when the software is near the end of alpha testing, this ratio tends to be 1:1 or 1:2, in favor of testers.

45. What is your role in your current organization if you are a QA Engineer?A: The QA Engineer's function is to use the system much like real users would, find all the bugs, find ways to replicate the bugs, submit bug reports to the developers, and to provide feedback to the developers, i.e. tell them if they've achieved the desired level of quality.

46. What software tools are in demand these days?A: There is no good answer to this question. The answer to this question can and will change from day to day. What is in demand today is not necessarily in demand tomorrow. To give you some recent examples, some of the software tools on end clients' lists of requirements include LabView, LoadRunner, Rational Tools, Winrunner and QTP. But, as a general rule of thumb, there are many-many other items on their lists, depending on the end client, their needs and preferences. It is worth repeating... the answer to this question can and will change from one day to the next. What is in demand today will not likely be in demand tomorrow.

47. What is software configuration management?A: Software Configuration management (SCM) relates to Configuration Management (CM). SCM is the control, and the recording of, changes that are made to the software and documentation throughout the software development life cycle (SDLC). SCM covers the tools and processes used to control, coordinate and track code, requirements, documentation, problems, change requests, designs, tools, compilers, libraries, patches, and changes made to them, and to keep track of who makes the changes. We, test engineers have experience with a full range of CM tools and concepts, and can easily adapt to an organization's software tool and process needs.

48. What are some of the software configuration management tools? A: Software configuration management tools include Rational ClearCase, DOORS, PVCS, CVS; and there are many others. Rational ClearCase is a popular software tool, made by Rational Software, for revision control of source code. DOORS or "Dynamic Object Oriented Requirements System" is a requirements version


control software tool. CVS, or "Concurrent Version System", is a popular, open source version control system to keep track of changes in documents associated with software projects. CVS enables several, often distant, developers to work together on the same source code. PVCS is a document version control tool, a competitor of SCCS. SCCS is an original UNIX program, based on "diff". Diff is a UNIX utility that compares the difference between two text files.

49. Which of these roles are the best and most popular?A: In testing, Tester roles tend to be the most popular. The less popular roles include the roles of System Administrator, Test/QA Team Lead, and Test/QA Managers.

50. What other roles are in testing?A: Depending on the organization, the following roles are more or less standard on most testing projects: Testers, Test Engineers, Test/QA Team Leads, Test/QA Managers, System Administrators, Database Administrators, Technical Analysts, Test Build Managers, and Test Configuration Managers. Depending on the project, one person can and often wears more than one hat. For instance, Test Engineers often wear the hat of Technical Analyst, Test Build Manager and Test Configuration Manager as well.

51. What's the difference between priority and severity?A: The simple answer is, "Priority is about scheduling, and severity is about standards." The complex answer is, "Priority means something is afforded or deserves prior attention; a precedence established by order of importance (or urgency). Severity is the state or quality of being severe; severe implies adherence to rigorous standards or high principles and often suggests harshness; severe is marked by or requires strict adherence to rigorous standards or high principles, e.g. a severe code of behavior."

52. What's the difference between efficient and effective?A: "Efficient" means having a high ratio of output to input; working or producing with a minimum of waste. For example, "An efficient test engineer wastes no time", or "An efficient engine saves gas". "Effective" on the other hand means producing or capable of producing an intended result, or having a striking effect. For example, "For automated testing, WinRunner/QTP is more effective than an oscilloscope", or "For rapid long distance transportation, the jet engine is more effective than a witch's broomstick".

53. What is the difference between verification and validation?


A: Verification takes place before validation, and not vice versa. Verification evaluates documents, plans, code, requirements, and specifications. Validation, on the other hand, evaluates the product itself. The inputs of verification are checklists, issues lists, walk-troughs and inspection meetings, reviews and meetings. The input of validation, on the other hand, is the actual testing of an actual product. The output of verification is a nearly perfect set of documents, plans, specifications, and requirements document. The output of validation, on the other hand, is a nearly perfect, actual product.

54. What is documentation change management?A: Documentation change management is part of configuration management (CM). CM covers the tools and processes used to control, coordinate and track code, requirements, documentation, problems, change requests, designs, tools, compilers, libraries, patches, changes made to them and who makes the changes.

55. What is up time?A: "Up time" is the time period when a system is operational and in service. Up time is the sum of busy time and idle time. For example, if, out of 168 hours, a system has been busy for 50 hours, idle for 110 hours, and down for 8 hours, then the busy time is 50 hours, idle time is 110 hours, and up time is (110 + 50 =) 160 hours.

56. What is upwardly compatible software?A: Upwardly compatible software is compatible with a later or more complex version of itself. For example, upwardly compatible software is able to handle files created by a later version of itself.

57. What is upward compression?A: In software design, upward compression means a form of demodularization, in which a subordinate module is copied into the body of a superior module.

58. What is usability?A: Usability means ease of use; the ease with which a user can learn to operate, prepares inputs for, and interprets outputs of a software product.

59. What is user documentation?A: User documentation is a document that describes the way a software product or system should be used to obtain the desired results.

60. What is a user manual?


A: User manual is a document that presents information necessary to employ software or a system to obtain the desired results. Typically, what is described are system and component capabilities, limitations, options, permitted inputs, expected outputs, error messages, and special instructions.

61. What is the difference between user documentation and user manual? A: When a distinction is made between those who operate and use a computer system for its intended purpose, separate user documentation and user manual is created. Operators get user documentation, and users get user manuals.

62. What is user friendly software?A: A computer program is user friendly, when it is designed with ease of use, as one of the primary objectives of its design.

63. What is a user friendly document?A: A document is user friendly, when it is designed with ease of use, as one of the primary objectives of its design.

64. What is a user guide?A: User guide is the same as the user manual. It is a document that presents information necessary to employ a system or component to obtain the desired results. Typically, what is described are system and component capabilities, limitations, options, permitted inputs, expected outputs, error messages, and special instructions.

65. What is user interface?A: User interface is the interface between a human user and a computer system. It enables the passage of information between a human user and hardware or software components of a computer system.

66. What is a utility?A: Utility is a software tool designed to perform some frequently used support function. For example: program to print files.

67. What is utilization?A: Utilization is the ratio of time a system is busy, divided by the time it is available. Utilization is a useful measure in evaluating computer performance.

68. What is V&V?


A: V&V is an acronym for verification and validation.

69. What is variable trace?A: Variable trace is a record of the names and values of variables accessed and changed during the execution of a computer program.

70. What is value trace?A: Value trace is same as variable trace. It is a record of the names and values of variables accessed and changed during the execution of a computer program.

71. What is a variable?A: Variables are data items whose values can change. One example is a variable we've named "capacitor_voltage_10000", where "capacitor_value_10000" can be any whole number between -10000 and +10000. Keep in mind, there are local and global variables.

72. What is a variant?A: Variants are versions of a program. Variants result from the application of software diversity.

73. What is verification and validation (V&V)?A: Verification and validation (V&V) is a process that helps to determine if the software requirements are complete, correct; and if the software of each development phase fulfills the requirements and conditions imposed by the pervious phase; and if the final software complies with the applicable software requirements.

74. What is a software version?A: A software version is an initial release (or re-release) of a software associated with a complete compilation (or recompilation) of the software.

75. What is a document version?A: A document version is an initial release (or a complete re-release) of a document, as opposed to a revision resulting from issuing change pages to a previous release.

76. What is VDD?A: VDD is an acronym. It stands for "version description document".

77. What is a version description document (VDD)?


A: Version description document (VDD) is a document that accompanies and identifies a given version of a software product. Typically the VDD includes a description, and identification of the software, identification of changes incorporated into this version, and installation and operating information unique to this version of the software.


78. What is a vertical microinstruction?A: A vertical microinstruction is a microinstruction that specifies one of a sequence of operations needed to carry out a machine language instruction. Vertical microinstructions are short, 12 to 24 bit instructions. They're called vertical because they are normally listed vertically on a page. These 12 to 24 bit microinstructions instructions are required to carry out a single machine language instruction. Besides vertical microinstructions, there are horizontal as well as diagonal microinstructions as well.

79. What is a virtual address?A: In virtual storage systems, virtual addresses are assigned to auxiliary storage locations. They allow those locations to be accessed as though they were part of the main storage.

80. What is virtual memory?A: Virtual memory relates to virtual storage. In virtual storage, portions of a user's program and data are placed in auxiliary storage, and the operating system automatically swaps them in and out of main storage as needed.

81. What is virtual storage?A: Virtual storage is a storage allocation technique, in which auxiliary storage can be addressed as though it was part of main storage. Portions of a user's program and data are placed in auxiliary storage, and the operating system automatically swaps them in and out of main storage as needed.

82. What is a waiver?A: Waivers are authorizations to accept software that has been submitted for inspection, found to depart from specified requirements, but is nevertheless considered suitable for use "as is", or after rework by an approved method.

83. What is the waterfall model?A: Waterfall is a model of the software development process in which the concept phase, requirements phase, design phase, implementation phase, test phase, installation phase, and checkout phase are performed in that order, probably with overlap, but with little or no iteration.

84. What are the phases of the software development process?A: Software development process consists of the concept phase, requirements phase, design phase, implementation phase, test phase, installation phase, and checkout phase.


85. What models are used in software development?A: In software development process the following models are used: waterfall model, incremental development model, rapid prototyping model, and spiral model.

86. What is SDLC?A: SDLC is an acronym. It stands for "software development life cycle".

87. What is the difference between system testing and integration testing?A: System testing is high level testing, and integration testing is a lower level testing. Integration testing is completed first, not the system testing. In other words, upon completion of integration testing, system testing is started, and not vice versa. For integration testing, test cases are developed with the express purpose of exercising the interfaces between the components. For system testing, on the other hand, the complete system is configured in a controlled environment, and test cases are developed to simulate real life scenarios that occur in a simulated real life test environment. The purpose of integration testing is to ensure distinct components of the application still work in accordance to customer requirements. The purpose of system testing, on the other hand, is to validate an application's accuracy and completeness in performing the functions as designed, and to test all functions of the system that are required in real life.

88. What are the parameters of performance testing?A: Performance testing verifies loads, volumes, and response times, as defined by requirements. Performance testing is a part of system testing, but it is also a distinct level of testing. The term 'performance testing' is often used synonymously with stress testing, load testing, reliability testing, and volume testing.

89. What is disaster recovery testing?A: Disaster recovery testing is testing how well the system recovers from disasters, crashes, hardware failures, or other catastrophic problems.

90. How do you conduct peer reviews?A: Peer reviews, sometimes called PDR, are formal meeting, more formalized than a walk-through, and typically consists of 3-10 people including the test lead, task lead (the author of whatever is being reviewed) and a facilitator (to make notes). The subject of the PDR is typically a code block, release, or feature, or document.


The purpose of the PDR is to find problems and see what is missing, not to fix anything. The result of the meeting is documented in a written report. Attendees should prepare for PDRs by reading through documents, before the meeting starts; most problems are found during this preparation. Why are PDRs so useful? Because PDRs are cost-effective methods of ensuring quality, because bug prevention is more cost effective than bug detection.

91. How do you check the security of your application?A: To check the security of an application, we can use security/penetration testing. Security/penetration testing is testing how well the system is protected against unauthorized internal or external access, or willful damage. This type of testing usually requires sophisticated testing techniques.

92. When testing the password field, what is your focus?A: When testing the password field, one needs to verify that passwords are encrypted.

93. What is the objective of regression testing?A: The objective of regression testing is to test that the fixes have not created any other problems elsewhere. In other words, the objective is to ensure the software has remained intact. A baseline set of data and scripts are maintained and executed, to verify that changes introduced during the release have not "undone" any previous code. Expected results from the baseline are compared to results of the software under test. All discrepancies are highlighted and accounted for, before testing proceeds to the next level.

94. What stage of bug fixing is the most cost effective?A: Bug prevention, i.e. inspections, PDRs, and walk-throughs, is more cost effective than bug detection.

95. What can you tell about white box testing?A: White box testing is a testing approach that examines the application's program structure, and derives test cases from the application's program logic. Clear box testing is a white box type of testing. Glass box testing is also a white box type of testing. Open box testing is also a white box type of testing.

96. What black box testing types can you tell me about?A: Black box testing is functional testing, not based on any knowledge of internal software design or code. Black box testing is based on requirements and functionality. Functional testing is also a black-box type of testing geared to


functional requirements of an application. System testing is also a black box type of testing. Acceptance testing is also a black box type of testing. Functional testing is also a black box type of testing. Closed box testing is also a black box type of testing. Integration testing is also a black box type of testing.

97. Is regression testing performed manually?A: It depends on the initial testing approach. If the initial testing approach was manual testing, then the regression testing is normally performed manually. Conversely, if the initial testing approach was automated testing, then the regression testing is normally performed by automated testing.

98. What is your view of software QA/testing?A: Software QA/testing is easy, if requirements are solid, clear, complete, detailed, cohesive, attainable and testable, and if schedules are realistic, and if there is good communication. Software QA/testing is a piece of cake, if project schedules are realistic, if adequate time is allowed for planning, design, testing, bug fixing, re-testing, changes, and documentation. Software QA/testing is relatively easy, if testing is started early on, and if fixes or changes are re-tested, and if sufficient time is planned for both testing and bug fixing. Software QA/testing is easy, if new features are avoided, and if one sticks to initial requirements as much as possible.

99. How can I be a good tester?A: We, good testers, take the customers' point of view. We are tactful and diplomatic. We have a "test to break" attitude, a strong desire for quality, an attention to detail, and good communication skills, both oral and written. Previous software development experience is also helpful as it provides a deeper understanding of the software development process.

100. What is the difference between software bug and software defect?A: A 'software bug' is a nonspecific term that means an inexplicable defect, error, flaw, mistake, failure, fault, or unwanted behavior of a computer program. Other terms, e.g. software defect and software failure, are more specific. While there are many who believe the term 'bug' is a reference to insects that caused malfunctions in early electromechanical computers (1950-1970), the term 'bug' had been a part of engineering jargon for many decades before the 1950s; even the great inventor, Thomas Edison (1847-1931), wrote about a 'bug' in one of his letters.

101. When is a process repeatable?


A: If we use detailed and well-written processes and procedures, we ensure the correct steps are being executed. This facilitates a successful completion of a task. This is a way we also ensure a process is repeatable.


102. What is test methodology?A: One test methodology is a three-step process. “Creating a test strategy”, “creating a test plan/design” and “executing tests”. This methodology can be used and molded to your organization's needs. Rob Davis believes that using this methodology is important in the development and ongoing maintenance of his customers' applications.

103. What does a Test Strategy Document contain?A: The test strategy document is a formal description of how a software product will be tested. A test strategy is developed for all levels of testing, as required. The test team analyzes the requirements, writes the test strategy and reviews the plan with the project team. The test plan may include test cases, conditions, the test environment, and a list of related tasks, pass/fail criteria and risk assessment. Additional sections in the test strategy document include: A description of the required hardware and software components, including test tools. This information comes from the test environment, including test tool data. A description of roles and responsibilities of the resources required for the test and schedule constraints. This information comes from man-hours and schedules. Testing methodology: This is based on known standards, functional and technical requirements of the application. This information comes from requirements, change request, technical, and functional design documents, requirements that the system cannot provide, ex: system limitations.

104. What is monkey testing?A: "Monkey testing" is random testing performed by automated testing tools. These automated testing tools are considered "monkeys", if they work at random. We call them "monkeys" because it is widely believed, if we allow six monkeys to pound on six typewriters at random, for a million years, they will recreate all the works of Isaac Asimov. There are "smart monkeys" and "dumb monkeys". "Smart monkeys" are valuable for load and stress testing, and will find a significant number of bugs, but they're also very expensive to develop. "Dumb monkeys", on the other hand, are inexpensive to develop, are able to do some basic testing, but they will find few bugs. However, the bugs "dumb monkeys" do find will be hangs and crashes, i.e. the bugs you least want to have in your software product. "Monkey testing" can be valuable, but they should not be your only testing.

105. What is stochastic testing?A: Stochastic testing is the same as "monkey testing", but stochastic testing is a more technical sounding name for the same testing process. Stochastic testing is black box testing, random testing, performed by automated testing tools.


Stochastic testing is a series of random tests over time. The software under test typically passes the individual tests, but our goal is to see if it can pass a large series of the individual tests.

106. What is mutation testing?A: In mutation testing, we create mutant software, we make mutant software to fail, and thus demonstrate the adequacy of our test case. When we create a set of mutant software each mutant software, differs from the original software by one mutation, i.e. one single syntax change made to one of its program statements, i.e. each mutant software contains only one single fault.When we apply test cases to the original software and to the mutant software, we evaluate if our test case is adequate. Our test case is inadequate, if both the original software and all mutant software generate the same output. Our test case is adequate, if our test case detects faults or if at least one mutant software generates a different output than does the original software for our test case.

107. What is PDR?A: PDR is an acronym. In the world of software QA/testing, it stands for "peer design review", or "peer review".

108. What is good about PDRs?A: PDRs are informal meetings. PDRs make perfect sense, because they're for the mutual benefit of you and your end client. Your end client requires a PDR, because they work on a product, and want to come up with the very best possible design and documentation. Your end client requires you to have a PDR, because when you organize a PDR, you invite and assemble the end client's best experts and encourage them to voice their concerns as to what should or should not go into the design and documentation, and why. When you're a developer, designer, author, or writer, it's also to your advantage to come up with the best possible design and documentation. Therefore you want to embrace the idea of the PDR, because holding a PDR gives you a significant opportunity to invite and assemble the end client's best experts and make them work for you for one hour, for your own benefit. To come up with the best possible design and documentation, you want to encourage your end client's experts to speak up and voice their concerns as to what should or should not go into your design and documentation, and why.

109. Why is that My Company requires a PDR?A: Your Company requires a PDR, because your company wants to be the owner of the very best possible design and documentation. Your company requires a PDR, because when you organize a PDR, you invite, assemble and encourage the


company's best experts to voice their concerns as to what should or should not go into your design and documentation, and why. Remember, PDRs are not about you, but about design and documentation. Please don't be negative; please do not assume your company is finding fault with your work, or distrusting you in any way. There is a 90+ per cent probability your company wants you, likes you and trusts you, because you're a specialist, and because your company hired you after a long and careful selection process. Your company requires a PDR, because PDRs are useful and constructive. Just about everyone - even corporate chief executive officers (CEOs) - attend PDRs from time to time. When a corporate CEO attends a PDR, he has to listen for "feedback" from shareholders. When a CEO attends a PDR, the meeting is called the "annual shareholders' meeting".

110. Give me a list of ten good things about PDRs!A:

1. PDRs are easy, because all your meeting attendees are your coworkers and friends.

2. PDRs do produce results. With the help of your meeting attendees, PDRs help you produce better designs and better documents than the ones you could come up with, without the help of your meeting attendees.

3. Preparation for PDRs helps a lot, but, in the worst case, if you had no time to read every page of every document, it's still OK for you to show up at the PDR.

4. It's technical expertise that counts the most, but many times you can influence your group just as much, or even more so, if you're dominant or have good acting skills.

5. PDRs are easy, because, even at the best and biggest companies, you can dominate the meeting by being either very negative, or very bright and wise.

6. It is easy to deliver gentle suggestions and constructive criticism. The brightest and wisest meeting attendees are usually gentle on you; they deliver gentle suggestions that are constructive, not destructive.

7. You get many-many chances to express your ideas, every time a meeting attendee asks you to justify why you wrote what you wrote.

8. PDRs are effective, because there is no need to wait for anything or anyone; because the attendees make decisions quickly (as to what errors are in your document). There is no confusion either, because all the group's recommendations are clearly written down for you by the PDR's facilitator.


9. Your work goes faster, because the group itself is an independent decision making authority. Your work gets done faster, because the group's decisions are subject to neither oversight nor supervision.

10. At PDRs, your meeting attendees are the very best experts anyone can find, and they work for you, for FREE!

111. What are the Exit criteria?A: "Exit criteria" is a checklist, sometimes known as the "PDR sign-off sheet", i.e. a list of peer design review related tasks that have to be done by the facilitator or other attendees of the PDR, during or near the conclusion of the PDR. By having a checklist, and by going through a checklist, the facilitator can... 1. Verify that the attendees have inspected all the relevant documents and reports, and2. Verify that all suggestions and recommendations for each issue have been recorded, and3. Verify that all relevant facts of the meeting have been recorded. The facilitator's checklist includes the following questions:1. "Have we inspected all the relevant documents, code blocks, or products?"2. "Have we completed all the required checklists?"3. "Have I recorded all the facts relevant to this peer review?"4. "Does anyone have any additional suggestions, recommendations, or comments?"5. "What is the outcome of this peer review?" At the end of the peer review, the facilitator asks the attendees of the peer review to make a decision as to the outcome of the peer review. I.e., "What is our consensus?" "Are we accepting the design (or document or code)?" Or, "Are we accepting it with minor modifications?” Or "Are we accepting it, after it is modified, and approved through e-mails to the participants?" Or "Do we want another peer review?" This is a phase, during which the attendees of the PDR work as a committee, and the committee's decision is final.

112. What are the Entry criteria?A: The entry criteria is a checklist, or a combination of checklists that includes the "developer's checklist", "testing checklist", and the "PDR checklist". Checklists are list of tasks that have to be done by developers, testers, or the facilitator, at or before the start of the peer review. Using these checklists, before the start of the peer review, the developer, tester and facilitator can determine if all the documents, reports, code blocks or software products are ready to be reviewed, and if the peer review's attendees are prepared to inspect them. The facilitator can ask the peer review's attendees if they have been able to prepare for the peer


review, and if they're not well prepared, the facilitator can send them back to their desks, and even ask the task lead to reschedule the peer review. The facilitator's script for the entry criteria includes the following questions:1. Are all the required attendees present at the peer review?2. Have all the attendees received all the relevant documents and reports?3. Are all the attendees well prepared for this peer review?4. Have all the preceding life cycle activities been concluded?5. Are there any changes to the baseline?

113. What are the parameters of peer reviews?A: By definition, parameters are values on which something else depends. Peer reviews depend on the attendance and active participation of several key people; usually the facilitator, task lead, test lead, and at least one additional reviewer. The attendances of these four people are usually required for the approval of the PDR. According to company policy, depending on your company, other participants are often invited, but generally not required for approval. Peer reviews depend on the facilitator, sometimes known as the moderator, who controls the meeting, keeps the meeting on schedule, and records all suggestions from all attendees. Peer reviews greatly depend on the developer, also known as the designer, author, or task lead, usually a software engineer, who is most familiar with the project, and most likely able to answer any questions or address any concerns that may come up during the peer review. Peer reviews greatly depend on the tester, also known as test lead, or bench test person -- usually another software engineer -- who is also familiar with the project, and most likely able to answer any questions or address any concerns that may come up during the peer review. Peer reviews greatly depend on the participation of additional reviewers and additional attendees who often make specific suggestions and recommendations, and ask the largest number of questions.

114. What types of review meetings can you tell me about?A: Of review meetings, peer design reviews are the most common. Peer design reviews are so common that they tend to replace both inspections and walk-throughs. Peer design reviews can be classified according to the 'subject' of the review. I.e., "Is this a document review, design review, or code review?" Peer design reviews can be classified according to the 'role' you play at the meeting. I.e., "Are you the task lead, test lead, facilitator, moderator, or additional reviewer?" Peer design reviews can be classified according to the 'job title of attendees. I.e., "Is this a meeting of peers, managers, systems engineers, or system integration testers?" Peer design reviews can be classified according to what is being reviewed at the meeting. I.e., "Are we reviewing the work of a


developer, tester, engineer, or technical document writer?" Peer design reviews can be classified according to the 'objective' of the review. I.e., "Is this document for the file cabinets of our company, or that of the government (e.g. the FAA or FDA)?" PDRs of government documents tend to attract the attention of managers, and the meeting quickly becomes a meeting of managers.


115. How can I shift my focus and area of work from QC to QA?A: 1. Focus on your strengths, skills, and abilities! Realize that there are MANY similarities between Quality Control and Quality Assurance! Realize that you have MANY transferable skills! 2. Make a plan! Develop a belief that getting a job in QA is easy! HR professionals cannot tell the difference between quality control and quality assurance! HR professionals tend to respond to keywords (i.e. QC and QA), without knowing the exact meaning of those keywords! 3. Make it a reality! Invest your time! Get some hands-on experience! Do some QA work! Do any QA work, even if, for a few months, you get paid a little less than usual! Your goals, beliefs, enthusiasm, and action will make a huge difference in your life! 4. Read all you can, and that includes reading product pamphlets, manuals, books, information on the Internet, and whatever information you can lay your hands on! If there is a will, there is a way! You CAN do it, if you put your mind to it! You CAN learn to do QA work, with little or no outside help!

116. What techniques and tools can enable me to migrate from QC to QA?A: Refer to above answers for question#115

117. What is the difference between build and release?A: Builds and releases are similar, because both builds and releases are end products of software development processes. Builds and releases are similar, because both builds and releases help developers and QA teams to deliver reliable software. Build means a version of software, typically one that is still in testing. Usually a version number is given to a released product, but, sometimes, a build number is used instead. Difference #1: Builds refer to software that is still in testing, release refers to software that is usually no longer in testing. Difference #2: Builds occur more frequently; releases occur less frequently. Difference #3: Versions are based on builds, and not vice versa. Builds, or usually a series of builds, are generated first, as often as one build per every morning, depending on the company, and then every release is based on a build, or several builds, i.e. the accumulated code of several builds.

118. What is CMM?A: CMM is an acronym that stands for Capability Maturity Model. The idea of CMM is, as to future efforts in developing and testing software, concepts and


experiences do not always point us in the right direction, and therefore we should develop processes, and then refine those processes. There are five CMM levels, of which Level 5 is the highest;CMM Level 1 is called "Initial".CMM Level 2 is called "Repeatable".CMM Level 3 is called "Defined".CMM Level 4 is called "Managed".CMM Level 5 is called "Optimized".There are not many Level 5 companies; most hardly need to be. Within the United States, fewer than 8% of software companies are rated CMM Level 4, or higher. The U.S. government requires that all companies with federal government contracts to maintain a minimum of a CMM Level 3 assessment. CMM assessments take two weeks. They're conducted by a nine-member team led by a SEI-certified lead assessor.

119. What are CMM levels and their definitions?A: There are five CMM levels of which level 5 is the highest.

CMM level 1 is called "initial". The software process is at CMM level 1, if it is an ad hoc process. At CMM level 1, few processes are defined, and success, in general, depends on individual effort and heroism.

CMM level 2 is called "repeatable". The software process is at CMM level 2, if the subject company has some basic project management processes, in order to track cost, schedule, and functionality. Software processes are at CMM level 2, if necessary processes are in place, in order to repeat earlier successes on projects with similar applications. Software processes are at CMM level 2, if there are requirements management, project planning, project tracking, subcontract management, QA, and configuration management.

CMM level 3 is called "defined". The software process is at CMM level 3, if the software process is documented, standardized, and integrated into a standard software process for the subject company. The software process is at CMM level 3, if all projects use approved, tailored versions of the company's standard software process for developing and maintaining software. Software processes are at CMM level 3, if there are process definition, training programs, process focus, integrated software management, software product engineering, inter group coordination, and peer reviews.

CMM level 4 is called "managed". The software process is at CMM level 4, if the subject company collects detailed data on the software process and product quality, and if both the software process and the software


products are quantitatively understood and controlled. Software processes are at CMM level 4, if there is software quality management (SQM) and quantitative process management.

CMM level 5 is called "optimized". The software process is at CMM level 5, if there is continuous process improvement, if there is quantitative feedback from the process and from piloting innovative ideas and technologies. Software processes are at CMM level 5, if there are process change management, and defect prevention technology change management.

120. What is the difference between bug and defect in software testing? A: In software testing, the difference between bug and defect is small, and depends on your company. For some companies, bug and defect are synonymous, while others believe bug is a subset of defect. Generally speaking, we, software test engineers, discover BOTH bugs and defects, before bugs and defects damage the reputation of our company. We, QA engineers, use the software much like real users would, to find BOTH bugs and defects, to find ways to replicate BOTH bugs and defects, to submit bug reports to the developers, and to provide feedback to the developers, i.e. tell them if they've achieved the desired level of quality. Therefore, we, software engineers, do not differentiate between bugs and defects. In our bug reports, we include BOTH bugs and defects, and any differences between them are minor. Difference number one: In bug reports, the defects are usually easier to describe. Difference number two: In bug reports, it is usually easier to write the descriptions on how to replicate the defects. Defects tend to require brief explanations only.

121. What is grey box testing?A: Grey box testing is a software testing technique that uses a combination of black box testing and white box testing. Gray box testing is not black box testing, because the tester does know some of the internal workings of the software under test.

In grey box testing, the tester applies a limited number of test cases to the internal workings of the software under test. In the remaining part of the grey box testing, one takes a black box approach in applying inputs to the software under test and observing the outputs. Gray box testing is a powerful idea. The concept is simple; if one knows something about how the product works on the inside, one can test it better, even from the outside.


Grey box testing is not to be confused with white box testing; i.e. a testing approach that attempts to cover the internals of the product in detail. Grey box testing is a test strategy based partly on internals. The testing approach is known as gray box testing, when one does have some knowledge, but not the full knowledge of the internals of the product one is testing. In gray box testing, just as in black box testing, you test from the outside of a product, just as you do with black box, but you make better-informed testing choices because you're better informed; Because you know how the underlying software components operate and interact.

122. What is the difference between version and release?A: Both version and release indicate a particular point in the software development life cycle, or in the lifecycle of a document. The two terms, version and release, are similar (i.e. mean pretty much the same thing), but there are minor differences between them. Version means a VARIATION of an earlier, or original, type; for example, "I've downloaded the latest version of the software from the Internet. The latest version number is 3.3." Release, on the other hand, is the ACT OR INSTANCE of issuing something for publication, use, or distribution. Release is something thus released. For example: "A new release of a software program."

123. What is data integrity? A: Data integrity is one of the six fundamental components of information security. Data integrity is the completeness, soundness, and wholeness of the data that also complies with the intention of the creators of the data. In databases, important data – including customer information, order database, and pricing tables -- may be stored. In databases, data integrity is achieved by preventing accidental, or deliberate, or unauthorized insertion, or modification, or destruction of data.

124. How do you test data integrity?A: Data integrity testing should verify the completeness, soundness, and wholeness of the stored data. Testing should be performed on a regular basis, because important data can and will change over time. Data integrity tests include the followings:1. Verify that you can create, modify, and delete any data in tables.2. Verify that sets of radio buttons represent fixed sets of values.3. Verify that a blank value can be retrieved from the database.


4. Verify that, when a particular set of data is saved to the database, each value gets saved fully, and the truncation of strings and rounding of numeric values do not occur.5. Verify that the default values are saved in the database, if the user input is not specified.6. Verify compatibility with old data, old hardware, versions of operating systems, and interfaces with other software.

125. What is data validity?A: Data validity is the correctness and reasonableness of data. Reasonableness of data means, for example, account numbers falling within a range, numeric data being all digits, dates having a valid month, day and year, spelling of proper names. Data validity errors are probably the most common, and the most difficult to detect, data-related errors. What causes data validity errors? Data validity errors are usually caused by incorrect data entries, when a large volume of data is entered in a short period of time. For example, 12/25/2005 is entered as 13/25/2005 by mistake. This date is therefore invalid. How can you reduce data validity errors? Use simple field validation rules. Technique 1: If the date field in a database uses the MM/DD/YYYY format, then use a program with the following two data validation rules: "MM should not exceed 12, and DD should not exceed 31". Technique 2: If the original figures do not seem to match the ones in the database, then use a program to validate data fields. Compare the sum of the numbers in the database data field to the original sum of numbers from the source. If there is a difference between the figures, it is an indication of an error in at least one data element.

126. What is the difference between data validity and data integrity?A:

Difference number one: Data validity is about the correctness and reasonableness of data, while data integrity is about the completeness, soundness, and wholeness of the data that also complies with the intention of the creators of the data.

Difference number two: Data validity errors are more common, while data integrity errors are less common. Difference number three: Errors in data validity are caused by HUMANS -- usually data entry personnel – who enter, for example, 13/25/2005, by mistake, while errors in data integrity are caused by BUGS in computer programs that, for example, cause the overwriting of some of the data in the database, when one attempts to retrieve a blank value from the database.


127. Tell me about 'Test Director'.A: Made by Mercury Interactive, 'Test Director' is a single browser-based application that streamlines the software QA process. It is a software tool that helps software QA professionals to gather requirements, to plan, schedule and run tests, and to manage and track defects/issues/bugs. Test Director’s Requirements Manager links test cases to requirements, ensures traceability, and calculates what percentage of the requirements are covered by tests, how many of these tests have been run, and how many have passed or failed. As to planning, test plans can be created, or imported, for both manual and automated tests. The test plans can then be reused, shared, and preserved. As to running tests, the Test Director’s Test Lab Manager allows you to schedule tests to run unattended, or run even overnight. The Test Director’s Defect Manager supports the entire bug life cycle, from initial problem detection through fixing the defect, and verifying the fix. Additionally, the Test Director can create customizable graphs and reports, including test execution reports and release status assessments.

128. What is structural testing?A: Structural testing is also known as clear box testing, glass box testing. Structural testing is a way to test software with knowledge of the internal workings of the code being tested. Structural testing is white box testing, not black box testing, since black boxes are considered opaque and do not permit visibility into the code.

129. What is the difference between static and dynamic testing?A: The differences between static and dynamic testing are as follows:

#1: Static testing is about prevention, dynamic testing is about cure. #2: She static tools offer greater marginal benefits. #3: Static testing is many times more cost-effective than dynamic testing. #4: Static testing beats dynamic testing by a wide margin. #5: Static testing is more effective! #6: Static testing gives you comprehensive diagnostics for your code. #7: Static testing achieves 100% statement coverage in a relatively short

time, while dynamic testing often achieves less than 50% statement coverage, because dynamic testing finds bugs only in parts of the code that are actually executed.

#8: Dynamic testing usually takes longer than static testing. Dynamic testing may involve running several test cases, each of which may take longer than compilation.

#9: Dynamic testing finds fewer bugs than static testing.


#10: Static testing can be done before compilation, while dynamic testing can take place only after compilation and linking.

#11: Static testing can find all of the following that dynamic testing cannot find: syntax errors, code that is hard to maintain, code that is hard to test, code that does not conform to coding standards, and ANSI violations.

130. What testing tools should I use?A: Ideally, you should use both static and dynamic testing tools. To maximize software reliability, you should use both static and dynamic techniques, supported by appropriate static and dynamic testing tools. Static and dynamic testing are complementary. Static and dynamic testing finds different classes of bugs. Some bugs are detectable only by static testing, some only by dynamic.Dynamic testing does detect some errors that static testing misses. To eliminate as many errors as possible, both static and dynamic testing should be used. All this static testing (i.e. testing for syntax errors, testing for code that is hard to maintain, testing for code that is hard to test, testing for code that does not conform to coding standards, and testing for ANSI violations) takes place before compilation. Static testing takes roughly as long as compilation and checks every statement you have written.

131. Why should I use static testing techniques?A: You should use static testing techniques because static testing is a bargain, compared to dynamic testing. Static testing is up to 100 times more effective. Even in selective testing, static testing may be up to 10 times more effective. The most pessimistic estimates suggest a factor of 4. Since static testing is faster and achieves 100% coverage, the unit cost of detecting these bugs by static testing is many times lower than that of by dynamic testing. About half of the bugs, detectable by dynamic testing, can be detected earlier by static testing. If you use neither static nor dynamic test tools, the static tools offer greater marginal benefits. If urgent deadlines loom on the horizon, the use of dynamic testing tools can be omitted, but tool-supported static testing should never be omitted.

132. What is the definition of top down design?A: Top down design progresses from simple design to detailed design. Top down design solves problems by breaking them down into smaller, easier to solve sub problems. Top down design creates solutions to these smaller problems, and then tests them using test drivers. In other words, top down design starts the design process with the main module or system, and then progresses down to lower level modules and subsystems. To put it differently, top down design looks at the whole


system, and then explodes it into subsystems, or smaller parts. A systems engineer or systems analyst determines what the top level objectives are, and how they can be met. He then divides the system into subsystems, i.e. breaks the whole system into logical, manageable-size modules, and deals with them individually.

133. How can I be effective and efficient, when I do black box testing of ecommerce web sites?A: When you're doing black box testing of e-commerce web sites, you're most efficient and effective when you're testing the sites' Visual Appeal, Contents, and Home Pages. When you want to be effective and efficient, you need to verify that the site is well planned. Verify that the site is customer-friendly. Verify that the choices of colors are attractive. Verify that the choices of fonts are attractive. Verify that the site's audio is customer friendly. Verify that the site's video is attractive. Verify that the choice of graphics is attractive. Verify that every page of the site is displayed properly on all the popular browsers. Verify the authenticity of facts. Ensure the site provides reliable and consistent information. Test the site for appearance. Test the site for grammatical and spelling errors. Test the site for visual appeal, choice of browsers, consistency of font size, download time, broken links, missing links, incorrect links, and browser compatibility. Test each toolbar, each menu item, every window, every field prompt, every pop-up text, and every error message. Test every page of the site for left and right justifications, every shortcut key, each control, each push button, every radio button, and each item on every drop-down menu. Test each list box, and each help menu item. Also check, if the command buttons are grayed out when they're not in use.

134. What is the difference between top down and bottom up design?A: Top down design proceeds from the abstract (entity) to get to the concrete (design). The Bottom up design proceeds from the concrete (design) to get to the abstract (entity). Top down design is most often used in designing brand new systems, while bottom up design is sometimes used when one is reverse engineering a design; i.e. when one is trying to figure out what somebody else designed in an existing system. Bottom up design begins the design with the lowest level modules or subsystems, and progresses upward to the main program, module, or subsystem. With bottom up design, a structure chart is necessary to determine the order of execution, and the development of drivers is necessary to complete the bottom up approach. Top down design, on the other hand, begins the design with the main or top level module, and progresses downward to the lowest level modules or subsystems. Real life sometimes is a combination of top


down design and bottom up design. For instance, data modeling sessions tend to be iterative, bouncing back and forth between top down and bottom up modes, as the need arises.

135. What is the definition of bottom up design?A: Bottom up design begins the design at the lowest level modules or subsystems, and progresses upward to the design of the main program, main module, or main subsystem. To determine the order of execution, a structure chart is needed, and, to complete the bottom up design, the development of drivers is needed. In software design - assuming that the data you start with is a pretty good model of what you're trying to do - bottom up design generally starts with the known data (e.g. customer lists, order forms), then the data is broken into chunks (i.e. entities) appropriate for planning a relational database. This process reveals what relationships the entities have, and what the entities' attributes are. In software design, bottom up design doesn't only mean writing the program in a different order, but there is more to it. When you design bottom up, you often end up with a different program. Instead of a single, monolithic program, you get a larger language, with more abstract operators, and a smaller program written in it. Once you abstract out the parts which are merely utilities, what is left is much shorter program. The higher you build up the language, the less distance you will have to travel down to it, from the top. Bottom up design makes it easy to reuse code blocks. For example, many of the utilities you write for one program are also useful for programs you have to write later. Bottom up design also makes programs easier to read.

136. What is smoke testing?A: Smoke testing is a relatively simple check to see whether the product "smokes" when it runs. Smoke testing is also known as ad hoc testing, i.e. testing without a formal test plan. With many projects, smoke testing is carried out in addition to formal testing. If smoke testing is carried out by a skilled tester, it can often find problems that are not caught during regular testing. Sometimes, if testing occurs very early or very late in the software development cycle, this can be the only kind of testing that can be performed. Smoke tests are, by definition, not exhaustive, but, over time, you can increase your coverage of smoke testing. A common practice at Microsoft, and some other software companies, is the daily build and smoke test process. This means, every file is compiled, linked, and combined into an executable file every single day, and then the software is smoke tested. Smoke testing minimizes integration risk, reduces the risk of low quality, supports easier defect diagnosis, and improves morale. Smoke testing does not have to be exhaustive, but should expose any major problems. Smoke testing


should be thorough enough that, if it passes, the tester can assume the product is stable enough to be tested more thoroughly. Without smoke testing, the daily build is just a time wasting exercise. Smoke testing is the sentry that guards against any errors in development and future problems during integration.At first, smoke testing might be the testing of something that is easy to test. Then as the system grows, smoke testing should expand and grow, from a few seconds to 30 minutes or more.

137. What is the difference between monkey testing and smoke testing? Difference#1: Monkey testing is random testing, and smoke testing is a

nonrandom check to see whether the product "smokes" when it runs. Smoke testing is nonrandom testing that deliberately exercises the entire system from end to end, with the goal of exposing any major problems.

Difference#2: Monkey testing is performed by automated testing tools. On the other hand, smoke testing, more often than not, is a manual check to see whether the product "smokes" when it runs.

Difference#3: Monkey testing is performed by "monkeys", while smoke testing is performed by skilled testers (to see whether the product "smokes" when it runs).

Difference#4: "Smart monkeys" are valuable for load and stress testing, but not very valuable for smoke testing, because they are too expensive for smoke testing.

Difference#5: "Dumb monkeys" are inexpensive to develop, are able to do some basic testing, but, if we use them for smoke testing, they find few bugs during smoke testing.

Difference#6: Monkey testing is not a thorough testing, but smoke testing is thorough enough that, if the build passes, one can assume that the program is stable enough to be tested more thoroughly.

Difference#7: Monkey testing does not evolve. Smoke testing, on the other hand, evolves as the system evolves from something simple to something more thorough.

Difference#8: Monkey testing takes "six monkeys" and a "million years" to run. Smoke testing, on the other hand, takes much less time to run, i.e. anywhere from a few seconds to a couple of hours.

138. Tell me about the process of daily builds and smoke testsA: The idea behind the process of daily builds and smoke tests is to build the product every day, and test it every day. The software development process at Microsoft and many other software companies requires daily builds and smoke tests. According to their process, every day, every single file has to be compiled,


linked, and combined into an executable program. And, then, the program has to be "smoke tested". Smoke testing is a relatively simple check to see whether the product "smokes" when it runs. You should add revisions to the build only when it makes sense to do so. You should to establish a Build Group, and build *daily*; set your *own standard* for what constitutes "breaking the build", and create a penalty for breaking the build, and check for broken builds *every day*. In addition to the daily builds, you should smoke test the builds, and smoke test them Daily. You should make the smoke test Evolve, as the system evolves. You should build and smoke test Daily, even when the project is under pressure. Think about the many benefits of this process! The process of daily builds and smoke tests minimizes the integration risk, reduces the risk of low quality, supports easier defect diagnosis, improves morale, enforces discipline, and keeps pressure-cooker projects on track. If you build and smoke test *daily*, success will come, even when you're working on large projects!

139. What is the purpose of test strategy?A:

Reason#1: The number one reason of writing a test strategy document is to "have" a signed, sealed, and delivered (approved document), where the document includes a written testing methodology, test plan, and test cases.

Reason#2: Having a test strategy does satisfy one important step in the software testing process.

Reason#3: The test strategy document tells us how the software product will be tested.

Reason#4: The creation of a test strategy document presents an opportunity to review the test plan with the project team.

Reason#5: The test strategy document describes the roles, responsibilities, and the resources required for the test and schedule constraints.

Reason#6: When we create a test strategy document, we have to put into writing any testing issues requiring resolution (and usually this means additional negotiation at the project management level).

Reason#7: The test strategy is decided first, before lower level decisions are made on the test plan, test design, and other testing issues.

140. What do you mean by 'the process is repeatable'?A: A process is repeatable, whenever we have the necessary processes in place, in order to repeat earlier successes on projects with similar applications. A process is repeatable, if we use detailed and well-written processes and


procedures. A process is repeatable, if we ensure that the correct steps are executed. When the correct steps are executed, we facilitate a successful completion of the task. Documentation is critical. A software process is repeatable, if there are requirements management, project planning, project tracking, subcontract management, QA, and configuration management. Both QA processes and practices should be documented, so that they are repeatable. Specifications, designs, business rules, inspection reports, configurations, code changes, test plans, test cases, bug reports, user manuals should all be documented, so that they are repeatable. Document files should be well organized. There should be a system for easily finding and obtaining documents, and determining what document has a particular piece of information. We should use documentation change management, if possible. 141. What is the purpose of a test plan?

A: Reason#1: We create a test plan because preparing it helps us to think through the efforts needed to validate the acceptability of a software product.

Reason#2: We create a test plan because it can and will help people outside the test group to understand the why and how of product validation.

Reason#3: We create a test plan because, in regulated environments, we have to have a written test plan.

Reason#4: We create a test plan because the general testing process includes the creation of a test plan.

Reason#5: We create a test plan because we want a document that describes the objectives, scope, approach and focus of the software testing effort.

Reason#6: We create a test plan because it includes test cases, conditions, the test environment, a list of related tasks, pass/fail criteria, and risk assessment.

Reason#7: We create test plan because one of the outputs for creating a test strategy is an approved and signed off test plan document.

Reason#8: We create a test plan because the software testing methodology a three step process and one of the steps is the creation of a test plan.

Reason#9: We create a test plan because we want an opportunity to review the test plan with the project team.

Reason#10: We create a test plan document because test plans should be documented, so that they are repeatable.


142. Give me one test case that catches all the bugs!A: If there is a "magic bullet", i.e. the one test case that has a good possibility to catch ALL the bugs, or at least the most important bugs, it is a challenge to find it, because test cases depend on requirements; requirements depend on what customers need; and customers can have great many different needs. As software systems are getting increasingly complex, it is increasingly more challenging to write test cases. It is true that there are ways to create "minimal test cases" which can greatly simplify the test steps to be executed. But, writing such test cases is time consuming, and project deadlines often prevent us from going that route. Often the lack of enough time for testing is the reason for bugs to occur in the field. However, even with ample time to catch the "most important bugs", bugs still surface with amazing spontaneity. The challenge is, developers do not seem to know how to avoid providing the many opportunities for bugs to hide, and testers do not seem to know where the bugs are hiding.

143. What is the difference between a test plan and a test scenario?A: Difference#1: A test plan is a document that describes the scope, approach, resources, and schedule of intended testing activities, while a test scenario is a document that describes both typical and atypical situations that may occur in the use of an application. Difference#2: Test plans define the scope, approach, resources, and schedule of the intended testing activities, while test procedures define test conditions, data to be used for testing, and expected results, including database updates, file outputs, and report results. Difference#3: A test plan is a description of the scope, approach, resources, and schedule of intended testing activities, while a test scenario is a description of test cases that ensure that a business process flow, applicable to the customer, is tested from end to end.

144. What is a test scenario?A: The terms "test scenario" and "test case" are often used synonymously. Test scenarios are test cases, or test scripts, and the sequence in which they are to be executed. Test scenarios are test cases that ensure that business process flows are tested from end to end. Test scenarios are independent tests, or a series of tests, that follow each other, where each of them dependent upon the output of the previous one. Test scenarios are prepared by reviewing functional requirements, and preparing logical groups of functions that can be further broken into test procedures. Test scenarios are designed to represent both typical and unusual situations that may occur in the application. Test engineers define unit test requirements and unit test scenarios. Test engineers also execute unit


test scenarios. It is the test team that, with assistance of developers and clients, develops test scenarios for integration and system testing. Test scenarios are executed through the use of test procedures or scripts. Test procedures or scripts define a series of steps necessary to perform one or more test scenarios. Test procedures or scripts may cover multiple test scenarios.

145. Give me some sample test cases you would write!A: For instance, if one of the requirements is, "Brake lights shall be on, when the brake pedal is depressed", then, based on this one simple requirement, for starters, I would write all of the following test cases: Test case#1: "Inputs: The headlights are on. The brake pedal is depressed. Expected result: The brake lights are on. Verify that the brake lights are on, when the brake pedal is depressed." Test case#2: "Inputs: The left turn lights are on. The brake pedal is depressed. Expected result: The brake lights are on. Verify that the brake lights are on, when the brake pedal is depressed."Test case number#3: "Inputs: The right turn lights are on. The brake pedal is depressed. Expected result: The brake lights are on. Verify that the brake lights are on, when the brake pedal is depressed."As you might have guessed, in the work place, in real life, requirements are more complex than this one; and, just to verify this one, simple requirement, there is a need for many more test cases.

146. How do you write test cases?A: When I write test cases, I concentrate on one requirement at a time. Then, based on that one requirement, I come up with several real life scenarios that are likely to occur in the use of the application by end users. When I write test cases, I describe the inputs, action, or event, and their expected results, in order to determine if a feature of an application is working correctly. To make the test case complete, I also add particulars e.g. test case identifiers, test case names, objectives, test conditions (or setups), input data requirements (or steps), and expected results. If I have a choice, I prefer writing test cases as early as possible in the development life cycle. Why? Because, as a side benefit of writing test cases, many times I am able to find problems in the requirements or design of an application. And, because the process of developing test cases makes me completely think through the operation of the application. You can learn to write test cases.

147. What is a parameter?


A: A parameter is an item of information - such as a name, a number, or a selected option - that is passed to a program by a user or another program. By definition, a parameter is a value on which something else depends. Any desired numerical value may be given as a parameter. We use parameters when we want to allow a specified range of variables. We use parameters when we want to differentiate behavior or pass input data to computer programs or their subprograms. Thus, when we are testing, the parameters of the test can be varied to produce different results, because parameters do affect the operation of the program receiving them. Example 1: We use a parameter, such as temperature that defines a system. In this definition, it is temperature that defines the system and determines its behavior. Example 2: In the definition of function f(x) = x + 10, x is a parameter. In this definition, x defines the f(x) function and determines its behavior. Thus, when we are testing, x can be varied to make f(x) produce different values, because the value of x does affect the value of f(x).When parameters are passed to a function subroutine, they are called arguments.

148. What is a constant?A: In software or software testing, a constant is a meaningful name that represents a number, or string, that does not change. Constants are variables whose value remains the same, i.e. constant, throughout the execution of a program. Why do developers use constants? Because if we have code that contains constant values that keep reappearing, or, if we have code that depends on certain numbers that are difficult to remember, we can improve both the readability and maintainability of our code, by using constants. To give you an example, let's suppose we declare a constant and we call it Pi. We set it to 3.14159265 and use it throughout our code. Constants, such as Pi, as the name implies, store values that remain constant throughout the execution of our program. Keep in mind that, unlike variables which can be read from and written to, constants are read-only variables. Although constants resemble variables, we cannot modify or assign new values to them, as we can to variables. But we can make constants public, or private. We can also specify what data type they are.

149. What is a requirements test matrix?A: The requirements test matrix is a project management tool for tracking and managing testing efforts, based on requirements, throughout the project's life cycle. The requirements test matrix is a table, where requirement descriptions are put in the rows of the table, and the descriptions of testing efforts are put in the column headers of the same table. The requirements test matrix is similar to the requirements traceability matrix, which is a representation of user requirements aligned against system functionality. The requirements traceability matrix


ensures that all user requirements are addressed by the system integration team and implemented in the system integration effort. The requirements test matrix is a representation of user requirements aligned against system testing. Similarly to the requirements traceability matrix, the requirements test matrix ensures that all user requirements are addressed by the system test team and implemented in the system testing effort.

150. What is reliability testing?A: Reliability testing is designing reliability test cases, using accelerated reliability techniques (e.g. step-stress, test/analyze/fix, and continuously increasing stress testing techniques), AND testing units or systems to failure, in order to obtain raw failure time data for product life analysis. The purpose of reliability testing is to determine product reliability, and to determine whether the software meets the customer's reliability requirements. In the system test phase, or after the software is fully developed, one reliability testing technique we use is a test/analyze/fix technique, where we couple reliability testing with the removal of faults. When we identify a failure, we send the software back to the developers, for repair. The developers build a new version of the software, and then we do test iteration. We track failure intensity (e.g. failures per transaction, or failures per hour) in order to guide our test process, and to determine the feasibility of the software release, and to determine whether the software meets the customer's reliability requirements.

151. Give me an example on reliability testing.A: For example, our products are defibrillators. From direct contact with customers during the requirements gathering phase, our sales team learns that a large hospital wants to purchase defibrillators with the assurance that 99 out of every 100 shocks will be delivered properly. In this example, the fact that our defibrillator is able to run for 250 hours without any failure, in order to demonstrate the reliability, is irrelevant to these customers. In order to test for reliability we need to translate terminology that is meaningful to the customers into equivalent delivery units, such as the number of shocks. We describe the customer needs in a quantifiable manner, using the customer’s terminology. For example, our goal of quantified reliability testing becomes as follows: Our defibrillator will be considered sufficiently reliable if 10 (or fewer) failures occur from 1,000 shocks. Then, for example, we use a test/analyze/fix technique, and couple reliability testing with the removal of errors. When we identify a failed delivery of a shock, we send the software back to the developers, for repair. The developers build a new version of the software, and then we deliver another 1,000 shocks into dummy resistor loads. We track failure intensity (i.e. number of


failures per 1,000 shocks) in order to guide our reliability testing, and to determine the feasibility of the software release, and to determine whether the software meets our customers' reliability requirements.

152. What is verification?A: Verification ensures the product is designed to deliver all functionality to the customer; it typically involves reviews and meetings to evaluate documents, plans, code, requirements and specifications; this can be done with checklists, issues lists, and walk-through and inspection meetings.

153. What is validation? A: Validation ensures that functionality, as defined in requirements, is the intended behavior of the product; validation typically involves actual testing and takes place after Verifications are completed.

154. What is a walk-through?A: A walk-through is an informal meeting for evaluation or informational purposes. A walk-through is also a process at an abstract level. It's the process of inspecting software code by following paths through the code (as determined by input conditions and choices made along the way). The purpose of code walk-through is to ensure the code fits the purpose. Walk-through also offers opportunities to assess an individual's or team's competency.

155. What is an inspection?A: An inspection is a formal meeting, more formalized than a walk-through and typically consists of 3-10 people including a moderator, reader (the author of whatever is being reviewed) and a recorder (to make notes in the document). The subject of the inspection is typically a document, such as a requirements document or a test plan. The purpose of an inspection is to find problems and see what is missing, not to fix anything. The result of the meeting should be documented in a written report. Attendees should prepare for this type of meeting by reading through the document, before the meeting starts; most problems are found during this preparation. Preparation for inspections is difficult, but is one of the most cost effective methods of ensuring quality, since bug prevention is more cost effective than bug detection.

156. What is quality?A: Quality software is software that is reasonably bug-free, delivered on time and within budget, meets requirements and expectations and is maintainable. However, quality is a subjective term. Quality depends on who the customer is


and their overall influence in the scheme of things. Customers of a software development project include end-users, customer acceptance test engineers, testers, customer contract officers, customer management, the development organization's management, test engineers, testers, salespeople, software engineers, stockholders and accountants. Each type of customer will have his or her own slant on quality. The accounting department might define quality in terms of profits, while an end-user might define quality as user friendly and bug free.

157. What is good code?A: A good code is code that works, is free of bugs and is readable and maintainable. Organizations usually have coding standards all developers should adhere to, but every programmer and software engineer has different ideas about what is best and what are too many or too few rules. We need to keep in mind that excessive use of rules can stifle both productivity and creativity. Peer reviews and code analysis tools can be used to check for problems and Enforce standards.

158. What is good design?A: Design could mean too many things, but often refers to functional design or internal design. Good functional design is indicated by software functionality can be traced back to customer and end-user requirements. Good internal design is indicated by software code whose overall structure is clear, understandable, easily modifiable and maintainable; is robust with sufficient error handling and status logging capability; and works correctly when implemented.

159. What is software life cycle?A: Software life cycle begins when a software product is first conceived and ends when it is no longer in use. It includes phases like initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting and phase-out.

160. What is the role of documentation in QA?A: Documentation plays a critical role in QA. QA practices should be documented, so that they are repeatable. Specifications, designs, business rules, inspection reports, configurations, code changes, test plans, test cases, bug reports, user manuals should all be documented. Ideally, there should be a system for easily finding and obtaining of documents and determining what document will have a particular piece of information. Use documentation change management, if possible.


161. Why are there so many software bugs?A: Generally speaking, there are bugs in software because of unclear requirements, software complexity, programming errors, changes in requirements, errors made in bug tracking, time pressure, poorly documented code and/or bugs in tools used in software development. There are unclear software requirements because there is miscommunication as to what the software should or shouldn't do. All of the followings contribute to the exponential growth in software and system complexity: Windows interfaces, client-server and distributed applications, data communications, enormous relational databases and the sheer size of applications. Programming errors occur because programmers and software engineers, like everyone else, can make mistakes. As to changing requirements, in some fast-changing business environments, continuously modified requirements are a fact of life. Sometimes customers do not understand the effects of changes, or understand them but request them anyway. And the changes require redesign of the software, rescheduling of resources and some of the work already completed have to be redone or discarded and hardware requirements can be effected, too. Bug tracking can result in errors because the complexity of keeping track of changes can result in errors, too. Time pressures can cause problems, because scheduling of software projects is not easy and it often requires a lot of guesswork and when deadlines loom and the crunch comes, mistakes will be made. Code documentation is tough to maintain and it is also tough to modify code that is poorly documented. The result is bugs. Sometimes there is no incentive for programmers and software engineers to document their code and write clearly documented, understandable code. Sometimes developers get kudos for quickly turning out code, or programmers and software engineers feel they cannot have job security if everyone can understand the code they write, or they believe if the code was hard to write, it should be hard to read. Software development tools, including visual tools, class libraries, compilers, scripting tools, can introduce their own bugs. There are other cases where the tools are poorly documented, which can create additional bugs.

162. Give me five common problems that occur during software development.A: Poorly written requirements, unrealistic schedules, inadequate testing, and adding new features after development is underway and poor communication. 1. Requirements are poorly written when requirements are unclear, incomplete, too general, or not testable; therefore there will be problems. 2. The schedule is unrealistic if too much work is crammed in too little time.


3. Software testing is inadequate if none knows whether or not the software is any good untilCustomers complain or the system crashes.4. It is extremely common that new features are added after development is underway.5. Miscommunication either means the developers don't know what is needed, or customers have unrealistic expectations and therefore problems are guaranteed.

163. What is a backward compatible design?A: The design is backward compatible, if the design continues to work with earlier versions of a language, program, code, or software. When the design is backward compatible, the signals or data that had to be changed, did not break the existing code. For instance, our mythical web designer decides that the fun of using Javascript and Flash is more important than backward compatible design, or, he decides that he doesn't have the resources to maintain multiple styles of backward compatible web design. This decision of his will inconvenience some users, because some of the earlier versions of Internet Explorer and Netscape will not display his web pages properly, as there are some serious improvements in the newer versions of Internet Explorer and Netscape that make the older versions of these browsers incompatible with, for example, DHTML. This is when we say, "This design doesn't continue to work with earlier versions of browser software. Therefore our mythical designer's web design is not backward compatible". On the other hand, if the same mythical web designer decides that backward compatibility is more important than fun, or, if he decides that he has the resources to maintain multiple styles of backward compatible code, then no user will be inconvenienced. No one will be inconvenienced, even when Microsoft and Netscape make some serious improvements in their web browsers. This is when we can say, "Our mythical web designer's design is backward compatible".


References

BooksROGER S. PRESSMAN: Software Engineering - A Practitioner’s approach, McGrawHill, 2001ILENE BURNSTEIN: Practical Software Testing, Springer, 2002

WILLIAM E. PERRY: Effective Methods for software testing, Willey 2000

C.R.PANDIAN: Software Metrics, Arunodaya Printers, 2003

Website Links http:// www. stickyminds .com

http://en.wikipedia.org/wiki/Software_testing

http://www.reference.com/browse/wiki/Software_testing

http://www.softwareqatest.com/

http://www.testingstuff.com/autotest.html


http://www.testingstuff.com/autotest.html

http://www.softwareqatest.com/

http://www.reference.com/browse/wiki/Software_testing

http://en.wikipedia.org/wiki/Software_testing

http://www.stickyminds.com/

The Software Development Life Cycle (SDLC) - Zeta Web viewThis section of the document will describe...

Documents

Transcript of The Software Development Life Cycle (SDLC) - Zeta Web viewThis section of the document will describe...