CSTE Skill Category 1 2014V03 - · PDF fileEssay (10 minutes): CSTE -Skill Category 1 -12...

CSTE - Skill Category 1 - 1 © Copyright 2014 / All rights reserved

Skill Category 1Software Testing Principles and Concepts

The following topics will be discussed in this Skill Category:

• Vocabulary & Quality Basics

• Understanding Defects

• Process and Testing Published Standards

• Software Testing

• Software Development Life Cycle (SDLC) Models

• Agile Development Methodologies

• Testing Throughout the SDLC

• Testing Schools of Thought and Approaches

• Test Categories and Testing Techniques


Testing VocabularyUnderstand the technical terms used to describe various testing techniques, tools, principles, concepts and activities

contingency


Quality 101


Quality Assurance• Activities that modify the development process to prevent the

introduction of flaws

– Staff function

– Implements management’s quality policies

– Responsible for continuous improvement of the software development process

• Proactive approach focused on defect prevention

• Examples:

– Defining change control procedures

– Analyzing metrics to identify opportunities for process improvement

– Facilitating quality improvement activities


Quality Control• Activities within the development process to detect the introduction of

flaws

– Test planning and execution

– Quality control measures a product against the existence of an attribute

– Determines whether the product conforms to a standard or procedure (also known as compliance checking).

• Proactive approach focused on defect detection

• Examples:

– Writing and executing test cases and scripts

– Participating in verification and validation activities

– Reporting defects to identify opportunities for process improvement

– Conducting post-project reviews


Exercise

Inspection of source code.Unit testing to validate that the program works.Analysis of defects to determine the stage of origin.Analyze metrics collected that measure the effectiveness of system and unit Testing

QA

QC

Quality Assurance or Quality Control?


ExerciseThe responsibilities of this job include facilitation, process

configuration, measurement, and risk analysis.a) Quality Controlb) Quality Assurance

The responsibilities of this job include conducting inspections, reviews, testing and focusing on the product.

a) Quality Controlb) Quality Assurance


Your department has always been titled the SQA (Software Quality Assurance) Group. The primary activity of your group is software testing with some minor QA responsibility. Your company is currently reorganizing and you have decided it is a good time to clearly delineate the QC and QA functions. Describe the activities of these two unique groups.

Essay (10 minutes):


Questions?


The Perspectives of QualityThere are five perspectives of Quality.

• Transcendent – I know it when I see it

• Product Based – Possesses desired features

• User Based – Fitness for use

• Development and Manufacturing Based – Conforms to requirements

• Value Based – At an acceptable cost


How Quality is DefinedProducer’s View of Quality Customer’s View of Quality

QUALITY IN FACT QUALITY IN PERCEPTION

Doing the right thing. Delivering the right product.

Doing it the right way. Satisfying our customer’s needs.

Doing it right the first time. Meeting the customer’s expectations.

Doing it on time. Treating every customer with integrity, courtesy, and respect.

Quality in Perception is more important to the final customer.


The Two Quality Gaps

Two Working Definitions:

• Producer’s ViewMeets requirements

• Consumer’s View Fit for use Start

ProducerGap

CustomerGap

Quality as delivered

An organization’s quality policy must define and view quality from their customer's perspectives. If there are conflicts, they must be resolved.


Quality Message• Focus on the Customer

• Customer Satisfaction is the essence of a Quality Product

• Customer may be internal or external

• Internal customer receives anything passed between groups

• External customer uses the products or services provided by the organization


Improving Software QualityThere are many ways to improve Software Quality

• Software Quality Objectives

• Software Quality Assurance Activities

• Testing Strategy

• Software Engineering Guidelines

• Formal and Informal Technical Reviews

• Audits

• Process Improvement

• Change Control Processes

• Measurement, Prototyping, and Proof


The Cost of Quality• Money spent beyond the cost to build the product

right the first time

• Frequently referred to today as Cost of Non Quality or Cost of Poor Performance


Cost of Quality• Preventive Costs - Costs associated with preventing errors

– Training– Establishing methods and procedures– Tool acquisition

• Appraisal Costs - Costs associated with the detection of errors– Inspections– Testing

• Failure Costs - Costs associated with defective products delivered to customers

– Analyze, correct and retest defects– Staffing Help Desk– Damage caused by defect– Damage caused by defect– Idle users


Mark “P” for Prevention; “A” for Appraisal; “F” for Failure:

1) Conducting Reviews and Inspections2) Implementing Standards3) Conducting Rework4) Attending Training/Education5) Planning for Quality6) Code Inspection7) Cancelled Projects8) Failure & Recovery of Data9) Defect Reporting Standards10) Quality Improvement11) Testing12) Conducting a Post Project Review13) Lost Benefits

Cost of Quality

AP

FP

PA

FFP

PA

AF


ExerciseWhich of the following is NOT a category of the Cost of Quality?

a) Failure Costb) Appraisal Costc) Build Costd) Preventive Cost

In defining the cost of quality, appraisal costs are BEST described as:a) Costs incurred to review completed products against

requirementsb) Costs which can not be recoupedc) All costs associated with defective productsd) None of the above


Cost of Quality – Iceberg Diagram

• Cost of Quality will vary from Organisation to Organisation


Question: Is Quality Free?


Cost of Quality - ROI


Essay (5 minutes):

The “cost of quality” (COQ) isn’t the price of creating a quality product or service. It’s the cost of NOT creating a quality product or service. Explain what methods you would use to reduce the Cost of Quality as a function of optimizing the development and testing process.


Questions?


Software Quality Factors

Correctness ReliabilityEfficiency Integrity

Usability

MaintainabilityFlexibilityTestability

PortabilityReusability

Interoperability




Usability



InteroperabilityA quality factor represents the behavioral characteristic of a system.

Examples: correctness, reliability, efficiency, testability, portability, …




Usability



Interoperability

Quality Categories Quality Factors Broad Objectives

Product Operation

CorrectnessReliabilityEfficiencyIntegrityUsability

Does it do what the customer wants?Does it do it accurately all of the time?Does it quickly solve the intended problem?Is it secure?Can I run it?

Product Revision

MaintainabilityTestabilityFlexibility

Can it be fixed?Can it be tested?Can it be changed?

ProductTransition

PortabilityReusabilityInteroperability

Can it be used on another machine?Can parts of it be reused?Can it interface with another system?



• Attributes of software

• Needed for trade off decisions

• Should be included in software requirements

• Should be included in the test plan

• Lack of the needed factors cause customer dissatisfaction


Software Quality FactorsFactors DefinitionCorrectness Extent to which a program satisfies its specifications and fulfills the user’s mission

objective.Reliability Extent to which a program can be expected to performs its intended function with

required precision.Efficiency The amount of computing resources and code required by a program to perform a

function.Integrity Extent to which access to software or data by unauthorized persons can be

controlled.Usability Effort required learning, operating, preparing input, and interpreting output of a

program.Maintainability Effort required locating and fixing an error in an operational program.

Testability Effort required testing a program to ensure that it performs its intended function.

Flexibility Effort required to modify an operational program.

Portability Effort required to transfer software from one configuration to another.

Reusability Extent to which a program can be used in other applications - related to the packaging and scope of the functions that programs perform.

Interoperability Effort required to couple on system with another.


Software Quality Criteria1. Access audit: Ease with which software and data can be checked for

compliance with standards or other requirements.2. Access control: Provisions for control and protection of the software and data.3. Accuracy: Precision of computations and output.4. Communication commonality: Degree to which standard protocols and

interfaces are used.5. Completeness: Degree to which a full implementation of the required

functionalities has been achieved.6. Communicativeness: Ease with which inputs and outputs can be assimilated7. Conciseness: Compactness of the source code, in terms of lines of code. 8. Consistency: Use of uniform design and implementation techniques and

notation throughout a project.9. Data commonality: Use of standard data representations.10.Error tolerance: Degree to which continuity of operation is ensured under

adverse conditions. 11.Execution efficiency: Run time efficiency of the software.12.Expandability: Degree to which storage requirements or software functions

can be expanded.


Software Quality Criteria13.Generality: Breadth of the potential application of software components.14.Hardware independence: Degree to which the software is dependent on the

underlying hardware.15. Instrumentation: Degree to which the software provides for measurement of

its use or identification of errors.16.Modularity: Provision of highly independent modules.17.Operability: Ease of operation of the software.18.Self-documentation: Provision of in-line documentation that explains

implementation of components.19.Simplicity: Ease with which the software can be understood.20.Software system independence: Degree to which the software is independent

of its software environment—nonstandard language constructs, operating system, libraries, database management system, etc.

21.Software efficiency: Run time storage requirements of the software.22.Traceability: Ability to link software components to requirements.23.Training: Ease with which new users can use the system.


Software Quality Criteria

A quality criterion is an attribute of a quality factor that is related to software development.

Example: •Modularity is an attribute of the architecture of a software system.•A highly modular software allows designers to put cohesive components in one module, thereby increasing the maintainability of the system.…


Map Software Quality Factors & Quality Criteria


Map Software Quality Factors & Quality Criteria

Each quality factor is positively influenced by a set of quality criteria, and the same quality criterion impacts a number of quality factors.

Example: Simplicity impacts reliability, usability, and testability.

If an effort is made to improve one quality factor, another quality factor may be degraded.

Portable code may be less efficient.

Some quality factors positively impact others. An effort to improve the correctness of a system will increase its reliability.


ISO 25010:2011ISO 25010:2011 Standard defines two models relating to software Product Quality

• Product Model applicable to both Computer Systems and Software Products with 8 characteristics and multiple sub-characteristics

• Quality in Use model applicable to the complete human-computer system with 5 characteristics and multiple sub-characteristics


You are on the product team which is initiating the development of an inventory control system. The system will maintain inventory status and facilitate requisitioning, reordering, and issuing of supplies. The planned life of the system is ten years.

Identify 3 functions or system characteristics of the software and provide at least one Software Quality Factor related to that function or system characteristic.

For this exercise, work in small groups and be prepared to present your essay answer.

Essay (5 minutes):


What is a Defect

• Process Defects

• Product Defects

Two Types of Defects

A defect is defined as the lack of a desirable state.


What is a Process?

A process is a set of activities that represent the way work is performed. This work effort includes efforts of people and equipment guided by policies, standards, and procedures. The outcome from a process is usually a product or service.


What is a Process Defect?

A process is considered defective when the output of that process is outside the accepted control limits.


Process Control

Out of control processes


Common Causes of Variation

• All processes contain some inherent variation, or common causes of variation. The amount of variation in a process is quantified with summary statistics.

• In a computer operation, abnormal terminations cause variation. Typical common causes of abnormal terminations included invalid data, no disk space, errors in operating or job control instructions, etc.


Common Causes of Variation


Special Causes of Variation

• Special causes of variation are not present in a process

• They occur because of special or unique circumstances

• In the IT example of abnormal terminations in a computer operation, special causes might include:

– citywide power outages

– earthquakes or hurricanes


Special Causes of Variation


How Are Processes Brought Under Control?

• Understand process maturity

• Continued process improvement

• Testers should be a contributor

Do Testers Need to KnowStatistical Process Control (SPC)?

• The concept of measuring and reducing variability is commonly called statistical process control (SPC).

• Testers need to understand process variability, because the more variance in the process the greater the need for software testing.


List three things that could cause common cause of variation within your test processes and three things that could be special causes of variation impacting your test processes.

For this exercise, work in your small groups and be prepared to present your essay answer.

Essay (5 minutes):


• Producer view

–A deviation from specification: missing, wrong, or extra

• Customer view

–Anything that causes customer dissatisfaction, whether in the specifications or not

What is a Product Defect?


Why Are Defects Hard to Find?

• Not looking Tests often are not performed becausea particular test condition was unknown

• Looking but not seeingLike losing your keys onlyto discover they were in plain sight the entire time

• The size and complexity of applications often makes it impossible to test everything


Test Your SensitivityFirst read the sentence enclosed in the box that next appears

Then, count the F’s in the sentence.Count them only once.

Do not go back and count them again!

FINISHED FILES ARE THE RESULT OF YEARS OF SCIENTIFIC STUDY

COMBINED WITH THE EXPERIENCE OF MANY YEARS.


What Were Your Results?• Number of F’s __________

• Number of F’s __________

• Number of F’s __________

• Number of F’s __________

• Number of F’s __________

• Number of F’s __________

• Number of F’s __________

• Number of F’s __________


Defects Typically Found in Software Systems AreThe Results of:

• IT improperly interprets requirements• Users specify the wrong requirements• Requirements are incorrectly recorded• Design specifications are incorrect• Program specifications are incorrect• Errors in program coding• Data entry errors• Testing errors• Mistakes in error correction• The corrected condition causes another defect


Questions?


Process and Testing StandardsSome development processes are more effective than others. The need for standards within the Software Engineering discipline were noted. Three are listed.

• CMMI-Dev – A process improvement model for software development

• TMMI – A process improvement model for software testing

• ISO29119 – A set of standards for software testing


SEI’s Five Levels of Process Maturity



Culture 1: Manage People• Ad hock and chaotic

processes• Success is not repeatable• Political environment• If you are breathing, you

are at level 1!



Culture 2: Manage Processes• Managed processes• Results predefined• Skills taught



Culture 3: Defined• Processes rigorously defined• Tailored processes• Proactive process management



Culture 4: Quantitatively Managed• Customer based objectives• Performance is predictable• Statistical analysis used



Culture 5: Optimization• Managed innovation• Continuous improvement• Process variations addressed


TMMI’s Five Levels of Test Maturity

http://www.tmmi.org/?q=node/64


You are currently operating at CMMI level 2 in your software testing department. Describe what it means to be operating at a level 2 maturity. Be specific and provide examples.

Essay (5 minutes):


The maturity of the productequals

the maturity of the process


ISO29119 – A set of process standards for software testing

There are 5 standards:

• ISO 29119-1: Concepts and Definitions

• ISO 29119-2: Test Processes

• ISO 29119-3: Test Documentation

• ISO 29119-4: Test Techniques

• ISO 29119-5: Keyword Driven Testing


Software TestingThe process of evaluating a deliverable with the intent of finding errors.

• Testing is not a stage/phase of the project • Testing is not just finding broken code• Testing is not a final exam • Testing is not “debugging”

What it’s Not!


General Testing Guidelines• Testing shows presence of defects

• Exhaustive testing is impossible

• Early testing

• Defect clustering

• Pesticide paradox

• Testing is context dependent

• Absence-of-errors fallacy

• Testing must be traceable to the requirements

• Defect data should be used to focus future efforts

• Testing should be done incrementally

• Focus on exceptions


Why Do We Test Software• To find defects• To reduce risk inherent in computer systems• To satisfy the customer• To prove that a program is good or no good• Developers are unable to build defect-free software• The user / customer does not know what they want• The development process is defective• To detect variations from specification/expectation• Establishing confidence that a program does what it is

supposed to do


Factors Affecting Software Testing

• People relationships

• Scope of testing ( producer vs. customer)

• Understanding life cycle testing

• Plan your work and work your plan

• Testing constraints


Factors Affecting Software Testing: People Relationships

Some attitudes causing a negative view of testing

• Testers hold up implementation, FALSE

• Giving testers less time to test will reduce the chance that they will find defects, FALSE

• Letting the testers find problems is an appropriate way to debug, FALSE

• Defects found in production are the fault of the testers, FALSE; and

• Testers do not need training; only programmers need training, FALSE!


Factors Affecting Software Testing: The Scope of Software Testing

• Software testing can help ferret out the true needs of the customer.

• Finding defects early in the software development process.

• Removing defects of all types prior to software going into production.

• Identifying weaknesses in the software development process.


Value of Life Cycle Testing

The Traditional view was that testing was a phase at the end of the project. This increased the cost because:

• errors found late in the product may cause severe consequences

• errors usually occur early in the development cycle.


Factors Affecting Software Testing: Testing Constraints

Movement of one of the testing constraints will

cause one or more of the other constraints to move

Scope Schedule

QualityResources Technology


Testing Cost Curve

After the optimum test point the cost of testing to uncover defects exceeds the losses from those defects.


Factors Affecting Software Testing: Misunderstanding Life Cycle Testing

Life Cycle Phase• Requirements• Design• Program (build/construction)• Test• Installation• Maintenance


Independent Testing• Ideally independent testers will report to the project manager

and have a reporting structure independent from the group designing or developing the application in order to assure that the quality of the application is given as much consideration as the project budget and timeline.

• The test manager’s responsibilities include:– Test planning and estimation– Designing the test strategy– Reviewing analysis and design artifacts– Chairing the Test Readiness Review– Managing the test effort– Overseeing acceptance tests


Independent Testing

• Testers are usually responsible for:– Developing test cases and procedures– Test data planning, capture, and conditioning– Reviewing analysis and design artifacts– Testing execution– Utilizing automated test tools for regression testing – Preparing test documentation– Defect tracking and reporting


Software Development Life Cycle Models

There are many Software Development Life Cycle Models• Ad-Hoc• Waterfall• V-Model• Incremental Mode• Iterative Development Model• Prototype/RAD Model• Spiral Model• Reuse Model



All the models embrace the following in some format:• System Conceptualization• System Design• Unit Development• Software Integration and Testing• Site installation• Training• Implementation• Maintenance



Ad-hoc• System Capability is unpredictable• Performance depends on individuals• Repeatability depends on individuals



WaterfallOne of the original models• System Conceptualization is completed• System Design is completed• Coding is completed• Testing is completedProblems• Real projects are uncertain and rarely follow the

sequential model• Process can be long



Waterfall



V-Model• Extension of Waterfall• Shows the relationship between specification

development and the associated dynamic testing phase• Shows inverse relationship high level specifications

moving to details and testing more details to high level


A Life Cycle Quality Approach

Testing throughout development life-cycle

Early development of test requirements

Early detection of errors

At pre-determined points

Involves static and/or dynamic test techniques

The V-Model focuses on:


The “V” Testing ConceptOperational orBusinessNeed

VerifyOperational

or Business Need

DefineRequirements

VerifyRequirements

DesignSystem

VerifyDesign

BuildSystem

VerifyConstruction

AcceptanceTest

ValidateOperational

or Business Need

SystemTest

ValidateRequirements

IntegrationTest

ValidateDesign

UnitTest

ValidateConstruction

Static Dynamic

Validates

Validates

Validates

Validates


• User Acceptance Test

• System Test

• Integration Test

• Unit Test

Basic Test Stages

Construction &Unit Test

IntegrationTest

SystemTest

UserAcceptance Test



IncrementalSuperset of Waterfall• Break Requirements into smaller buildable projects• Use full Lifecycle model for each smaller subset• Each successive increment adds functionality until the

final product is produced• Variants include Agile, Spiral, and RAD



Incremental



Iterative• Break Project into small parts• Get user feedback• Some variations allow immediate implementation into

productionProblems• Extensive user involvement• Communications take center stage• Requests for changes may cause confusion



Prototyping

• Creation of the major user interfaces without any substantive coding in the background in order to give the users a “feel” for what the system will look like

• Development of an abbreviated version of the system that performs a limited subset of functions; development of a paper system (depicting proposed screens, reports, relationships etc.)

• Use of an existing system or system components to demonstrate some functions that will be included in the developed system



Rapid Application Development (RAD)

Variation of Prototyping

• Less emphasis on detailed requirements

• User interacts with development team

• Four phases:Requirements Planning PhaseUser Design PhaseConstruction PhaseCutover Phase



Rapid Application Development (RAD)



Spiral Model

Waterfall plus Prototyping

• Initial version developed and then repeatedly modified

• Each version designed based on Waterfall method

• Spiral out from the centre



Spiral



Reuse Model

Build using Existing Components

• Library of Procedural Modules

• Library of Database Modules

• Borrow and use in the new function



Agile

Approaches which include the following are collectively known as Agile

• Scrum

• Crystal Clear

• Adaptive Software Development

• Feature Driven Development

• Dynamic Systems Development Methodology (DSDM)

• Extreme Programming



Agile Practices

• Detailed Planning for each iteration

• Test Driven Development

• Refactor Relentlessly

• Continuous Integration

• Paired Programming

• Onsite Customer



Creating and Combining Models

Models should be combined or modified based the relationship between the Information System and its organizational environment. Environments can be classified

• Unchanging

• Turbulent

• Uncertain

• Adaptive


Questions?


You feel that a Software Development Life Cycle testing model should be implemented in your testing approach. What justification would you provide to your manager to sell a defined life cycle approach?

Essay (5 minutes)


Testing Throughout the Lifecycle

• Continuous testing throughout the development process

• Must have formalized lifecycle

• Pre-determined deliverables


There are a number of constraints that must be recognized and planned for accordingly. An example of a constraint might be budgetary constraints. List 3 additional constraints of which a test manager might need to be aware and describe how trade-off decisions might be necessary regarding these three constraints.

Essay (10 minutes)


Static vs Dynamic TestingVerification – Static Test

• Walkthroughs

• Reviews

• Inspections

Validation – Dynamic Test• Test cases

• Scripting

• Feasibility reviews• Requirements reviews• Design reviews• Code walkthroughs• Code Inspections• Requirements tracing

• White Box

• Black Box


Verification and Validation Techniques

• Techniques for quantifiably assuring that a work product meets its stated objectives:

– Verification: Performed during development on key artifacts

• Walkthroughs, reviews and inspections

• Mentor feedback, training, checklists and standards

– Validation: Performed after a work product is produced

• Validating against established criteria

• Ensuring product integrates correctly into the environment

• Reactive approach focused on defect detection and removal


Verification and Validation Techniques

• Techniques for quantifiably assuring that a work product meets its stated objectives:

– Verification: Performed during development on key artifacts

• Walkthroughs, reviews and inspections

• Mentor feedback, training, checklists and standards

– Validation: Performed after a work product is produced

• Validating against established criteria

• Ensuring product integrates correctly into the environment

• Reactive approach focused on defect detection and removal

EXAM HINTThere is some variation in the industry regarding these definitions. Know them as stated for exam!


Walkthroughs, Reviews, Inspections


• Walkthroughs and inspections are very disciplined procedures aimed at removing the major responsibility for verification from the developer.

• The main goal of a review is to identify defects within the stage or phase of the project where they originate, rather than in later test stages; this is referred to as “stage containment.”


Walkthroughs, Reviews, Inspections


• Walkthroughs and inspections are very disciplined procedures aimed at removing the major responsibility for verification from the developer.

• The main goal of a review is to identify defects within the stage or phase of the project where they originate, rather than in later test stages; this is referred to as “stage containment.”

Described in detail in STBOK Section 6


Unit Testing Testing individual programs or components to

validate that the logic works according to specification

Validates technical quality of the code

Conducted by the developer who created the component

Begins once component development is complete


Integration Testing Tests the integration of components that have been

successfully unit-tested

Validates the technical quality of the design

Validates the integration of the application and the environment

Conducted by development with support from the test team

Begins once the first components have passed unit test


System Testing Tests the entire assembled system

Validates delivery of functional and non-functional requirements

Validates interface to upstream and downstream applications

Conducted by independent test team

Begins when integration testing has been successfully completed


User Acceptance Test

Validates system is “fit for use”

Evaluates how system integrates with manual business processes

Conducted by end users, customers or designated representatives

Begins when system test has been successfully completed


Essay (6 minutes)In software testing there are at least four stages of testing. Listthe sequence in which those four stages of testing should occurfrom one, (which is the first), to four, (which is the last). Brieflyexplain the objective of each of these four stages of testing.


ExerciseUnit and Integration testing primarily focuses on which of the following?

a) Verification

b) Validation

c) Test planning

d) Both a & c

Tests that validate system requirements are often called.

a) Functional Tests

b) Structural Tests


ExerciseUnit and Integration testing primarily focuses on which of the following?

a) Verification

b) Validation

c) Test planning

d) Both a & c

Tests that validate system requirements are often called.

a) Functional Tests

b) Structural Tests

EXAM HINTFor multiple choice parts, read each question’s stem twice, then read ALL the responses


Traceability Matrix

• Translation from Stage to Stage

• Requirements to be traced throughout the Lifecycle

Requirement ID Func Rqmt 1.1

FuncRqmt1.2

Func Rqmt 1.3

Func Rqmt 1.x

Func Rqmt

x.x

Technical Rqmt 1.1

Technical Rqmt 1.2

Technical Rqmt 1.x

TestCases 3 2 3 1 2 1 1 1

1.1.1 x

1.1.2 x x


Testing Schools of Thought

A ‘School of Thought’ is a system of beliefs shared by a group. Some of the Schools of thought are:

• Analytical School – testing is rigorous and technical

• Factory School – reduction of testing tasks to basic routines or repetitive tasks

• Quality (Control) School – process and standards

• Context-driven School –focus on product and people

• Agile School – continuous delivery of incremental products


Testing Approaches

The approaches do not relate to a model or school:

• Requirements-based Testing – quality of Requirements

• Risk-based Testing – list, prioritize, and test based on the assessed risk

• Model-based Testing – create a (simplified) model and build testcases based on the model

• Exploratory Testing - used by professional testers based on their knowledge of the system

• Keyword-driven Testing - define keywords or actions for each function to drive the testing


Test Categories and Testing Techniques

• Structural

• Functional

• Non-Functional

Will be discussed on the following slides.


Test Categories• Structural Tests

Tests that validate system architecture is structural testing. Also considered white box testing because knowledge of the internal logic of the system is used to develop test cases.

• Functional Tests

Tests that validate system are called functional testing. This testing addresses the overall behavior of the program by testing transacting flows, input validation, and functional completeness but no knowledge of the internal logic system is used (black box).

• Non-Functional Tests (Software Quality Factors)

Tests that validate system characteristics, such as performance, stability, maintainability, usability, and security. Can be considered a user’s point of view.


Structural Test TechniquesStructural system testing is designed to verify that the developed system and programs work.


• Structural test technique• Testing based on knowledge of internal code structure and

logic - usually logic driven

White-Box Testing

Internal View

What is the system doing?How is it doing it?

Input

Output


White-Box Techniques• Statement coverage

• Decision Coverage

• Condition Coverage

• Decision/Condition Coverage

• Multiple Condition Coverage


Functional Test TechniquesFunctional system testing ensures that the system requirements and specifications are achieved.

Technique Description ExampleRequirements System performs as specified. * Prove system requirements

* Compliance to policies, regulations

Error Handling Errors can be prevented or detected and then corrected.

* Error introduced into test* Errors re-entered

Intersystem Data is correctly passed from system to system.

* Intersystem parameters changed* Intersystem documentation updated

Control Controls reduce system risk to an acceptable level.

* File reconciliation procedures work* Manual controls in place

Parallel Old system and new system are run and the results compared to detect unplanned differences.

* Old and new systems


Black-Box Testing• Functional test technique

• Testing based on external specifications without knowledge of how the system is constructed - usually data or business process driven

• Validates that each input produces the appropriate output

External View

What did the application do?What should it have done?

Input

Output


Black-Box TechniquesEquivalence partitioning

Test cases generated using a subset of data to represent a larger class of data

Example:

A business rule edits credit limits within a given range ($10,000 - $15,000) could have three equivalence classes:

Less than $10,000 (Invalid)Between $10,000 and $15,000 (Valid)Greater than $15,000 (Invalid)


Black-Box Techniques (continued)

Boundary analysis

Test cases designed using data from the limits of the input and output domain classes

Example (continued)

Testing the boundaries of a business rule that edits credit limits within a given range ($10,000 - $15,000) would test:

Low boundary +/- one ($9,999 and $10,001)On the boundary +/- one ($10,000 and $15,000)Upper boundary +/- one ($14,999 and $15,001)



Other types include:

• Decision Table Testing

• State Transition Testing

• All-pairs (pairwise) Testing

• Cause-Effect Graphing



Ad Hoc or Error Guessing

Test cases and data designed based upon experience and intuition of the tester

Example

Testing the date-driven activities performed on credit reports, the tester might test the dates below based upon experience that year-end, month-end, and other dates usually cause problems:

2/29/2006


Non-Functional Test Types

• Accessibility Testing

• Conversion Testing

• Maintainability Testing

• Reliability Testing

• Stability Testing

• Usability Testing


Incremental Testing• Top-Down - testing begins with module

highest in the hierarchy using stubs to simulate lower interfacing modules; modules are added in descending hierarchical order

• Bottom-Up - testing begins with module at bottom of hierarchy using drivers or test harnesses to simulate higher interfacing modules; modules are added in ascending hierarchical order


Thread Testing• Often used during the integration of units or components

• Demonstrates key functional capabilities by testing a string of units or components that accomplish a specific function in the application

• Can be performed simultaneously with incremental testing


Regression Testing

Regression testing is a decision to re-test something that has already been tested looking for inadvertently introduced defects. Regression Testing should be performed:

• New releases of packaged software

• Application is enhanced or changes are made

• Support software changes

• Either side of a system interface is changed

• Changes to configuration

• Whenever changes are made after a testing stage is completed


Exercise

A. InspectionsB. Black BoxC. Capture/PlaybackD. Stress TestingE. Regression Testing

1. Testing previously verified logic

2. Testing Branches/paths

3. Re-use of test data

4. Test correct implementation of specifications

5. Testing processing limits

Match the items to the numbered list

1

2

34

5


Questions?


Skill Category 1 - SummarySoftware Testing Principles and Concepts

The following topics were discussed in this Skill Category:

• Vocabulary & Quality Basics

• Understanding Defects

• Process and Testing Published Standards

• Software Testing

• Software Development Life Cycle (SDLC) Models

• Agile Development Methodologies

• Testing Throughout the SDLC

• Testing Schools of Thought and Approaches

• Test Categories and Testing Techniques

CSTE Skill Category 1 2014V03 - · PDF fileEssay (10 minutes): CSTE -Skill Category 1 -12...

Documents

Transcript of CSTE Skill Category 1 2014V03 - · PDF fileEssay (10 minutes): CSTE -Skill Category 1 -12...