SSD9: Software Specification, Testing, and SSD9: Software Specification, Testing, and MaintenanceMaintenance

Unit 6. Build and Test the Solution Unit 6. Build and Test the Solution

6.16.1 Implementation Implementation 6.2 Module Testing 6.2 Module Testing 6.3 Integration Testing 6.3 Integration Testing 6.4 Product and Acceptance Testing 6.4 Product and Acceptance Testing 6.5 CASE Technology 6.5 CASE Technology

Assessments Assessments Exercise 7 Multiple-Choice Quiz 6

66.1 Implementation.1 Implementation

6.1.1 Reuse 6.1.1 Reuse 6.1.2 Choice of Programming Language 6.1.2 Choice of Programming Language 6.1.3 Good Programming Practices and Coding 6.1.3 Good Programming Practices and Coding

Standards Standards

6.1.1  Reuse 6.1.1  Reuse

Module Reuse Module Reuse Costs and Constraints in Reuse Costs and Constraints in Reuse What Can Be Reused and When? What Can Be Reused and When? Reuse and Maintenance Reuse and Maintenance

Module ReuseModule Reuse

The first issue one faces in implementation is The first issue one faces in implementation is whether to code everything from scratch 只靠自己的双手

从头做起？ whether to reuse existing components.

Although components from Although components from all phasesall phases of the of the software process can be reusedsoftware process can be reused requirements, specifications, design, documentation, and testing suites,

module reusemodule reuse is by far is by far the most common type of the most common type of reusereuse..

Reuse Reuse

ReuseReuse refers to taking the components of one product refers to taking the components of one product to to facilitate the development of a different productfacilitate the development of a different product with a with a different functionality. different functionality.

Reuse can be Reuse can be accidentalaccidental 机会重用机会重用 if developers realize that a previously developed component can be

reused reuse can be reuse can be deliberatedeliberate. . 计划重用计划重用

components are constructed with the idea of possible future reutilization in mind.

Deliberate reuse is better, insofar as such components are likely to be better documented, better tested, and designed in a uniform style that makes for easier maintenance.

However, even However, even reused componentsreused components will have to will have to be retestedbe retested during integration testing, because during integration testing, because 重用组件要重新测试！重用组件要重新测试！ the range of input they get in a new product might go beyond that of the

previous product or products.

Code reuseCode reuse

In the early days of system building, there was In the early days of system building, there was no no reusereuse: everything was constructed from scratch. : everything was constructed from scratch.

The first common source of code reuseThe first common source of code reuse was was subroutine or class librariessubroutine or class libraries, of which there are now , of which there are now many in the public domain for different languages. many in the public domain for different languages. many scientific computing subroutines have been developed

and refined for optimal performance, so in the case of each of these subroutines there is no need to recode one.

Application Programming Interfaces (APIs) are usually implemented as operating system calls, but they can be viewed as subroutine libraries.

Code reuse is also a feature of the many varieties of toolkits for building graphical user interfaces (GUIs).

Costs and Constraints in ReuseCosts and Constraints in Reuse

In reality, only a relatively small percentage of any In reality, only a relatively small percentage of any software product serves a novel purpose or provides software product serves a novel purpose or provides novel functionalitynovel functionality ( ( 全新的功能 全新的功能 ) ) approximately 15 %. That means that, in theory, about 85% could be standardized

and reused. In practice, on average, In practice, on average, only 40 percentonly 40 percent is reused. is reused. Smaller modulesSmaller modules are frequently more reusable are frequently more reusable

because they are likely because they are likely 软件组件的粒度问题软件组件的粒度问题 to be documented better and less application specific.

Why only 40% reuse rateWhy only 40% reuse rate

There are several good and some not so good reasons There are several good and some not so good reasons that tend to that tend to limit the extent to reuselimit the extent to reuse. . One common reason is ego( 自负了点 ): the fact that many

software professionals would prefer to rewrite a piece of code from scratch rather than use someone else’s. 不愿用别人的

A related reason has to do with quality. Developers would reuse code developed by others if they could be assured that it would not introduce faults in the new product. 怕出问题 While this concern is sometimes justified,

both reasons reflect the attitude that both reasons reflect the attitude that no piece of code no piece of code is as good as one’s ownis as good as one’s own! ! 真的是这样吗？真的是这样吗？

practical problems associated with reusepractical problems associated with reuse

Retrieval Retrieval 大海捞针大海捞针 unless code is deliberately developed and organized for easy

reuse, it may be difficult to locate it in a database of hundreds and thousands of software modules.

planning for reuse has a price planning for reuse has a price an increase of 60 percent or higher in development cost — plus

the cost of reusing the component and the cost of implementing a reuse process.

there may be legal issues to consider. there may be legal issues to consider. 避免官司！避免官司！ In contract software development, the software belongs to the

client, unless it is just licensed. Internal software reuseInternal software reuse is is free from this constraintfree from this constraint..

Object-oriented systemsObject-oriented systems

OO systems seem to be particularly suited to OO systems seem to be particularly suited to code reuse. code reuse. 貌似很适合貌似很适合 reusereuse Objects are self-contained, which can make both

development and maintenance easier because it allows a large product to be constructed as a collection of smaller products.

Maintenance is generally easier and cheaper, and some classes of objects can be reused.

但尺有所短，寸有所长但尺有所短，寸有所长…………

reuse in OO systems presents its own problemsreuse in OO systems presents its own problems specifically due to inheritancespecifically due to inheritance ！！ the fragile base class problem, occurs when a base class the fragile base class problem, occurs when a base class

changes. changes. All classes depending on the base class have to be recompiled at least,

and possibly changed. indiscriminate use of inheritance: indiscriminate use of inheritance: 滥用继承滥用继承

because dependent classes not only inherit all the attributes of their superclasses but also have their own attributes, they can get big very quickly!

the use of polymorphism and dynamic binding.the use of polymorphism and dynamic binding. 多态与动态多态与动态绑定绑定 Polymorphism and dynamic binding represent the ability to invoke the

same method with arguments of different kinds and have the run-time system determine the appropriate method to use based on the type of argument.

As powerful as it is, this mechanism makes maintenance and reuse of code more difficult to understand and troubleshoot.

What Can Be Reused and When?What Can Be Reused and When?

ReuseReuse is commonly perceived is commonly perceived as being something you do with individual code modules, as in the case of subroutine libraries,

but code is but code is not the only thingnot the only thing that can be worth that can be worth reusing. reusing. if one can browse the inheritance tree of an object-oriented

system, its design can become reusable as well. 继承树本身就可以重用的！

two approaches to incorporating reusetwo approaches to incorporating reuse

There are There are 22 ways ways to incorporating reuse into the to incorporating reuse into the code design and implementationcode design and implementation phases: phases: One can develop the control logic for a product from scratch

and reuse pieces of code that incorporate specific operations of the product.

Alternatively, one can reuse an application framework that already provides the desired control logic and develop application specific operations.

the set of classes used for the design of a compiler. One will need to code from scratch all classes specific to the language

and desired target machine.

Application framework reuse results in higher productivityApplication framework reuse results in higher productivity

Application framework reuse Application framework reuse results in higher results in higher productivityproductivity than toolkit or library reuse because than toolkit or library reuse because more design is reused (recall that design typically has a

higher financial cost than implementation); logic is harder to design and code than are the specific

operations; and because the control logic will have been tested.

Code frameworks, those for performing Code frameworks, those for performing basic basic window manager operationswindow manager operations are similar to are similar to application frameworks in reusability. application frameworks in reusability. 所有操作系所有操作系统平台的统平台的 GUIGUI 界面的窗口管理都有类似的！界面的窗口管理都有类似的！

Design patterns are another kind of design reuse.Design patterns are another kind of design reuse.

A A design patterndesign pattern is a solution to a general design problem in is a solution to a general design problem in the form of the form of a set of closely interacting classesa set of closely interacting classes that have to be that have to be customized to create a specific design. customized to create a specific design.

The idea of design pattern is The idea of design pattern is drawn from the field of drawn from the field of building architecturebuilding architecture. .

A A GUI generator GUI generator is an example of a design patternis an example of a design pattern a tool that assists the user in constructing a graphical user interface,. provides a set of abstract widget classes that the developer can combine

to create a custom user interface. A widget is a graphical object such as a window, a menu, a radio button,

The abstract widget classes have concrete subclasses whose

instantiations are widgets with a specific "look-and-feel“ — such as Motif, SunView, and others.

Software architectureSoftware architecture

Software architectureSoftware architecture is another design concept that, in the is another design concept that, in the future, may lend itself to reuse. future, may lend itself to reuse.

The term "The term "software architecturesoftware architecture" refers to design issues " refers to design issues concerning concerning the organization of a product in terms of its components, product-level control structures, issues of communication and synchronization, databases and data access, physical distribution of the components, performance, and choice of design alternatives.

Object-oriented architectures UNIX pipes and filters client-server architectures

Reuse and MaintenanceReuse and Maintenance

Software reuse Software reuse makes economic sensemakes economic sense, particularly in an , particularly in an organization that produces software for a specific organization that produces software for a specific application domain. application domain.

The higher up-front costs of designing and implementing for The higher up-front costs of designing and implementing for reuse are more likely to pay off during development of reuse are more likely to pay off during development of future products, resulting in both significant savings of cost future products, resulting in both significant savings of cost and time. and time. 考虑越早，省时省钱效果越好考虑越早，省时省钱效果越好

However, in order for reuse to bear fruit, there must be a However, in order for reuse to bear fruit, there must be a commitment from managementcommitment from management: programmer initiative is : programmer initiative is not enough. not enough.

potential beneficial effects of reusepotential beneficial effects of reuse

Although the traditional reason for Although the traditional reason for reusereuse has been to has been to shorten developmentshorten development, the potential beneficial effects of reuse , the potential beneficial effects of reuse also extend to maintenancealso extend to maintenance. .

reuse has reuse has more impactmore impact on maintenance on maintenance than on developmentthan on development 回忆以前提过的各阶段时间成本比例回忆以前提过的各阶段时间成本比例 because the cost of maintenance over a product’s lifetime is roughly

twice that of development (from requirements analysis through integration testing and delivery).

It is worth repeating that the crucial advantage of reuse is It is worth repeating that the crucial advantage of reuse is the availability of the availability of well-documented and well-tested codewell-documented and well-tested code. . Reusing code that does not have these characteristics has little benefit

and potentially can cause significant harm.

6.1.2  Choice of Programming Language 6.1.2  Choice of Programming Language

Programming Language Types Programming Language Types Rapid-Prototyping Language Rapid-Prototyping Language Final Product Language Final Product Language

Programming Language TypesProgramming Language Types

Software practitioners often talk about Software practitioners often talk about fourth-fourth-generation languagesgeneration languages. .

First-generation languagesFirst-generation languages

First-generation languages were First-generation languages were binary machine binary machine codecode, sequences of , sequences of 0s0s and and 1s1s, encoding information , encoding information at the level at which at the level at which machines process it directlymachines process it directly. .

First-generation languages were First-generation languages were quite tedious to quite tedious to write in and very hard to debugwrite in and very hard to debug. .

Second-generation languagesSecond-generation languages

Things improved a bit when Things improved a bit when assembly languagesassembly languages, , second-generation languages, were developed in the second-generation languages, were developed in the late 1940s and early 1950s. late 1940s and early 1950s. 机器指令换成助记符机器指令换成助记符

Although more understandable, assembly languages Although more understandable, assembly languages were were still very processor dependentstill very processor dependent. .

The instructions reflected the The instructions reflected the architecture of the architecture of the processorprocessor ( (how many registers there werehow many registers there were). ).

Individual instructions had to be very specific (Individual instructions had to be very specific (copy copy the contents of register 1 to register 3the contents of register 1 to register 3) and complex ) and complex control structures such as loops had to be spelled control structures such as loops had to be spelled out in excruciating detail.(out in excruciating detail.( 需要详细分解，跳需要详细分解，跳转转………… ))

; 汇编语言程序结构例二（子程结构） ; 用二进制显示中断向量表中数据 D0H 的个数key EQU 0D0H ; 用符号表示常量 ( 关键字 )code SEGMENT ; 代码段开始 ASSUME CS:code begin: MOV AX , 0000H

MOV DS , AX MOV SI , 0000H MOV CX , 0400H MOV BX , 0

MOV AL , key next: CMP [ SI ] , AL

JNZ point INC BX

point: INC SI LOOP next CALL display ; 调用显示子程 MOV AH, 4CH ; 返回 DOS INT 21H

; 用二进制显示 BX 内容子程display PROC

MOV CX , 16 rotate: ROL BX , 1

MOV DL , BL AND DL , 01H ADD DL , 30H MOV AH , 2H INT 21H LOOP rotate

RET ; 子程返回display ENDP

code ENDS ; 代码段结束 END begin ; 指示程序结束和 ; 程序入口

Third-generation languagesThird-generation languages

third-generation languages:third-generation languages: processor independence processor independence and and significantly more abstract control structuressignificantly more abstract control structures. .

These include familiar These include familiar high-level languageshigh-level languages developed developed in the 60s and 70s, such as FORTRAN, Algol, Pascal, in the 60s and 70s, such as FORTRAN, Algol, Pascal, COBOL, C, and Lisp. COBOL, C, and Lisp.

Third-generation languages Third-generation languages are still procedural are still procedural languageslanguages, but each instruction in a high-level , but each instruction in a high-level language corresponds to language corresponds to five to ten machine code five to ten machine code instructions.instructions.

Third-generation languagesThird-generation languages

Object-oriented languagesObject-oriented languages can also be considered can also be considered third-generation languagesthird-generation languages though they organize data and computation in a very different

way. The first object-oriented language was Smalltalk, followed soon

thereafter by C++ and more recently Java. Object-oriented programming has become more and more

popular. Even artificial intelligence languages such as Lisp, designed for

symbolic computation, now have support for object-oriented programming.

Fourth-generation languagesFourth-generation languages

4GLs were intended to abstract out even more 4GLs were intended to abstract out even more procedural detail by encoding in a single statement even procedural detail by encoding in a single statement even more lines of machine code (more lines of machine code (30–50 lines30–50 lines). ).

Several of them areSeveral of them are non-procedural non-procedural — — they specify what should happen but not how it will happen.

several special purpose 4GLs have been developed.several special purpose 4GLs have been developed.

A well-known 4GL is A well-known 4GL is SQLSQL, a language for accessing , a language for accessing information in relational databases.information in relational databases.

A probably less known example of a 4GL is A probably less known example of a 4GL is PrologProlog, , commonly used in commonly used in artificial intelligence applications especially in natural language processing. A Prolog statement can be used to specify that a sentence is

composed by a noun, followed by a verb, followed by another noun.

This statement can be used to analyze a text sentence and determine whether it fits the noun-verb-noun pattern.

Exactly how that happens is hidden from the user in the search mechanism that is deep inside the Prolog interpreter.

4GL would increase programmer productivity significantly?4GL would increase programmer productivity significantly?

Because of the Because of the high ratio of number of 4GL code lines high ratio of number of 4GL code lines to machine code lines( 1:30-50)to machine code lines( 1:30-50), one would think that , one would think that use of a 4GL would increase programmer use of a 4GL would increase programmer productivity significantly. productivity significantly.

In fact, the results have been In fact, the results have been inconsistentinconsistent, with , with reports ranging fromreports ranging from a tenfold increase in productivity, to a negligible increase, to a significant decrease.

It is difficult to evaluate!It is difficult to evaluate!

It is It is difficult to evaluatedifficult to evaluate these reports and make a these reports and make a general statement because general statement because much depends on the particular 4GL and its suitability for the

application. Other factors that influence the impact of introducing a 4GL

into an organization include the availability of CASE tools and the maturity of the organization. 组织的能力成熟度， CMM

There are also serious risks in using 4GLs. There are also serious risks in using 4GLs.

Many were intended for end-user Many were intended for end-user programming programming ，， for allowing end-users for allowing end-users 如二次开发如二次开发等，等， to bypass the programmers in the organization and to write the code required by their specific information access

needs on their own. 用户自行编码 The danger is that end users who are The danger is that end users who are not experienced not experienced

programmersprogrammers may may place too much trust in the correctness of the output from their

4GL programs make harmful decisions based on erroneous output. write an incorrect program modifying a database and corrupting

shared data. 最糟糕的情形了

Rapid-Prototyping LanguageRapid-Prototyping Language

For implementing the final product, there may be For implementing the final product, there may be substantially substantially less freedomless freedom in the choice of a in the choice of a programming language: programming language: the client or the client’s platform may determine it.

On the other hand, the client may simply specify On the other hand, the client may simply specify ""the most suitablethe most suitable" programming language, " programming language, giving the developer complete freedom and the responsibility

for making the right choice.

What is the right choice?What is the right choice?

There is probably no single right choice, but here There is probably no single right choice, but here are some valuable criteria: are some valuable criteria: A language that the client has experience and tools for A language that suits the application type A language that maximizes informational cohesion The language that has the best cost-benefit ratio The language with the lowest risk (risk analysis)

The criteria are The criteria are notnot independentindependent of each other. of each other.

E.g.,E.g., choosing a language that the developer choosing a language that the developer does does not have much experiencenot have much experience in or tools for will in or tools for will almost certainly result in higher costs for training the

programming staff and in higher risk both with respect to being able to 还带来

这些问题 meet deadlines produce high quality code.

special comment for the second criterionspecial comment for the second criterion

The criterion: The criterion: a language that suits the application a language that suits the application type.type.

There is a tendency, currently, to There is a tendency, currently, to use OO languagesuse OO languages for every type of applicationfor every type of application — — C++C++. . C++ compiles into C code. C++ will work on any

platform that supports C. C programmers must be specially trained in the C programmers must be specially trained in the

object-oriented paradigm, which object-oriented paradigm, which may be costlymay be costly. .

special comment for the second criterionspecial comment for the second criterion

Failure to invest the time and money to train them Failure to invest the time and money to train them properly properly poorly designed and implemented code poorly designed and implemented code hard to maintain hard to maintain even costlier even costlier 花的钱更多花的钱更多 If the client wants a scientific computing application,

chances are that the developer should opt for an old-fashioned language like FORTRAN, for which a number of public subroutine libraries are already available.

informational cohesion informational cohesion

Another consideration in choosing a programming Another consideration in choosing a programming language is language is that of maximizing informational cohesion choosing a language that allows co-location of independent

operations performed on the same data structure. Object-oriented languagesObject-oriented languages are particularly good in are particularly good in

this regard,this regard, Other languages also support the creation of Other languages also support the creation of

abstract data typesabstract data types as well. as well.

6.1.3 Good Programming Practices and Coding Standards6.1.3 Good Programming Practices and Coding Standards

Good Programming Practices Good Programming Practices Coding Standards Coding Standards

Good Programming PracticesGood Programming Practices

Choosing the programming language Choosing the programming language

implementing the product design. implementing the product design. A few A few general guidelinesgeneral guidelines should be mentioned for should be mentioned for

building code that is building code that is easier to understand easier to test easier to maintain

Coding style guidelines Coding style guidelines

Coding style guidelinesCoding style guidelines ( (indent the program linesindent the program lines) ) are language specific. are language specific.

For a For a particular programming languageparticular programming language have a have a particular guideparticular guide on good programming practice on good programming practice

Coding style Coding style should keep in mindshould keep in mind that that 替维护者着替维护者着想想 future program maintainers have to understand the code.

This including using This including using identifiers for procedures, variables, constants, object classes,

etc., that are not meaningful only to the author of the code. The variable name xCoordRobotArm is significantly clearer

than just xCoord or xCdRbtArm.

self-documenting codeself-documenting code Although Although self-documenting codeself-documenting code exists, it is very exists, it is very seldomseldom

found. found. The following is an example of a code fragment that is largely The following is an example of a code fragment that is largely

self-documenting. self-documenting. // Read in the server response. // Read in the server response. buf.setLength(0);buf.setLength(0);

while (bytesToRead > 0) {while (bytesToRead > 0) {c = in.read();c = in.read();buf.append((char)c);buf.append((char)c);bytesToRead--;bytesToRead--;}}c = in.read(); // what’s this?c = in.read(); // what’s this?

No line-by-line explanation is needed for the while loop, but No line-by-line explanation is needed for the while loop, but the last line needs a comment. the last line needs a comment.

the last read statement consume an extra control character,what for?

Coding StandardsCoding Standards

Coding standardsCoding standards may be formulated with fixed rules. may be formulated with fixed rules. a module will have between 35 and 50 lines of executable code. While statements such as these may be well motivated, they fail

to convey their real justification 硬性规定行数不恰当！ the modules desire to build are logically and functionally

cohesive. 尽量做到这样 ! A module has A module has logical cohesionlogical cohesion when it performs a when it performs a

series of related actions, one of which is selected by series of related actions, one of which is selected by the the calling modulecalling module.. performs all input and output operations in an application.

A module has A module has functional cohesionfunctional cohesion when it performs when it performs exactly one action or achieves a single goalexactly one action or achieves a single goal calculates sale commission.

Coding StandardsCoding Standards

A seemingly arbitrary coding standard such as the 35 A seemingly arbitrary coding standard such as the 35 – 55 lines of code per module example given above is – 55 lines of code per module example given above is likely either to be ignored or to give rise to modules likely either to be ignored or to give rise to modules with with coincidental cohesioncoincidental cohesion: 35 – 50 lines of : 35 – 50 lines of unrelated code lumped together from smaller modules that do

not belong together 无关代码杂凑在一起 or modules unnecessarily split up 或硬性分开到不同模块

Modules so constructed are Modules so constructed are hard to understand hard to maintain hard to reuse

What a Coding standards do?What a Coding standards do?

Coding standards shouldCoding standards should be justified by providing the real motivation behind the standard and should be phrased in terms that leave some room for programmers

to make the correct decisions, checking with their managers if those decisions result in code that goes

outside the guideline boundaries.

In general, an organization should In general, an organization should try to balance

the usefulness of the restrictions with the burden they impose on programmers

keeping in mind that poorly justified coding standards are a waste of time especially if they cannot be checked by machine

6.2 Module Testing 6.2 Module Testing

the overview of the life-cycle phases that the overview of the life-cycle phases that different kinds of different kinds of testingtesting must be performed on code at different times: must be performed on code at different times: testing of individual modules happens during the implementation phase testing of the product as a whole happens during the integration phase.

In factIn fact because there are some complications involved in testing modules

completely independently of each other and because there are schedule constraints, more temporal overlap in the two types of testing than the above

description suggests.

It is more likely It is more likely testing individual modules pass SQA be integrated in accordance

with the chosen integration strategy

verification techniques verification techniques 验证技术验证技术 VerificationVerification is correctness checking is correctness checking

the process of determining whether a phase of the software process has been carried out correctly.

Collectively termed Collectively termed validationvalidation that must be that must be performed performed before the product is delivered to the before the product is delivered to the clientclient. .

contentscontents

6.2.1 Execution-Based Testing 6.2.1 Execution-Based Testing 6.2.2 Non-execution-Based Testing 6.2.2 Non-execution-Based Testing 6.2.3 Other Testing Approaches 6.2.3 Other Testing Approaches 6.2.4 A Comparison of Module-Testing Techniques 6.2.4 A Comparison of Module-Testing Techniques

6.2.1  Execution-Based Testing 6.2.1  Execution-Based Testing

General Remarks General Remarks Black-Box Module-Testing Techniques Black-Box Module-Testing Techniques Glass-Box Module-Testing Techniques Glass-Box Module-Testing Techniques

General RemarksGeneral Remarks

Programmers are Programmers are expected to test the correctnessexpected to test the correctness of of the modules before implementation of a module is the modules before implementation of a module is considered complete considered complete 程序员测试以保证其代码正确程序员测试以保证其代码正确

but it is the but it is the responsibility of the developer’s responsibility of the developer’s SQA SQA groupgroup to perform methodical testing of each module to perform methodical testing of each module when it leaves the programmer’s hands.when it leaves the programmer’s hands.

There are two basic approaches to testing modules, There are two basic approaches to testing modules, each with its own weaknesses.each with its own weaknesses. Testing to Specifications Testing to Code

Testing to SpecificationsTesting to Specifications

seeks to verify that the module conforms to the specified seeks to verify that the module conforms to the specified input and output while input and output while ignoring the actual codeignoring the actual code. .

also known asalso known as black-box Structured (? 与教材有冲突，教材将其归为代码测试 ) data-driven functional input/output-driven testing,

It is It is rarely possiblerarely possible to to test all modulestest all modules for for all possible input all possible input casescases, which may be a huge number of modules. , which may be a huge number of modules.

Usually time constraints permit only about 1000 test cases Usually time constraints permit only about 1000 test cases per module, so per module, so tests should be chosen carefullytests should be chosen carefully. .

Exhaustive testing(Exhaustive testing( 穷尽测试穷尽测试 ) is feasible only when input ) is feasible only when input combinations are small. combinations are small. 绝大多数时候不可行！绝大多数时候不可行！

Testing to CodeTesting to Code

is quite the opposite of Testing to Specifications. is quite the opposite of Testing to Specifications. It ignores the specification but tests It ignores the specification but tests each path each path

through the codethrough the code. also called . also called glass-box white-box behavioral (? 与教材有冲突，教材将其归为规格说明测

试 ) logic-driven path-oriented testing

Unfortunately, testing Unfortunately, testing each path through the codeeach path through the code is is generally generally not feasiblenot feasible, even for simple flowcharts. , even for simple flowcharts. 一个小程序的可能路径 10 N 数量级的

it is possible to test every path without finding existing it is possible to test every path without finding existing faults, faults, 有时即使穷尽也测不出错误，有时即使穷尽也测不出错误， whywhy ？？ the fault lies in the decision criterion for selecting between paths.

it is possible to exercise successfully all paths with data that it is possible to exercise successfully all paths with data that would not show the fault. would not show the fault. E.g., taking the square root of a negative number. If code preceding the operation yielded only a nonnegative result, when

run with the test data, the fault would not be uncovered. 前边的代码不产生负数结果，就没有机会进入这个操作，产生错误

Testing to CodeTesting to Code

所以所以 exhaustive testingexhaustive testing 也不能保证也不能保证 exhaustive testingexhaustive testing to either code or specifications is to either code or specifications is

neither feasible nor, even if it were feasible, is neither feasible nor, even if it were feasible, is guaranteed to uncover all possible implementation guaranteed to uncover all possible implementation faultsfaults. .

A compromise is needed A compromise is needed 寻求折中的方法寻求折中的方法 one that will highlight as many faults as possible, 只能尽量

多地找错，而无法… .. while accepting that not all faults will be detected. 接受残酷

的现实

Black-Box Module-Testing TechniquesBlack-Box Module-Testing Techniques

The objective inThe objective in black-box black-box testing is to testing is to select test cases that achieve the broadest possible testing

coverage maximizing the chances of detecting a fault while minimizing the possibility of wasting test cases by

having the same fault tested redundantly by more than one test case. 最好不要多个用例检出一个错误

Equivalence testing Equivalence testing 等价测试等价测试 Equivalence testingEquivalence testing is a technique based on the idea is a technique based on the idea

that that the input specifications give ranges of values for which the

software product should work the same way. These ranges of values are termed "equivalence classes" of

input. Boundary value analysisBoundary value analysis seeks to seeks to

test the product with input values that lie on to the side of boundaries between equivalence classes. 经验：选择处于等价类边界的测试用例时，检出错误的

可能性增大

a novel implementation of the division operationa novel implementation of the division operation

there are there are 77 equivalence classes: equivalence classes: the positive numbers greater than 1 1 the positive numbers between 0 and 1 0 the negative numbers between 0 and -1 -1 the negative numbers less than -1

Boundary value analysisBoundary value analysis would test the code with would test the code with the values the values 0, 1, -1,0, 1, -1, and a value for each of the and a value for each of the remaining remaining four equivalence classesfour equivalence classes. .

Output specificationsOutput specifications

Output specifications may be used similarly to Output specifications may be used similarly to establish equivalence classes and boundary values determine the input test cases required.

These two techniques, These two techniques, used in combinationused in combination, make up , make up a powerful approach to discovering faults.a powerful approach to discovering faults.

functional testingfunctional testing

In In functional testingfunctional testing the tester identifies each item of functionality or each

function implemented in the module uses data to test each function.

Functional testing is Functional testing is notnot simplistic since functions simplistic since functions are usually intertwined. are usually intertwined. 底层功能纠结在一起底层功能纠结在一起 A more realistic process for testing functionality called

functional analysis. However, However, functional testingfunctional testing is is problematicproblematic in in

general because functionality often general because functionality often spans across spans across module boundariesmodule boundaries. . 多模块实现一个功能！多模块实现一个功能！ As a result, integration testing and module testing become

blurred. 不能明显区分界限，有时要同时进行！

Glass-Box Module-Testing TechniquesGlass-Box Module-Testing Techniques

One technique for testing to code is One technique for testing to code is structural structural testingtesting. .

In its simplest way, In its simplest way, structural testing is called statement coverage and amounts to running tests in which every statement in the

code is executed at least once, using a CASE tool to keep track of statements that are yet to

be executed. 通常需要工具帮忙 A better version of this technique, A better version of this technique, branch coveragebranch coverage, ,

makes sure that each branching point is tested at least once.

path coverage path coverage 路径覆盖 路径覆盖 A further improvement is A further improvement is path coveragepath coverage. . 功能最强功能最强

大的功能测试方法大的功能测试方法 Researchers are working on techniques for Researchers are working on techniques for reducingreducing

the potentially the potentially very large number of pathsvery large number of paths through through constructs such as loops. constructs such as loops.

路径数量极大，需设法减小！路径数量极大，需设法减小！ An advantage of structural testing is An advantage of structural testing is

when done thoroughly, it can help detect so-called "dead-paths" through the code

code fragments that can never be reached with the given code control structure.

Path 1 test case:Path 1 test case: value(k) = valid input, where k < i for 2 ≤ i ≤ 100 value(i) = 999 where 2 ≤ i ≤ 100 Expected results: Correct average based on k values and proper totals. Note: Path 1 cannot be tested stand-alone but must be tested as part of

path 4, 5, and 6 tests.

Path 2 test case:Path 2 test case: value(1) = 999 Expected results: Average = 999; other totals at initial values.

Path 3 test case:Path 3 test case: Attempt to process 101 or more values. First 100 values should be valid. Expected results: Same as test case 1.

Path 4 test case:Path 4 test case: value(i) = valid input where i < 100 value(k) < minimum where k < i Expected results: Correct average based on k values and proper totals.

Path 5 test case:Path 5 test case: value(i) = valid input where i < 100 value(k) > maximum where k <= i Expected results: Correct average based on n values and

proper totals. Path 6 test case:Path 6 test case:

value(i) = valid input where i < 100 Expected results: Correct average based on n values and

proper totals.

6.2.2  Non-execution-Based Testing6.2.2  Non-execution-Based Testing

Code Walkthroughs Code Walkthroughs Code Inspections Code Inspections Remarks on Non-execution-Based Testing Remarks on Non-execution-Based Testing

Techniques Techniques

When view the code?When view the code?

Black-boxBlack-box and and glass-boxglass-box techniques test the code by techniques test the code by executing itexecuting it. .

Non-execution-based techniquesNon-execution-based techniques rely on review of rely on review of code either code either before or afterbefore or after it has been tested through it has been tested through execution.execution.

about reviewsabout reviews

Software Life Cycle, Software Life Cycle, reviewsreviews are used are used not just with not just with codecode but also with products of but also with products of other stages of the other stages of the software life cyclesoftware life cycle, such as , such as Requirements documents Specification documents

Review tasks should be Review tasks should be assigned to someone other than the original author of the

document and are probably best performed by more than one reviewer

Two basic types of reviewsTwo basic types of reviews

Two basic types of reviews are Two basic types of reviews are code walkthroughs 代码走查 code inspections 代码审查

They differ primarily in They differ primarily in code walkthroughs have fewer steps and code walkthroughs are more informal.

Code Walkthroughs Code Walkthroughs

The The walkthrough teamwalkthrough team should consist of should consist of 4 to 64 to 6 individuals, including individuals, including at leastat least a representative and the manager of the phase being tested

(implementation), a representative of the team that will perform the next phase

in the product’s life cycle (integration testing), and a client representative.

Code Walkthroughs Code Walkthroughs

The team should be The team should be chaired by a member of the chaired by a member of the SQA groupSQA group, , because SQA is the group in the developer’s organization

that has the greatest stake 职责所在 in assuring the correctness of the code.

Participants shouldParticipants should receive material before the walkthrough and prepare lists of items

they do not understand they believe to be incorrect.

The goal of the walkthrough team is to detect faultsThe goal of the walkthrough team is to detect faults

The goal of the walkthrough team is to The goal of the walkthrough team is to detect faultsdetect faults, , not to not to correct themcorrect them. .

The person leading the walkthrough guides the other The person leading the walkthrough guides the other members of the team through the code. members of the team through the code.

The walkthrough can be driven by …… with team members The walkthrough can be driven by …… with team members raising their concernsraising their concerns at the appropriate time. at the appropriate time. 两种方式两种方式 the lists of issues compiled by team members the code itself

In both cases, In both cases, each issueeach issue will be discussed as it comes up and will be discussed as it comes up and resolved into resolved into either a fault that needs to be addressed or 需要指出的错误 a point of confusion that will be cleared up in the discussion 需要澄清

的问题

Code InspectionsCode Inspections

An An inspectioninspection is a is a far more formalfar more formal activity than a activity than a code code walkthroughwalkthrough. .

It is conducted by a team of It is conducted by a team of 3 to 63 to 6 people that people that includes includes representatives from the group responsible for the current

phase (implementation, testing) and representatives from the next phase or phases (integration

testing). One member of the team plays the role of moderator, leading

and managing the team. Another team member is the recorder, who writes down the

faults found during the inspection.

The review process consists of The review process consists of 55 formal steps: formal steps:

the overview step the overview step the preparation stepthe preparation step the inspection stepthe inspection step the rework stepthe rework step the follow-up stepthe follow-up step

The review process consists of The review process consists of 55 formal steps: formal steps:

In the overview step In the overview step the author of the module gives a presentation to the team.

In the preparation stepIn the preparation step the participants try to

understand the code in detail and compile lists of issues ranked in order of severity. use a checklist of potential faults which to be found to help in

this process

The review process consists of The review process consists of 55 formal steps: formal steps:

In the inspection stepIn the inspection step perform a thorough walkthrough of the code aiming for fault detection through complete coverage of the code. Within a day of the inspection, the moderator produces a meticulous

written report 详细的书面报告 .

In the rework stepIn the rework step the individual responsible for the code resolves

all faults issues noted in the written report.

In the follow-up stepIn the follow-up step the moderator must make sure that each issue has been resolved by

either fixing the code or clarifying confusing points.

How to do with the faultHow to do with the fault

An important product of an inspection is An important product of an inspection is the the number and kinds of faultsnumber and kinds of faults found found rated by severityrated by severity. . 统计错误数量及类型，给错误分级统计错误数量及类型，给错误分级

In an inspection, if a module In an inspection, if a module exhibiting a exhibiting a significantly larger number of faultssignificantly larger number of faults than other than other modules in the system modules in the system 远远超过平均水平远远超过平均水平 it is a good candidate for rewriting. 就要重写

If the inspection of two or three modules reveals a If the inspection of two or three modules reveals a large number of large number of errors of specific typeserrors of specific types, , 存在大量存在大量特定类型的错误特定类型的错误 this may warrant (re)checking other modules for similar

errors. 其他模块也会存在！

Something helpfulSomething helpful

Something help in conducting a Something help in conducting a second inspectionsecond inspection of of the module later, such as the module later, such as A detailed breakdown A detailed breakdown 分分类类 ofof types severity number of errors found

If If more than 5 percent of the materialmore than 5 percent of the material inspected inspected must be reworked, the team must reconvene for a must be reworked, the team must reconvene for a full re-inspection. full re-inspection. 如此高的出错率，要找原因了！如此高的出错率，要找原因了！

Remarks on Non-execution-Based Testing TechniquesRemarks on Non-execution-Based Testing Techniques

WalkthroughsWalkthroughs and and inspectionsinspections have been shown to be have been shown to be a very a very powerful methodpowerful method for finding faults in code and other for finding faults in code and other documents. documents. 注意不仅是代码，其他阶段产品也使用该方法注意不仅是代码，其他阶段产品也使用该方法

Some studies have shown that inspections are Some studies have shown that inspections are capable of capable of finding the vast majority of faultsfinding the vast majority of faults in a product in a product in the in the design design and implementation phaseand implementation phase. .

The additional time spent on code reviews and design The additional time spent on code reviews and design document reviews more than pays for itself by leading to document reviews more than pays for itself by leading to early fault detection. early fault detection. 检查越早开始越好，时间成本越小检查越早开始越好，时间成本越小

it is it is less time consumingless time consuming and and more cost effectivemore cost effective to conduct to conduct reviews than reviews than to rely on expensive and not altogether reliable execution-based testing

techniques.

some impediments some impediments （阻碍） （阻碍） to effective non-to effective non-execution-based testing techniquesexecution-based testing techniques

FirstFirst unless the product is modularly decomposable, it will be very

difficult to conduct an effective code review. SecondSecond

during the review process, it is sometimes necessary to refer to documents produced by earlier phases, so these must be completed and up to date for the current phase of the project.

some impediments to effective non-execution-based some impediments to effective non-execution-based testing techniquestesting techniques

ThirdThird unless they are carefully managed, both walkthroughs and

inspections can easily degenerate into performance evaluation sessions for team members 对小组成员的表现评估，？？

where points are scored for finding the maximum number of faults or 给发现最多错误的人计分

where team members are evaluated negatively for not finding faults. 反之……

FinallyFinally it is important that a review session be kept to a maximum of two

hours, since performance of the team is likely to deteriorate if the time for the review extends beyond that. 时间太长，人易疲劳，效率下降

6.2.3 Other Testing Approaches6.2.3 Other Testing Approaches

Correctness Proofs Correctness Proofs Complexity Metrics Complexity Metrics Fault Statistics and Reliability Analysis Fault Statistics and Reliability Analysis The Cleanroom Technique The Cleanroom Technique

Correctness ProofsCorrectness Proofs

Correctness proofsCorrectness proofs are formal, are formal, mathematical proofsmathematical proofs that a code fragment satisfies its input and output that a code fragment satisfies its input and output specifications. specifications.

IdeallyIdeally, they should be developed along with the , they should be developed along with the code, not as an afterthought. code, not as an afterthought. 而不是事后再证明！而不是事后再证明！

Textbook section 6.5.1 Textbook section 6.5.1 （（ P119P119 ）） provides aprovides a simple example of a correctness proof.

Do we need Do we need correctness proofs every time?correctness proofs every time?

Many software engineering practitioners claim that Many software engineering practitioners claim that correctness proofs correctness proofs should not be viewed should not be viewed as a standard as a standard software engineering technique for a variety of software engineering technique for a variety of reasons.reasons.

The technique is The technique is not suited to every software product and best left for critical portions ( 最重要的部分 ) of an application for situations when no other technique is able to provide the

desired certainty that the code is correct. 其他方法无能为力时 Why no longer or not always valid.Why no longer or not always valid. next page…… next page……

Many of these reasons are no longer or not always valid.-1 Many of these reasons are no longer or not always valid.-1

First, it is claimed that correctness proofs First, it is claimed that correctness proofs require a require a mathematical preparationmathematical preparation that that most software most software practitioners do not havepractitioners do not have. . 需要很强的数学背景需要很强的数学背景 However,YOU , computer science graduates currently receive

sufficient training in mathematics and logic to be able to construct such proofs, or to learn how to, on the job. 你们学习过哪些相关课程？

A second claim is that A second claim is that correctness provingcorrectness proving is is too too expensiveexpensive. . Whether this is in fact the case should be determined by a cost-

benefit analysis on a project-by-project basis.

Many of these reasons are no longer or not always valid.-2 Many of these reasons are no longer or not always valid.-2

Another claim has been that Another claim has been that proving the correctness proving the correctness of code is too hardof code is too hard, , but several systems have been proved correct and nowadays the

process is facilitated by the use of tools such as theorem provers.

Still, there are Still, there are some difficultiessome difficulties in formally proving in formally proving the correctness of codingthe correctness of coding including the need to rely on the correctness of the theorem-

proving program itself. 所依据的定理本身正确吗？

Complexity Metrics Complexity Metrics 复杂性度量复杂性度量 Although Although complexity metricscomplexity metrics might be considered a might be considered a

glass-box techniqueglass-box technique because they because they look at the codelook at the code, , they are not used for testing modules but rather for determining which modules should be

emphasized in testing. They are based on the assumption that the more complex a

module is, the more likely it is to contain faults. 越复杂越容易出错

Different complexity metrics have been developedDifferent complexity metrics have been developed

the simplest of which is the simplest of which is based just on the number of lines of code.

More sophisticated predictors have incorporated More sophisticated predictors have incorporated information such as information such as the number of binary decisions, 二元决策（ if-then-else ） the total number of operators and operands, and the number of distinct operators and operands.

complexity metrics can be a useful tool complexity metrics can be a useful tool

There is evidence that There is evidence that complexity metrics can be a useful tool for determining

which modules should be given special attention, although it’s not clear that

more complex metrics provide significant improvement as a criterion for measuring complexity 具有明显的提升作用

Than over just lines of code.

Fault Statistics and Reliability AnalysisFault Statistics and Reliability Analysis

Fault statisticsFault statistics provide a useful metric for determining provide a useful metric for determining whether to continue testing a module or product or whether to recode it.

The number of faultsThe number of faults detected via execution- and non- detected via execution- and non-execution-based techniques execution-based techniques must be recordedmust be recorded. .

Data on different types of faults found via code inspections Data on different types of faults found via code inspections misunderstanding the design initializing improperly using variables inconsistently

These data can be incorporated into These data can be incorporated into checklistschecklists 数据还可以数据还可以用来用来………… for use during later reviews of the same product and future products.

Reliability analysisReliability analysis

Reliability analysisReliability analysis uses statistical-based techniques uses statistical-based techniques to provide estimates of to provide estimates of how many faults are remaining and how much longer it is desirable to keep testing.

It can also be applied in It can also be applied in both implementation and integration phases.

It is based on the assumption that It is based on the assumption that 基于一个假定基于一个假定 the chance of a failure occurring decreases exponentially( 成

指数降低 ) as testing proceeds the longer a product runs without failure, the greater the

likelihood that the product is fault free.

An example of a statistical-based testing techniqueAn example of a statistical-based testing technique

the zero-failure techniquethe zero-failure technique. . 零故障技术零故障技术 It uses It uses

the number of faults stipulatedin the product’s specifications, 在规格说明中规定的故障数量

the number of faults detected so far during testing, and the total number of hours of testing up to the last failure,

determine determine how long to testhow long to test in order to be confident in order to be confident that the product’s that the product’s fault ratefault rate satisfies specifications. satisfies specifications.

The Cleanroom Technique The Cleanroom Technique 净室技术净室技术 The The Cleanroom techniqueCleanroom technique is a is a combination ofcombination of several several

differentdifferent software development techniques. software development techniques. Under this technique, a module Under this technique, a module isn’t compiledisn’t compiled until until

it has passed an inspection, or another kind of non-it has passed an inspection, or another kind of non-execution-based review, execution-based review, a code walkthrough or inspection.

The relevant metric is testing The relevant metric is testing fault ratefault rate,, 故障率故障率 the total number of faults detected per KLOC (thousand lines

of code) a commonly used measure of code quality in the software

industry.

Comparison of the metricComparison of the metric

However, in However, in other techniquesother techniques this metric this metric is applied by the SQA group after the programmer has informally tested a module, and does not include faults found and corrected by the

programmer during development and desk checking. 开发调试时发现的故障

In contrast, in the In contrast, in the Cleanroom techniqueCleanroom technique, the metric , the metric includes the faults found from the time of compilation and

through execution, 编译执行时的故障 but not the ones found by previous inspections and other

non-execution-based techniques. 不含走查时发现的故障的数目

Cleanroom technique has considerable successCleanroom technique has considerable success

The Cleanroom technique has had The Cleanroom technique has had considerable successconsiderable success in in finding and weeding out faults before execution-based testing.

In one case, In one case, all faults were found via correctness-proving techniques, using largely

informal proofs, but with a few formal proofs as well. The resulting code was found to be free of errors both at compilation

and at execution time.

In a study of 17 other Cleanroom software productsIn a study of 17 other Cleanroom software products, the , the products did not perform quite as faultlessly, but they still products did not perform quite as faultlessly, but they still achieved achieved remarkably low fault ratesremarkably low fault rates, an average of 2.3 faults , an average of 2.3 faults per KLOC. per KLOC.

6.2.4  A Comparison of Module-Testing Techniques6.2.4  A Comparison of Module-Testing Techniques

we have surveyed a number of module-testing we have surveyed a number of module-testing techniques, techniques, execution-based non-execution-based

Different studies have tried to draw conclusions Different studies have tried to draw conclusions about the about the relative effectivenessrelative effectiveness of these techniques, of these techniques, but the results have but the results have not always been consistent or not always been consistent or conclusive(conclusive( 确定的确定的 )). .

The most general conclusion The most general conclusion

The The most general conclusionmost general conclusion that can be drawn is that all that can be drawn is that all techniques are techniques are roughly equally effectiveroughly equally effective from from the perspective of finding faults, and each presents some advantages and disadvantages.

One study concluded that One study concluded that professional programmers were able to detect more errors more quickly

with code reading, while advanced students did equally well whether it was with a black-

box or code-reading technique. （ why ？） Both groups performed better with black-box and code reading than

with glass-box techniques, which also tend to be quite expensive on a per fault basis.

OverallOverall code reading found more interface faults, black box found more control faults.

special special difficultiesdifficulties when to when to test object-oriented codetest object-oriented code

Methods associated with object classes may change Methods associated with object classes may change the internal state of object instances, but the internal state of object instances, but the correctness of the resulting internal state cannot be tested

without developing methods and sending messages that give the value of state variables. 需要开发方法

In addition, even though a method may have been In addition, even though a method may have been tested for a superclass, it will still need to be tested for a superclass, it will still need to be retested retested for instances of subclassesfor instances of subclasses, , due to the fact that inheritance permits superclasses and

subclasses of objects to have different methods. 子类经过了扩充，加入了许多新的方法

What the management must do?What the management must do?

Because module-testing practices, except Because module-testing practices, except correctness correctness proofsproofs, are still largely an , are still largely an imperfect scienceimperfect science, ,

management needs management needs to judge testing processto judge testing process, in , in particular with respect to the kind of testing and its particular with respect to the kind of testing and its exhaustiveness.exhaustiveness. 详尽程度 详尽程度 , , 到什么程度为止到什么程度为止

Ultimately, the management needs to decide Ultimately, the management needs to decide whether the cost of proving correctness exceeds the benefit

of checking that the product satisfies its specifications. when to stop testing.

A set of techniques from A set of techniques from reliability analysisreliability analysis can be can be used to provide statistical estimates of the remaining used to provide statistical estimates of the remaining number of faults.number of faults.

What the management must do?What the management must do?

Another decision that management may need to make is Another decision that management may need to make is whether to keep testing a module that is found to contain many faults or whether to have it recoded from scratch.

We mentioned earlier that We mentioned earlier that finding a significantly larger than average number of faults in a module

argues for recoding that module. the probability that further faults will remain increases with the number

of problems found during development. 继续测，会有更多错误出来 Management must Management must

decide the maximum acceptable number of faults, and request that a module be recoded if that number is exceeded.

6.3  Integration Testing6.3  Integration Testing

Top-Down Implementation and Integration Top-Down Implementation and Integration Bottom-Up Implementation and Integration Bottom-Up Implementation and Integration Sandwich Implementation and Integration Sandwich Implementation and Integration General Remarks on Integration Testing General Remarks on Integration Testing

Integration testing Integration testing Integration testingIntegration testing tests the modules' ability to work tests the modules' ability to work

together correctly.together correctly. it is it is difficultdifficult to test modules in isolation to test modules in isolation, particularly , particularly

modules that call each other. modules that call each other.

module calls other modulesmodule calls other modules it is necessary to create stubs — minimal versions of the it is necessary to create stubs — minimal versions of the

called modules. called modules. such a stub for module X should at the very least give a such a stub for module X should at the very least give a

message saying: "Module X was called."message saying: "Module X was called." Better yet, a stub should return values corresponding to Better yet, a stub should return values corresponding to

preplanned test cases.preplanned test cases.

module called by other modulesmodule called by other modules the module being tested is itself called by other modules, the module being tested is itself called by other modules,

a driver must be coded to pass the appropriate arguments.

The effort put into creating drivers and stubs may be The effort put into creating drivers and stubs may be minimal, but it minimal, but it still consumes resourcesstill consumes resources for work that for work that will be thrown away. will be thrown away. 貌似没有什么工作量，但还是要貌似没有什么工作量，但还是要消耗资源的消耗资源的

combine module and integration testingcombine module and integration testing For this reason, if possible, it is best to For this reason, if possible, it is best to combine combine

module and integration testingmodule and integration testing. . 结合单元测试与集成测结合单元测试与集成测试，安排好模块实现的次序！试，安排好模块实现的次序！ First, modules should be tested individually. After integrating two modules, the tester should check that the

partial product continues to behave as it did before adding the new module. 加入新模块后，是否一样正常

three approaches for performing integration testingthree approaches for performing integration testing

There are at least There are at least 3 3 approaches for performing approaches for performing integration testing, each with its own advantages integration testing, each with its own advantages and disadvantages.and disadvantages. Top-Down Implementation and Integration Bottom-Up Implementation and Integration Sandwich Implementation and Integration

Top-Down Implementation and IntegrationTop-Down Implementation and Integration

Suppose that a system has the interconnection Suppose that a system has the interconnection pattern shown in the graph belowpattern shown in the graph below

top-down integration testing sequencetop-down integration testing sequence

A top-down integration strategy begins by testing A top-down integration strategy begins by testing the integration of module the integration of module aa with modules with modules bb, , cc, and , and dd. . It then goes on to testing the integration of modules It then goes on to testing the integration of modules bb and and ee, , cc and and ff, , dd and and ff, , dd and and gg, and so on, until the , and so on, until the leaf modules are reached.leaf modules are reached.

advantages to top-down integrationadvantages to top-down integration

Top-down integration helps Top-down integration helps isolate faults to lower isolate faults to lower modulesmodules as they are being added as they are being added if the higher modules have been tested, faults appear when

new lower modules are added faults can be attributed to those lower modules. 或存在

与与之的接口中 Design faultsDesign faults also show up early in the testing also show up early in the testing

process. Because……process. Because…… Software products include two kinds of modules:

Logic modules, which incorporate the decision-making, flow-of-control aspects of the product; usually found at upper levels.

Operations modules, which perform the actual operations of the product; usually found at the lower levels.

It is wiser to code and test logic modules before It is wiser to code and test logic modules before operations modules.operations modules.

Logic modulesLogic modules are usually are usually more complexmore complex,, design faults are likely to show up in them.

Operation modulesOperation modules are are likely to be reusablelikely to be reusable, even if , even if they are coded first. they are coded first. 一般是原子操作，一般是原子操作，文件读写等文件读写等

if top-level modules are if top-level modules are re-implementedre-implemented to correct to correct logic faultslogic faults the interconnections between the logic and operational

modules will still need to be retested, resulting in unnecessary work. 需要做另外的测试工作 设计错误发现越晚，修改的成本越大

some disadvantages to top-down integrationsome disadvantages to top-down integration

The lower and potentially reusable operational modules The lower and potentially reusable operational modules may may be inadequately testedbe inadequately tested as the testing process runs up against time constraints in the

development schedule. 时间紧迫时首先牺牲的是对底层模块的详细测试

操作模块的可重用性，也会使程序员忽略对之的详细测试 软件被设计的“太好”，模块也可能测试不彻底软件被设计的“太好”，模块也可能测试不彻底

logic tests may prevent the lower modules from ever being tested it promotes defensive programming. （保护程序的存在）

If (X>=0) then y = SquqreRoot(x, errorflag); the caller module include safety checks. input (X = -2) should not occur , 不能通过条件检查

A better practice is responsibility-driven design, in which safety checks (possibly assertions) are built into the called modules. 如将 X 的取值范围测试放到 SquareRoot内

Bottom-Up Implementation and IntegrationBottom-Up Implementation and Integration

In In bottom-up integrationbottom-up integration, , lower-level modules are coded and tested first, higher modules coded as drivers.

coding and testing sequencecoding and testing sequence the leaf modules (h, i, l, m, k) would be coded and tested

first, then their callers (modules e, f, j, g) would be coded and

tested (possibly at the same time), integration of each caller-called module pair would be tested.

Advantage and disadvantageAdvantage and disadvantage

Bottom-up integration Bottom-up integration isolates faults to the upper modules results in thorough testing of the lower-level operations

modules. 专门开发的测试驱动程序，没有保护程序屏蔽，使底层模块充分地被测试

The drawback is that major design faults will be The drawback is that major design faults will be detected late in the process.detected late in the process. 因为体现因为体现设计设计的的逻辑模逻辑模块块集成后期再测试的，设计错误发现很晚集成后期再测试的，设计错误发现很晚

Sandwich Implementation and IntegrationSandwich Implementation and Integration

The The sandwich integrationsandwich integration strategy strategy combines the top-down and bottom-up approach yielding the advantages relieving the disadvantages of both approaches. 扬长避短！

Sandwich Implementation and IntegrationSandwich Implementation and Integration

The sandwich integration strategy The sandwich integration strategy

sequencesequence Logic modules are implemented and integrated top down operations modules are implemented and integrated bottom

up. The interfaces between the two groups of modules are tested

one by one. Note that there is Note that there is some flexibility in determiningsome flexibility in determining

which modules belong to which group. which modules belong to which group. 灵活划分灵活划分 module j could be treated as an operational module while g

could be treated as a logic module. module j could be included among the logic modules or g

among the operational modules.

General Remarks on Integration TestingGeneral Remarks on Integration Testing

Access to inconsistent copies of the design documentAccess to inconsistent copies of the design document is a is a likely cause of type of problem as following. likely cause of type of problem as following. 实现依据不一实现依据不一致造成的致造成的 Inconsistent assumptions held by different programmers — may

produce faults that show up during the integration phase. such as the range of input the number of arguments that a module must accept

所以应该由第三人来决定谁对谁错！所以应该由第三人来决定谁对谁错！ it is better to let the integration test plan and the test phase it is better to let the integration test plan and the test phase

itself be run by the itself be run by the SQA groupSQA group. . Because none of the individuals who participated in coding the

inconsistent modules may be willing to admit that they were responsible for the mistake,

This group also has the most to lose if testing is not performed properly.质保小组的职责所在！

项目实践统计表明项目实践统计表明 The three approaches to integration testing apply The three approaches to integration testing apply

equally wellequally well to to object-orientedobject-oriented and and non-object-non-object-oriented oriented systems. systems. top-down bottom-up sandwich

Integration testing of products with GUI, however, Integration testing of products with GUI, however, poses particular problems.poses particular problems.

In non-GUI testing, it is possible to use a rudimentary In non-GUI testing, it is possible to use a rudimentary CASE tool (CASE tool ( 一般的一般的 CASECASE 工具工具 ) to set up ) to set up input test input test casescases and and expected outcomesexpected outcomes, allowing tests to be , allowing tests to be rerun when new modules get integrated. rerun when new modules get integrated. （（输入与期输入与期望输出数据放在一个文件中与实际输出进行比较，得望输出数据放在一个文件中与实际输出进行比较，得到测试结果到测试结果））

This will This will not worknot work for products that incorporate a for products that incorporate a GUI GUI because because user-input eventsuser-input events （ （ mouse clicksmouse clicks ） ） cannot be stored in a filecannot be stored in a file as other test data can. as other test data can. The solution is to use a special CASE tool, a GUI testing

environment such as QAPartner or Xrunner. this type of tool create a machine-readable version of a manual

GUI test case, called a script that can be run as a normal test case.

6.4  Product and Acceptance Testing6.4  Product and Acceptance Testing

Product Testing Product Testing Acceptance Testing Acceptance Testing

Module testing and integration testing are Module testing and integration testing are verification activitiesverification activities. they check to see . they check to see the individual modules are correct the modules communicate correctly with each other.

integration testing is completed integration testing is completed ………… the the product is delivered product is delivered test the whole product of other behavioral properties.

Who do this?Who do this? it is the responsibility of the SQA group in the developer’s

organization Ensure to find and remove any residual product faults are

Product TestingProduct Testing

The goal of The goal of product testingproduct testing is validation of the is validation of the productproduct checking that the software meets behavioral specifications

including, but not limited to, correctness.

For COTS softwareFor COTS software

the SQA team’s primary concern is that the product the SQA team’s primary concern is that the product be free of faultsbe free of faults, since it will go to as many , since it will go to as many customers as possible. customers as possible.

the financial and public image cost of selling faulty the financial and public image cost of selling faulty COTS productsCOTS products can be prohibitivecan be prohibitive 经费及公众形象经费及公众形象方面，代价巨大方面，代价巨大

after internal product testing is satisfactorily after internal product testing is satisfactorily completedcompleted

the product is shipped to the product is shipped to alpha and betaalpha and beta test - test - sites (prospective buyers) sites (prospective buyers)

receive feedback on receive feedback on any remaining faultsany remaining faults that the that the SQA team may have overlooked. SQA team may have overlooked.

For COTS softwareFor COTS software

Another important test to be performed both in-Another important test to be performed both in-house and at test sites concerns the house and at test sites concerns the utility of the utility of the product,(product,( 产品效用产品效用 )) ， ， the extent to which the extent to which a a product meets customer needsproduct meets customer needs when it is correct when it is used under the conditions permitted by its

specifications. The concept of utility includes The concept of utility includes

whether the product does something useful, whether it is easy to use, and whether it is cost effective relative to competing products. 如金山 wps, 与微软 ms office

For contract softwareFor contract software

most of the remaining testing before delivery will most of the remaining testing before delivery will take place take place in-housein-house. .

The SQA’s primary responsibility is to The SQA’s primary responsibility is to ensureensure the product will pass the acceptance test.

Failure to do so Failure to do so 否则 否则 ………… reflects badly on the management capabilities of the

developer organization 管理无能！ may create a major public relations problem.

So , The SQA group must perform a number of So , The SQA group must perform a number of tests.tests.

What we should do?What we should do?

The product as a whole must be subjected to The product as a whole must be subjected to correctness testingcorrectness testing using black-box techniques similar to the testing at the module level.

The reliabilityThe reliability of the product must be tested. includeof the product must be tested. include estimates of mean time between failures the severity of the effects of a failure.

Severity results must be broken down to includeSeverity results must be broken down to include mean time to repair the product when a fault occurs the time required to repair the consequences of that fault (the

corruption of client data as a result of the fault occurring).

What we should do?What we should do?

The The robustness of a productrobustness of a product under a range of under a range of operating conditionsoperating conditions must be tested. must be tested.

when it receives input that when it receives input that falls outside the range of falls outside the range of legal inputlegal input given in the specifications. given in the specifications. 非法输入时非法输入时 The product should not crash it should minimally respond with an error message saying

that the input could not be processed preferably giving the reason why the input could not work.

What we should do?What we should do?

The product should be subjected to The product should be subjected to stress testingstress testing to to check its behavior check its behavior 强度测试强度测试 under very high user loads (many users logged in at once) volume testing (very large data files, or a large number of

transactions). The product should also be tested against any other The product should also be tested against any other

specified constraints, such asspecified constraints, such as performance (response times) storage, and security.

All documentation should be checkedAll documentation should be checked against against standards set out by the standards set out by the software project software project management planmanagement plan (SPMP). (SPMP).

What we should do?What we should do?

Finally, the product should be checked for Finally, the product should be checked for compatibility with other softwarecompatibility with other software used by the client used by the client (installation testing). (installation testing).

When the SQA group determines that the product When the SQA group determines that the product has passed all these tests, has passed all these tests, the product is ready to be delivered to the client for

acceptance testing.

Acceptance TestingAcceptance Testing

Acceptance testingAcceptance testing is is driven by the clientdriven by the client. . It can be performed

directly by the client, by the developer’s SQA in the presence of a client representative by an SQA group hired by client.

Acceptance testingAcceptance testing may may include all of the types of testinginclude all of the types of testing that were performed by the developer that were performed by the developer duringduring product product testingtesting, but the , but the 44 primary components are testing primary components are testing correctness, robustness, performance, and documentation.

Unlike testing at the developer’s organization, Unlike testing at the developer’s organization, acceptance testingacceptance testing is performed is performed using the using the customer's hardware and actual datacustomer's hardware and actual data. .

If test data should be a reflection of actual data, If test data should be a reflection of actual data, the specifications may be incorrect the SQA may have misunderstood them.

If the product If the product replaces an existing productreplaces an existing product 软件更新软件更新 the specifications should say that the new product would be

run in parallel with the old one until the client is satisfied that it works satisfactorily.

At that time, the old product can be retired.

6.5  CASE Technology6.5  CASE Technology

CASE Technology in General CASE Technology in General Coding Tools Coding Tools Version Management Tools Version Management Tools Build Tools Build Tools Integrated Environments Integrated Environments When to Use CASE Technology When to Use CASE Technology

CASE Technology in GeneralCASE Technology in General

When discussing CASE tools (computer-assisted When discussing CASE tools (computer-assisted software engineering tools), it is important to keep software engineering tools), it is important to keep in mind the following distinctions:in mind the following distinctions: Programming-in-the-small:

coding at the level of the code of a single module Programming-in-the-large:

software development at the module level, including aspects such as architectural design and integration

Programming-in-the-many: software production by a team either at the module level or at the code level

CASE tool scan aid in all aspects of software developmentCASE tool scan aid in all aspects of software development

CASE tools, workbenches, and environments can CASE tools, workbenches, and environments can aid in all aspects of software development. are also helpful in keeping online a single consistent version

of documentation and Aid in communication among developers. store email exchanges over product development issues provide a record of design decisions.

A A particularparticular CASE tool assists in CASE tool assists in only one aspectonly one aspect of of software production. software production. Code editorCode editor

CASE tools are typically classified as CASE tools are typically classified as front-end or front-end or upperCASEupperCASE, , 不同阶段起作用不同阶段起作用 if they assist with the requirements, specification, and design

phases. Similarly, Similarly, back-endback-end oror lowerCASElowerCASE tools tools

assist with implementation, integration, and maintenance. programs for constructing graphical representations of

software such as data flow diagrams tools for constructing data dictionaries.

CASE workbenches and environmentsCASE workbenches and environments

In addition to CASE tools, there are also In addition to CASE tools, there are also CASE CASE workbenches and environmentsworkbenches and environments. .

Workbenches are Workbenches are collections of tools thatcollections of tools that support support a a small number of related activitiessmall number of related activities — like — like editing, compiling, linking, testing, and debugging.

Debugging ， IDE is relevant throughout the is relevant throughout the entire software processentire software process, ,

or or at least a large portionat least a large portion of it. of it.

Coding ToolsCoding Tools Coding toolsCoding tools assist the programmer with many aspects of assist the programmer with many aspects of

programming. programming. Commonly available coding tools include: Commonly available coding tools include:

Structure editors: tools for writing code that know about the programming language in which the code is written and that can catch syntax errors;

Pretty printers: tools that produce nicely laid out programs; Online interface checkers: tools that check that procedures, functions,

and/or methods exist and are being called with the right number and type of arguments;

Operating system front ends: tools that facilitate compiling, linking, and loading;

Informative error messages: tools capable of displaying run-time errors at a level appropriate for the programming language (for example, "Line 24: division by 0," instead of "Segmentation fault" for a FORTRAN program); and

Source-level debuggers: run-time tools that produce trace output or can be used to debug the code interactively by setting breakpoints.

structure editorstructure editor

a structure editor a structure editor incorporates some or all of the other coding tools, support for online documentation, the resulting collection of tools constitutes a programming

workbench.

Version Management ToolsVersion Management Tools

During development and maintenance of a software During development and maintenance of a software product, product, several revisions of each moduleseveral revisions of each module may arise may arise in response to requests in response to requests for fault removals enhancements

These changes produce These changes produce multiple versions of multiple versions of modulesmodules, which can become a , which can become a nightmarenightmare unless unless properly managed. properly managed. 程序员的恶梦！程序员的恶梦！

The CASE tools The CASE tools help manage software versionshelp manage software versions.. Version control should begin when the module has passed

SQA inspection. Before that, there will be too many versions for each module.

Software-versioning toolsSoftware-versioning tools

Software versioning toolsSoftware versioning tools help organizations help organizations manage manage multiple versions

Software-versioning tools help developers keep track of different versions of the software product or of components of the product, such as

following an incremental life-cycle model 增量开发 making revisions during maintenance.维护时修改

variations. 模块的异构版 Matters are further complicated by the coexistence of module

variations. different implementations of the same module for different hardware

platforms.

Revision and variationsRevision and variations

Configuration-control toolsConfiguration-control tools are used to specify are used to specify which versionswhich versions and and which variationswhich variations of each of each component component are requiredare required forfor a particular version of a particular version of the complete productthe complete product — that is, its configuration. — that is, its configuration. 不同的配置，形成不同的最终完整产品不同的配置，形成不同的最终完整产品

Baselines and access control systemsBaselines and access control systems prevent prevent programmers from programmers from working on the same module working on the same module simultaneouslysimultaneously. .

How the Baselines and access control systems do?How the Baselines and access control systems do?

The maintenance manager creates a The maintenance manager creates a baseline baseline configurationconfiguration for the product. for the product.

A programmer needing to work on a moduleA programmer needing to work on a module copies that configuration into his or her private workspace freezes or locks the module that needs to be changed. 做标记，锁定！ Microsoft sourcesafe

No other programmer is allowedNo other programmer is allowed to work on a to work on a frozen module. frozen module.

the changes have been implemented and tested the changes have been implemented and tested the new version of the module is installed and the new version of the module is installed and unfrozen unfrozen changing the baseline.changing the baseline.

Build ToolsBuild Tools

Build toolsBuild tools assist in assist in selecting and assemblingselecting and assembling all the all the correctcorrect versions of the code. versions of the code.

A build tool needs to A build tool needs to be used in conjunction with a be used in conjunction with a configuration-control toolconfiguration-control tool, ideally to be , ideally to be integrated integrated withwith it. it. the UNIX make command, which uses a makefile to specify

all components of the system that must be compiled linked in which order.

Integrated EnvironmentsIntegrated Environments

In the context of CASE, integrated environments In the context of CASE, integrated environments usually in reference to usually in reference to user interface integrationuser interface integration — — an environment where tools

all have the same visual appearance and provide access to similar functionality in similar ways.

Macintosh and the Microsoft Windows environments.

Process integrationProcess integration refers to an environment that refers to an environment that supports a supports a specific software processspecific software process. . 与模型有关与模型有关

In In tool integrationtool integration, all tools communicate via the , all tools communicate via the same data formatsame data format. .

Team integrationTeam integration takes place when a CASE takes place when a CASE environment promotes effective team environment promotes effective team coordination coordination and communicationand communication. . 调度与通信调度与通信

management integrationmanagement integration refers to a CASE refers to a CASE environment that supports management of the environment that supports management of the process. process.

Until now, by and large Until now, by and large only only tool integrationtool integration has has been achievedbeen achieved..

When to Use CASE TechnologyWhen to Use CASE Technology

CASE technology can CASE technology can improve programmer productivity, shorten development time, and increase product user satisfaction.

However, CASE technology requires training and one However, CASE technology requires training and one should should use it with careuse it with care. .

CASE environments, like programming languages, need to CASE environments, like programming languages, need to be be matched with the situation of usematched with the situation of use. .

Low maturityLow maturity organizations can use CASE tools and organizations can use CASE tools and workbenches, workbenches, but not environmentsbut not environments. . Environments automate a process, so it is important that it be the right

process for the organization. When there is no process in place, they automate chaos. 使用不当，适得其反！