Nonbehavioral Specifications

08100810

Non-behavioral Characteristics

PortabilityPortability ReliabilityReliability EfficiencyEfficiency Human Engineering Human Engineering TestabilityTestability UnderstandabilityUnderstandability ModifiabilityModifiability

Portability

Degree to which software running on one Degree to which software running on one host computer environment can be host computer environment can be converted to run on another.converted to run on another.

Not necessary for all applications Not necessary for all applications (embedded systems, single-use systems).(embedded systems, single-use systems).

Some applications, it is essential.Some applications, it is essential.

Problems with specification

May be impossible to quantify: “The May be impossible to quantify: “The maximum time to port to host system X maximum time to port to host system X shall be …”shall be …” We don’t know what the next generation We don’t know what the next generation

will be;will be; Does the maximum time affect the design Does the maximum time affect the design

of the system?of the system?

Approaches to Specifying Portability

Source languageSource language Java: JVM ported to lots of platformsJava: JVM ported to lots of platforms Ada: DoD certifies – no extentions, no subsetsAda: DoD certifies – no extentions, no subsets Ideally, language is a design issue, but if its effect on Ideally, language is a design issue, but if its effect on

portability is critical, it’s a requirementportability is critical, it’s a requirement Language selection tends to be political in organizationsLanguage selection tends to be political in organizations

Host operating systemsHost operating systems Say which ones up front, if you knowSay which ones up front, if you know

Compiler selectionCompiler selection Ansi Standard compilersAnsi Standard compilers

Reliability

This is difficult in software:This is difficult in software: ““The system shall be 99.999% reliable.”The system shall be 99.999% reliable.”

What does this mean?What does this mean? It could mean that the phone system may lose a It could mean that the phone system may lose a

call now and again, but the entire system must call now and again, but the entire system must not fail for more than 5 minutes a year.not fail for more than 5 minutes a year.

In a patient monitoring system, it may mean In a patient monitoring system, it may mean that if it does go down, it alerts staff. It must that if it does go down, it alerts staff. It must not make a mistake in monitoring more than not make a mistake in monitoring more than one patient in 100,000.one patient in 100,000.

Traditional reliability (hardware) MTTFMTTF

MTTF of system is well defined in terms of MTTF of components.MTTF of system is well defined in terms of MTTF of components. Redundant components improve reliability because failure of components Redundant components improve reliability because failure of components

is independent.is independent. Hardware degrades in its environment.Hardware degrades in its environment. Bathtub curve for electronicsBathtub curve for electronics

time

failures

Burn-in

MTTF and Software

Software doesn’t degrade over time.Software doesn’t degrade over time. Suppose you run a program for 10 years Suppose you run a program for 10 years

without failure, then it suddenly fails. without failure, then it suddenly fails. Why?Why?Software was changed.Software was changed.Software was used a new way.Software was used a new way.

Bugs

Failure: Software does not do what is required Failure: Software does not do what is required (specified). Behavior is different from that needed.(specified). Behavior is different from that needed.

Fault: A cause of a failure. Fault: A cause of a failure. Not all faults result in failure.Not all faults result in failure. All failures result from faults.All failures result from faults. A state in which there is a fault without a A state in which there is a fault without a

failure is a failure is a hazardhazard state. state. Error: A design or implementation flaw.Error: A design or implementation flaw.

Testing

The purpose of testing is not to demonstrate The purpose of testing is not to demonstrate correct execution of the program.correct execution of the program.

The purpose of testing is to discover faults.The purpose of testing is to discover faults.

Problems specifying reliability in terms of bugs. Assume quality is designed in from the start:Assume quality is designed in from the start:

The software testing shall require no more than two The software testing shall require no more than two months.months.

Software testing shall discover no more than 10 bugs.Software testing shall discover no more than 10 bugs. The software shall have no more than one bug per The software shall have no more than one bug per

thousand lines of code.thousand lines of code. We want zero bugs, so this must be better than 5 per We want zero bugs, so this must be better than 5 per

1000.1000. How do we know how many there are? Wait until How do we know how many there are? Wait until

software is retired, then count them.software is retired, then count them.

Fault Seeding

Before testing, insert some bugsBefore testing, insert some bugs Track how many of these are found in Track how many of these are found in

testing.testing. Total bugs in system = Total bugs in system =

(#seeded * total_detected)/seeded_detected(#seeded * total_detected)/seeded_detected

Example

Secretly seed 10 bugs.Secretly seed 10 bugs. Test team discovers 120 bugs, 6 of which Test team discovers 120 bugs, 6 of which

are seeds.are seeds. Bugs = 10 * 120 / 6 = 200 bugsBugs = 10 * 120 / 6 = 200 bugs # bugs remaining = 200 – 120 = 80, 4 of # bugs remaining = 200 – 120 = 80, 4 of

which are “known”.which are “known”.

Notes

Not all bugs are equalNot all bugs are equal Equally difficult to findEqually difficult to find Equally difficult to repairEqually difficult to repair

Seeding is harder than it looks.Seeding is harder than it looks.

IntuitiveIntuitive Measure of testing effectivenessMeasure of testing effectiveness

Reliability

Levels of criticalityLevels of criticality Cause mild inconvenienceCause mild inconvenience Cause minor financial lossCause minor financial loss Cause major embarrassmentCause major embarrassment Cause major financial lossCause major financial loss Injure peopleInjure people Kill a few peopleKill a few people Kill many peopleKill many people Destroy humankindDestroy humankind

Example Reliability Requirement

No more than five bugs per 10K lines of executable No more than five bugs per 10K lines of executable code may be detected during integration and code may be detected during integration and system testing. No more than ten bugs per 10K system testing. No more than ten bugs per 10K lines of executable code may remain in the system lines of executable code may remain in the system after delivery, as calculated by the Monte Carlo after delivery, as calculated by the Monte Carlo seeding technique defined in Appendix III. The seeding technique defined in Appendix III. The system must be 100% operational 99.9% of the system must be 100% operational 99.9% of the calendar time during its first year of operation.calendar time during its first year of operation.

Efficiency

The utilization of scarce resources.The utilization of scarce resources. MemoryMemory CPUCPU DiskDisk CommunicationCommunication

Easy to specify if limits are givenEasy to specify if limits are given Example: ATC system: Example: ATC system: The system shall trace The system shall trace

the movements of up to fifty aircraft.the movements of up to fifty aircraft.

Need to say how it will degrade:

What if there are 51 aircraft? Possibilities:What if there are 51 aircraft? Possibilities: Software fails.Software fails. Track first 50, ignore 51Track first 50, ignore 51stst.. Software notifies 51Software notifies 51stst pilot to leave area. pilot to leave area.

Human Engineering: Levels of Specification The system shall have an easy-to-use The system shall have an easy-to-use

human interface.human interface. The system shall be menu driven.The system shall be menu driven. The system shall be menu-driven. Appendix The system shall be menu-driven. Appendix

A shows sample menus.A shows sample menus. The system shall be menu-driven. Appendix The system shall be menu-driven. Appendix

A shows all menus to be used.A shows all menus to be used.

Human Engineering: Error Messages Unless there is a sound understanding of the Unless there is a sound understanding of the

types of error messages the system can types of error messages the system can generate, there is insufficient knowledge of generate, there is insufficient knowledge of the system’s expected normal behavior.the system’s expected normal behavior.

It is a good idea to have an appendix that It is a good idea to have an appendix that specifies the text of all error messages.specifies the text of all error messages.

Testability, Modifiability, Understandability Very difficult to quantify.Very difficult to quantify. These are important contributors to cost These are important contributors to cost

(maintenance).(maintenance). One suggestion is to specify conformity to a One suggestion is to specify conformity to a

set of programming standardsset of programming standards

Programming standards specify

Naming conventionsNaming conventions Invocation conventionsInvocation conventions

Calls, interrupts, synchronizationCalls, interrupts, synchronization Message formatsMessage formats

Component headersComponent headers Format and contentFormat and content

In-line documentation styleIn-line documentation style Use of global constructs and variablesUse of global constructs and variables Use of named constructsUse of named constructs Modularity standardsModularity standards

Where to put these?

Be sure you really have these requirementsBe sure you really have these requirements Sections 5 and 6 in SRSSections 5 and 6 in SRS

5. NON-BEHAVIORAL REQUIREMENTS5.1. PERFORMANCE REQUIREMENTS

5.2. SECURITY

5.3. QUALITATIVE REQUIREMENTS

5.3.1. Availability

5.3.2. Maintainability

5.3.3. Portability

5.3.4. Design and Implementation Constraints

6. Other Requirements

6.1. DATABASE

6.2. OPERATIONS

6.3. SITE ADAPTATION