Download - Formal Methods Diffusion

Transcript
Page 1: Formal Methods Diffusion

Bundesamt für Sicherheit in der Informationstechnik

Postfach 20 03 63, 53133 Bonn, Germany

achieving dependable systems

Adelard, Coborn House, 3 Coborn Road, London E3 2DATel: +44 (0)181 983 1708, Fax: +44 (0)181 983 1845

Formal Methods Diffusion:Formal Methods Diffusion:

Prospects

© BSI 2000

Version 1.0 September 2000

Page 2: Formal Methods Diffusion
Page 3: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 3 of 71

Version 1.0 September 2000

Foreword by the Sponsor

The idea to initiate this study was born during the discussion of a 'Formal Methods Road Map'during the workshop “Current Trends in Applied Formal Methods”.in Boppard (Germany),1998. There it was impossible between the participants of the workshop to agree on a commonview on the future role of formal Methods in practice. The judgement was quite varying. Manyexperts were optimistic in their opinion of the increasing use of formal methods in safety andsecurity critical applications in the future. On the other hand there were quite a few experts whodid not share this optimistic view regarding the fact that there has been a lot of financial supportfor formal methods during the last decade - without real success.The result of the discussion during the workshop was not a statement but a question:

'What are the identifying factors that lead to success or failure of the application of formalmethods in software development ?'

In this study we carefully try to find an answer to this question.

This report is the result of a study by Adelard plc, London, United Kingdom for the Bundesamtfür Sicherheit in der Informationstechnik (BSI), Bonn, Germany. It is advanced by the BSI withselected German perspectives and programmes in the past.

Page 4: Formal Methods Diffusion

Page 4 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

Summary

The objective of this study is to identify factors leading to the success or failure of theapplication of formal methods as exemplified by their use by industry and through past R&Dprogrammes. The overall aim is to inform future formal methods dissemination activities andother initiatives. The objective has been achieved through the review of existing surveys, thereview of programme evaluations, and interviews with formal methods practitioners, sponsorsand other (past or present) technology stakeholders.

The application of formal methods has a long history but the software engineering communityhas not substantially adopted them at large and we have identified numerous reasons for thefailure to adopt. However there is a significant take up of formal methods in critical industries.Large hardware vendors are either developing in-house capabilities in formal verification (e.g.Intel, Siemens) or are adopting externally developed technologies and in the area of safety thereis active research and serious application of formal methods throughout the safety lifecycle. Wecalculate that about $1-1.5B is spent annually on formal methods activities world-wide.

We identify factors to increase the adoption of formal methods. One major recommendation isthat unlike other R&D programmes we have investigated, any future programme should adopt asystematic technology adoption framework, of which we provide two examples, and take amore explicit view of how the market in high technology products actually develops. Weconsider this to be the single most likely factor to increase the chance of successful adoption.We also identify the need for sustained investment in tools and continued R&D.

Authors

R E BloomfieldD Craigen (ORA Canada)

Page 5: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 5 of 71

Version 1.0 September 2000

Contents

1 Introduction...........................................................................................................................7

2 The study approach ................................................................................................................7

3 Results ..................................................................................................................................93.1 R&D programmes..................................................................................................9

3.1.1 Alvey .....................................................................................................93.1.2 Esprit ...................................................................................................103.1.3 NASA and US Programmes..................................................................133.1.4 R&D funding........................................................................................14

3.2 The formal methods landscape - the conference circuit .........................................153.3 Key industrial areas and applications....................................................................19

3.3.1 Introduction..........................................................................................193.3.2 Safety related systems...........................................................................193.3.3 Security applications.............................................................................223.3.4 Hardware and microcode verification....................................................25

3.4 Preliminary conclusions .......................................................................................26

4 Analysis ..............................................................................................................................284.1 The formal methods market..................................................................................28

4.1.1 Characteristics of the market.................................................................284.1.2 Economic and other drivers ..................................................................294.1.3 The impact of development process failure ...........................................294.1.4 The impact of communities...................................................................294.1.5 Impact of government and standardisation bodies .................................31

4.2 Technology adoption models................................................................................324.2.1 Introduction..........................................................................................324.2.2 Technology diffusion............................................................................324.2.3 Developing high technology markets ....................................................354.2.4 Adoption of the TALC approach...........................................................384.2.5 Success and failure factors....................................................................38

5 Recommendations and conclusions......................................................................................415.1 Past failures .........................................................................................................415.2 The current landscape ..........................................................................................425.3 Present limits .......................................................................................................425.4 Increasing adoption of formal methods.................................................................43

5.4.1 Apply technology adoption models .......................................................435.4.2 Sustained investment in tools................................................................445.4.3 Address differences in target users........................................................445.4.4 Focus on critical application areas ........................................................445.4.5 Continue R&D ......................................................................................44

6 Acknowledgements .............................................................................................................45

7 References...........................................................................................................................45

Appendix A What are Formal Methods?..................................................................................51

Appendix B Future Programmes in US....................................................................................55

Page 6: Formal Methods Diffusion

Page 6 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

B.1 NASA ................................................................................................................55B.2 Critical infrastructure .........................................................................................56B.3 Other US programmes ........................................................................................57

Appendix C Protecting America’s critical infrastructures: PDD 63 .........................................59

Appendix D Keeping America secure for the 21st century: ......................................................61

Appendix E Trust in cyberspace ..............................................................................................63

Appendix F European R&D projects .......................................................................................65F.1 Esprit projects......................................................................................................65F.2 ESSI....................................................................................................................67

Appendix E Selected German R&D Programmes ....................................................................68

Page 7: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 7 of 71

Version 1.0 September 2000

1 IntroductionThe application of formal methods has along history but they have not beensubstantially adopted by the softwareengineering community at large. To gain aperspective of what is working and what isnot in the formal methods area we havereviewed their use by industry and theresults of past R&D programmes. Theobjective is to identify crucial factorsleading to the success or failure of theapplication of formal methods and in doingso provide a perspective on the currentformal methods landscape. The overall aimis to inform future formal methodsdissemination activities and otherinitiatives.

The report is organised as follows. Theoverall approach to the study is outlined inSection 2. The results of the reviews arepresented in Section 3 as follows:

• from the perspective of Europeanand US R&D programmes(Section 3.1)

• from the viewpoints providedfrom the current conferencecircuit (Section 3.2)

• we then review applications inkey industrial areas of safety,security and chip manufacture(Section 3.3)

We then make some further analysis, froma market point of view, of the size and

nature of the formal methods markets(Section 4.1) and in Section 4.2 provide ananalysis based on technology diffusionmodels and the technology adoptionlifecycle. This analysis is then drawntogether in Section 5 into a set ofconclusions and recommendations.

Appendix A provides a brief introduction toformal methods.

2 The study approach

The study was based on a review ofexisting surveys (especially [Survey]), thereview of programme evaluations (e.g.[SPRU91]), a proposal for a newprogramme [AFM] and interviews withformal methods practitioners, sponsors andother (past or present) technologystakeholders.

An interview brief was developed to act as“aide memoire” for those conducting theinterviews. In our experience it is notappropriate to conduct this type ofinterview through a rigid question andanswer format: we are dealing with seniorand technically sophisticated interviewees.Instead it is more productive to have anumber of topics that we wish to cover anduse these to trigger lines of discussion andto revisit at the end of the interview. Oftenthe interviewees had a particular story totell and we wished to hear it. The topicsraised in the interviews are shown in Table1 below.

Page 8: Formal Methods Diffusion

Page 8 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

Table 1: Interview brief

Preliminaries

Describe role of interviewee in past and present including role in formal methods.

Outline the objectives of study.

People

What is the team and the interviewee doing now? E.g. left FM work on: otherR&D, industry as software engineer, a different discipline.

Tools and methods

Where have the tools gone?

What is used now?

Where did it come from? What type of programme?

Ideas

What persists? What key ideas have flourished, where have they come from?

Impact

What is the experience with technology transfer?

Research policy

Has the right balance been struck between competition vs dilution of ideas andloss of focus?

What about the idea of picking winners?

We augmented the interview results withthe formal methods page of the World WideWeb Virtual Library [FMVL] and ourobservations from a cross-section ofconferences:

• The 5th and 6th InternationalSPIN Workshops on PracticalAspects of Model Checking[SPIN99].

• Computer Aided Verification,11th International Conference,

CAV’99 [CAV99].

• Fundamental Approaches toSoftware Engineering, FirstInternational Conference[FASE98].

• Formal Methods for TrustworthyComputer Systems [FM89].

• FM’99: World Congress onFormal Methods [FM99].

Page 9: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 9 of 71

Version 1.0 September 2000

• First and Second Conferences onFormal Methods in Computer-Aided Design [FMCAD96][FMCAD98].

• Applied Formal Methods: FM-Trends 98 [FMTrends].

• Formal Software Verification,Intel Research Symposium[Intel98].

• Theorem Proving in HigherOrder Logics [TPHOL99].

• ZUM’98: The Z FormalSpecification Notation [ZUM98].

We also considered three key industrialareas: safety, security and chipdevelopment.

The intention is that in addressing thesedifferent viewpoints, we provide asufficiently broad and accurate picturewithout resorting to an exhaustive survey.

We then developed a more market orientedanalysis of the results with a discussion ofthe formal methods adoption through twokey models. The first of these is the generictechnology diffusion work of Rogers andthe second is the high technology marketingof Moore.

3 Results

3.1 R&D programmes

3.1.1 Alvey

The Alvey programme was a five yearprogramme of pre-competitive R&D in ITthat started in 1983 as the UK’s response tothe Japanese 5th Generation computingproject. It supported 192 collaborativeprojects involving a mix of academic andindustrial partners and about 117 “Uncled”projects, and ran in parallel to ESPRIT1.

Government funding was £200M withabout £27M in software engineering. Thesoftware engineering part of the programmehad a strong academic flavour and many ofthe small projects were expected to lead totools for in-house use or commercialexploitation. There were also some largeindustrially led projects. The officialevaluation [SPRU91] was that exploitationperformance was low. It identifies barriersto uptake as:

• lack of skilled user base

• high investment costs

The reasons for projects failing were alsoassessed, and changes to objectives andover ambition were seen as more commonthan technical problems. The turbulence inthe UK IT industry often meant that changein ownership of companies and subsequentchanges in strategy occurred during theprogramme. Some of the lessons, in termsof needing product groups from largecompanies to be involved, not just R&Dgroups, have fed through to otherprogrammes since Alvey. In terms of theevaluation of the programme [SPRU91] theformal methods component had littlesuccess in promoting widespread adoptionor developing lasting tools. However it wassuccessful in:

• Raising awareness andexpectation of formal methodsaround the time Def Stan 00-55was being planned anddeveloped.

• Raising the perception of the UKstrength in formal methods andthe perceived formal methodsgap between the US and the UK(a gap debated at FM89 [FM89]).

• Leading to a significant numberof people with some researchexperience in formal methods.

Page 10: Formal Methods Diffusion

Page 10 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

3.1.2 Esprit

Programme overview

Esprit— the European Strategic Programmeon Research on Information Technology—was a large multi-annual programme in fourphases:

Esprit 4: 1994-1998

Esprit 3: 1990-1994

Esprit 2: 1988-1991

Esprit 1: 1984-1988

There was also significant IT activitywithin programmes on advanced telematics(ACTS) in which the European Infosecwork was supported and stimulated. TheR&D policy underpinning Esprit wasaiming to provide for the new informationinfrastructure:

• providing and demonstrating thebuilding blocks for informationsociety applications

• led by user and market needs

• emphasising access toinformation & technologies, onusability and on best practice

• focusing on applicability.

Esprit focused on eight intertwined areas ofresearch:

Long-Term Research aimed toensure that, at any one time, thepotential for the next wave ofindustrial innovation was maintainedand that the expertise underpinningEuropean information technologyR&D was replenished in those areaswhere it was most needed. This areawas open for new ideas and people,responsive to industrial needs, andproactive with respect to

technologies that would shape futuremarkets.

A further three areas dealt withunderpinning technologies:

Software Technologies aimed tomaintain a strong base of high qualityand relevant skills and keytechnologies within all sectors of theEuropean economy for whichsoftware development formed animportant component of businessactivity.

Technologies for Components andSubsystems concerned thedevelopment and broad exploitationof a wide range of microelectronicssolutions for electronic systems.Work encompassed equipment,materials and processes used inmanufacturing semiconductors,through to electronic design tools,packaging and interconnect solutions.The area included work on peripheralsubsystems such as storage anddisplays, and work on microsystems.

Multimedia Systems encouraged thedevelopment of the technologies andtools necessary for industry toimplement multimedia end-usersystems.

The other four areas were “focusedclusters”— sets of projects andcomplementary measures combined andmanaged in order to achieve particularresearch and industrial objectives.

The Open Microprocessor SystemsInitiative’s strategic goal was toprovide Europe with a recognisedcapability in microprocessor andmicrocontroller systems, and topromote their worldwide use.

The High-Performance Computingand Networking cluster emphasisedareas that are only now nearing wideapplicability. For example, the use of

Page 11: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 11 of 71

Version 1.0 September 2000

parallel systems for the substitutionof simulation for experimentationand testing.

Technologies for Business Processesaimed to support the change andtransformation of enterprises to takebest advantage of informationtechnologies, business process re-engineering and human resources.

Integration in Manufacturing aimedto accelerate and enhance the abilityof European manufacturing industryto capitalise on the emergence of a

powerful global informationinfrastructure.

The participation in ESPRIT wasapproximately equally divided betweenR&D organisations (30%), SMEs (30%)and large companies (40%). About 1100organisations were selected in the first 6calls of ESPRIT with a budget of 1089Mecu with, on average, later projects beingsmaller. In all 1 in 4 proposals wereaccepted for funding. A condition of mostindustrial R&D projects was that theybrought together companies and researchinstitutions from at least two EU/EEAcountries.

Domain Number Funds/Mecu

Software Technologies 376 186

Technologies for Components and Subsystems 150 283

Multimedia Systems 64 90

Long Term Research 91 82

Open Microprocessor Systems Initiative 59 97

High Performance Computing and Networking 114 125

Technologies for Business Processes 88 85

Integration in Manufacturing (IiM) 79 111

A significant part of the programme wasdevoted to measures designed to increaseinteraction between users and developers,disseminate results more widely, build trialapplications, and boost product and processadoption in the market. Thesecomplementary measures represent around20% of overall funding in Esprit 4.

Formal methods aspects

Formal methods related projects andactivities have been funded throughoutEsprit . We have searched a number ofCommission databases to try and establishthe scope and extent of the funding forformal methods R&D. Searching on“formal methods”, “formal verification”,

“correctness” and “model checking “produced a list of some 150 projects.Filtering out those who use “formalmethods” in less specific sense or for whomwe judge the formal aspect was not a largecomponent leaves about 60 projects. In all,9 projects mention “model checking”. Aselected list of projects that mention formalmethods in their description is provided inAppendix F.

The type of project varies enormously fromfundamental work in computer science(e.g., the independent discovery in Europeof model checking, work on concurrency)through to application experiments. Theearlier programmes contain more onmethods and tools with a much more

Page 12: Formal Methods Diffusion

Page 12 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

specific and application driven approach inthe last phase (Esprit 4). Of the 60 projectsabout 5–10 are basic research fundinguniversity groups, about another 10–15 aresmall process improvement experiments ordissemination activities, leaving about 40that one might consider as core formalmethods projects. If we take the averagefunding to be 1Mecu this gives an R&Dspend by the Commission of 40Mecu whichwill rise to about 60Mecu if the industrycontribution is included. Overall formalmethods are a very small part of the Espritprogramme, representing some 0.6–1% ofthe projects funded.

There is also a database of success stories(Promosa) and a search using keywords offormal verification, formal methods andmodel checking produced the followingresults:

A surface compiler for temporallogic (SCTL)Temporal surface languagerepresentations can now be translatedinto a classical logic language andelaborated using fully automatictools, for easier formal verification ofsystems.

Automatic Shiploading OptimisationSystem (ANKO)A ship load planning & optimisationsoftware package which uses formalmethods in its design andspecification, so as to achieve veryhigh levels of accuracy andreliability.

Automating Safety Analysis (SAM)Based on standard fault analysistechniques, a systematic andautomated methodology foranalysing safety-critical softwarereduces total development costs andimproves overall safety levels.

Development Environment for PLC-based applications (HERACLES)HERACLES helps suppliers andproducers of automated systems to

save time— and thus cut costs— fromthe design phase onwards.HERACLES shortens theengineering process and reduces thenumber of errors. The path from thedesign stage to the finalimplementation of the automatedsystem becomes simpler, faster, morepredictable and economical.

EDA tools verify hardware designs(CHECK-OFF)Using formal methods, the CheckOfffamily of tools provides levels ofverification of digital hardwaredesigns which would be impossibleusing traditional simulation basedtechniques.

SPECTRUM shows feasibility of theintegration of two industriallyrelevant formal methods (VDM andB) (VDM+B)VDM and B are two mature formalmethods currently in use by industryand supported by commercial tools.Though the methods are basicallysimilar, the coverage of theirsupporting tools differ significantly.The SPECTRUM project has shownthe feasibility of integrating supportfor the two methodologies.

The right tools for design anddevelopment (TOOLS)By using a single common designframework that integrates existingand new system design tools,designers will be able to reducedesign cycles and time-to-marketdrastically, for complex embeddedsystems.

Toolset for the development of SafetyCritical Real-time Embedded Systems(SACRES)The results of the SACRES projectare a set of tools and the supportingmethodologies for designers ofembedded systems. The emphasiswithin the project is on formal

Page 13: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 13 of 71

Version 1.0 September 2000

development of systems, providingformal specification, model checkingtechnology and validated code-generation.

The search also highlighted a number ofawareness activities that have a formalmethods component: the European Networkof Clubs for Reliability and Safety ofSoftware Industrial Seminars on FormalMethods, and Information Resources onFormal Methods.

The “success” stories and the focus of thelater Esprit projects highlight the themes of:

• domain specificity (safetycritical, PLCs)

• the transition of model checkinginto the software verification area

• use of B and VDM

• hardware verification

However, overall the investment in formalmethods R&D, especially in the earlierprojects, has not led to any significantindustrial take up of formal methods. In thedescription of the LACOS (Large-ScaleCorrect Systems Using Formal Methods)project it is claimed that “Generallyspeaking, industry still needs evidence thatformal methods can be used in largeapplications in practice. The aim of theLACOS project is to establish anddemonstrate formal methods, specificallyRAISE, as a viable industrial technique inthe scalable production of large, correct ITsystems”. This does not appear to havebeen achieved. It would also appear that thekey tools that seem to be being used havenot come from partly funded industrialcollaborative projects but from basicresearch or industrial investment.

In order to improve the exploitation of laterEsprit projects, projects were required todevelop exploitation and business plans to

demonstrate a return on their investment.Interestingly the guidance on these projectsdid not reference the work of Moore[Moore1] or provide a specific model forthe diffusion of high technology products.All too often, we suspect, companies wouldseek to capture a small percentage of a largemarket with incremental increases in salesor services without any real credibility thatthey would achieve this.

3.1.3 NASA and US Programmes

The U.S. National Aeronautics and SpaceAdministration’s (NASA) past programmeson formal methods have been well regardedand produced substantial technical results.For example, see [NASA], whichsummarises the NASA Langley ResearchCenter research programme. NASA Ameshas also been involved with formal methodsR&D (as in their use of model checking todiscover errors in the Deep Space I probesoftware). NASA’s positive technicalresults with formal methods have only beenmildly reflected in technology transfer.There has been some limited uptake inplaces such as Rockwell-Collins and UnionSwitch and Signal, but there is little doubtthat within NASA, formal methods are stillvery much in a missionary market. Thegeneral approach at NASA appears to be

1. building up a corpus of formal methodssuccesses

2. participating in various RTCAstandards committees with the aim ofincluding formal methods in relevantstandards (usually as a complementarytechnology

3. encouraging teaming between formalmethods experts and researchers withindustrial players

NASA’s strategy is long term and hasfunded or undertaken a wide variety ofwork. NASA Langley has been a significantfunder of SRI’s PVS system and hassupported the development of thefundamental concepts that led to abstract

Page 14: Formal Methods Diffusion

Page 14 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

datatypes, a tabular notation, and the formalsemantics of the language. NASA Langleyalso funded the creation of a PVS validationsuite. Other tools are also used in theprogramme for such things as hardwareverification and the reverse engineering ofAda programs.

Other programs, depending uponperspective, have been less satisfactory.The discussion below on ComputationalLogic Incorporated (CLI) is indicative of aprogramme where there were successfulresults on a technical level, but limited(perhaps negative) results with regards tomission orientation.

For some programmes, the sponsoringorganisations had a vague sense as to whyinvestment into formal methods could be ofimportance to them, but the organisationsdid not clearly enunciate their interests interms of mission orientation, diffusedresearch responsibilities through variousdepartments, and often used relativenewcomers to monitor professional calibreR&D groups. Without the technicalknowledge and mission orientation, it wasnext to impossible to usefully directsponsored R&D groups and, consequently,the R&D groups became primarilytechnically focused with little considerationof technology transfer and market realities.

3.1.4 R&D funding

World-wide, governments have beeninvesting in technology development andtransfer. While we would not wish to paintourselves as experts in the variousinternational investment strategies, loosely,one can identify two types of investment:Programme Driven and Company Driven.By Programme Driven, we mean thatsponsoring organisations define the primarythemes and research directions. Thedefinition may be loose by defining ageneral area of interest (e.g., CriticalInfrastructure) to quite specific (e.g., applyModel Checking to AuthenticationProtocols). By Company Driven, we mean

that the primary R&D agenda is providedby commercial entities with Governmentsponsorship through various tax easementsor proportionate investments/subsidies.Programme Driven often requirescollaboration between organisations, atleast in Europe.

It should be noted that there is a grey areain which the distinction betweenProgramme Driven and Company Driven isblurred as most programmes such as Espritdefine themes within which a particularconsortium has to convince the funders thatthey have a commercial, yet pre-competitive, interest in the work.

Some example programmes are:

• Industrial Research AssistanceProgramme (IRAP) [Canada].This programme providesproportionate investment/subsidyfor pre-commercialisationdemonstrations of innovativetechnologies. However, IRAPrestricts the areas of interest toenvironmental technologies,enabling technologies (inadvanced manufacturing,biotechnology, informationtechnology and advancedmaterials), and Aerospace andDefence. [Niche market driven.]

• Scientific Research andExperimental Development(SR&ED) [Canada]. Dependingupon whether the commercialentity is a Canadian controlled orforeign controlled entity, andupon the size of the entity’srevenues, the Government willeither provide proportionaterefundable tax credits (i.e.,cheques) or non-refundable taxcredits (credits against taxowing). The programme providestax incentives based on scientificand technical research defined bythe corporation. However, the

Page 15: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 15 of 71

Version 1.0 September 2000

company must identify theresearch objectives, the likelyadvancements, the scientific andtechnological uncertainty andsummarise the work performed.Revenue Canada makes use oftechnical experts to reviewsubmissions so as to determinewhether the purported research isactual research. [Market driven.]

• Small Business InnovationResearch (SBIR) [U.S.] andSmall Business TechnologyTransfer (STTR) [U.S.] Thesetwo programmes are used toharness innovative talents ofsmall U.S. technologycompanies. SBIRs fund early-stage R&D whereas STTRs fundco-operative industry/researchorganisation R&D. The secondprogramme is primarily atechnology transfer vehicle.Funding is phased, with a firstphase focusing on testing thescientific, technical andcommercial merit and feasibilityof a particular concept. If the firstphase proves successful, then acompany can apply for secondphase funding that furtherdevelops the concept to aprototype stage. SBIRs arecompetitive. [Programme driven;niche marketing]

• ESSI This European programmeprovides funding for softwareprocess improvementexperiments as defined by theneeds of the user company

• NASA Research Announcements(NRAs) [U.S.] and Broad AreaAnnouncements (BAAs) [U.S.]Though there may be sometechnical distinctions, we havecombined these two forms ofU.S. government procurements.Both types of announcements

describe generalised researchgoals (perhaps with someindicative research approaches)and solicit proposals to achievethe goals. For example, therecent NASA Langley NRA forresearch, development,prototyping and implementationof flight critical systems designand validation technologiesincludes proposals related todesign correctness andcertification, fault-tolerantintegrated modular avionics andoperational malfunctionmitigation. Formal methodsR&D is to be supported underthis NRA. [Programme driven].

It is not within our purview or expertise topass judgement on these programmes, but afew observations can be made. A number ofthese programmes can be used to acceleratetechnology adoption and to provide fundingleverage for achieving prototypedevelopment and experimentation. IRAP,SBIRs, STTRs all provide potentiallyuseful commercialisation channels if thesupported market niche is one in which truecommercialisation opportunities exist.However, the success or failure of suchefforts is not only dependent upon theactual concept being developed, but alsoupon the commercial acumen and interestsof the commercial company. There arecompanies that seem to exist on SBIRs andare unable to actually move anything to atrue commercial opportunity. There maywell be cultural forces that make it difficultfor a company with the R&D culture thatseeks to win SBIRS and Esprit projects tobreak out and commercialise wider thetechnologies being developed.

3.2 The formal methodslandscape - the conferencecircuit

The formal methods page of the WorldWide Web Virtual Library (FMVL) has areasonably comprehensive listing of

Page 16: Formal Methods Diffusion

Page 16 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

notations, methods, tools, publications andindividuals involved with formal methods.There is, as one would suspect, somethingof an academic bias to the overview, but itis still a good starting point.

As of late August 1999, there is a listing ofeighty notations, methods and tools inFMVL. (Collectively, we will call thenotations, methods and tools “formalartefacts.”) Geographically, the substantialmajority of the formal artefacts are fromeither Europe or North America. Many ofthe formal artefacts have their roots ineither academic organisations or corporateR&D groups. Few of the formal artefactshave had much impact on industry. Themain underlying theme of the formalartefacts is an underlying view, by theirdevelopers, of the importance ofmathematics to the development ofcomputer systems.

In 1989, a survey of formal methods toolswas taken of the attendees of a workshopon formal methods for trustworthy systems(FM’89) [FM89]. While biased towardswork performed in Canada, the UnitedKingdom and the United States, it provideda good indication of where significant R&Dresources were being spent (especially bythose interested in computer security). Ofall the systems mentioned, few appear tohave survived the 1990s or been usedextensively outside the core tooldevelopment group(s). Those that haveappear to be Asspegique, NQTHM (or itssuccessor ACL2), EHDM (as manifested byits successor PVS), EVES (primarily asmanifested by its successor Z/EVES), HOL,Larch, OBJ, SPARK and CADiZ.

What is particularly striking about FM’89 isthe lack of any real discussion of modelchecking (which is also true of theinternational survey [Survey] discussedbelow), arguably the most successfulcommercial application of formal methodsto date. Tools and techniques fordeveloping and analysing critical systemswere a predominant theme of the workshop.

The recent Formal Methods in Computer-Aided Design (FMCAD) conferences[FMCAD96, FMCAD98] are particularlynoteworthy for their mix of industry andacademic participants and the realexcitement (especially in 1996) that formalverification techniques (primarily asembodied by various model checkingtechnologies) were being seriously adoptedby major hardware vendors. Conferencesponsorship by companies such as Cadence,Hewlett-Packard, Synopsys and Intel weresuggestive of corporate interests. So toowas the presence of corporate headhunterslooking for scarce formal verificationtalent. (Apparently, there is an ongoingshortage of talent well-versed in hardwaredesign and formal analysis.) Much like theCAV (Computer Aided Verification)conferences, FMCAD is a good mix ofacademic research motivated by realisticindustrial concerns. However, one doeshave the impression that the complexity oftoday’s chips is moving faster than theanalysis technologies used to validate chipbehaviour. This is also true for softwarewith each generation being larger than thatbefore, as exemplified by the purported 60million lines of code estimated forWindows 2000.

A significant amount of research work onBinary Decision Diagrams (BDDs), theiroptimisation and applications has beenreported at FMCAD. Theorem proving,however, was not absent as indicated bypapers on the application of PVS andACL2. There were also research papersdiscussing how an industry standardhardware description language (VHDL)could be used formally.

Industrial interest in formal “software”verification was further demonstratedthrough a recent Intel Research Symposiumheld in Santa Clara [Intel98]. At thissymposium a collection of leading expertsin formal methods (John Rushby, DanCraigen, Ed Clarke, Gunnar Stalmarck,Michael Norrish, J Moore and DavidStringer-Calvert) were invited to give

Page 17: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 17 of 71

Version 1.0 September 2000

presentations relating to softwareverification. Interestingly, it was thecomplexity issue that is a prime motivationfor Intel’s (and we suspect others’) interestsin new technologies. Recent hirings byMicrosoft Research are also suggestive thatthe increasing complexity of the softwareunderlying their products requires newdevelopment and verification technologies,and Microsoft has recently started a jointproject with Oxford University PRG.

As discussed in [CAV99], the Computer-Aided Verification conferences arededicated to the advancement of the theoryand practice of computer-assisted formalanalysis methods for software and hardwaresystems. The conference covers a spectrumfrom theoretical results to concreteapplications. Contributions have come fromboth academia and industry and fromresearchers and practitioners. According toone of the conference series founders (EdClarke, CMU), the principal CAV ethic isone of solving real problems. The idealCAV paper would consists of theoreticaladvances plus supporting experimentaldata.

Presentation topic areas were processorverification, protocol verification andtesting, infinite state space, theory ofverification, linear temporal logic,modelling of systems, symbolic model-checking, theorem proving, automata-theoretic methods, abstraction, and toolpresentations.

The World Congress on Formal Methods(FM99) [FM99] was probably the largestformal methods conference held to datewith over 700 attendees. The conferenceconsisted of a technical symposiumfeaturing 92 scientific papers, a toolsexhibition featuring commercial andexperimental tools, twelve user groupmeetings and workshops (Abstract StateMachines, B, CoFI (Common FrameworkInitiative for Algebraic Specification and

Development of Software), Coq, Larch,Obj/CafeOBJ/Maude, ProCos (ProvablyCorrect Systems), PVS, SPIN, TLA+(Temporal Logic of Actions), VDM and theZ User Group Meeting), and twelveindustrial tutorials (Avionics, B, Embeddedsafety critical systems development assupported by the ESPRESS approach,formal methods and human computerinteraction, Petri Nets, PVS, RailwaySystems, Requirements elicitation andspecification, security, telecommunications,TLA+, and testing). As summarised by theFM’99 general chair (Dines Bjorner), thetechnical symposium consisted of

• 20 papers on softwareengineering issues

• 19 papers on theoretical aspectsof formal methods

• 22 papers on the applicationareas of avionics, safety, securityand telecommunications

• 31 papers on tools, notablymodel checking and notationsincluding ASM, B, CSP-OZ,Esterel, Larch,OBJ/CafeOBJ/Maude,Statecharts, VSPEC and Z

Though it is true that the tools exhibitionwas advertised to have a mix of commercialand academic tools, the tools arepredominantly from academic institutions.Interestingly, most of the commercialcompanies involved in the tools exhibitionare small in size (e.g., B-Core, FormalSystems Ltd., Prover Technology andIFAD).

Summarising the associated user groupmeetings and workshops:

• ASM, focusing on the practicalaspects of the Abstract StateMachine method. (Note thatMicrosoft Research has recently

Page 18: Formal Methods Diffusion

Page 18 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

hired one of the main ASMresearchers: Yuri Gurevich).

• The B-Method, focusing on toolsand applying B in an industrialcontext.

• CoFI: Primarily an introductionto CoFI, focusing on the featuresof the CASL language,methodology and tools.

• Coq: Recent developments withthe Coq proof environment,exchange of experiences, andgeneral discussions.

• Larch: Focusing on discussion ofinterface specification languages.

• OBJ/CafeOBJ/Maude: Generalpresentations on various topicspertaining to OBJ, CafeOBJ andMaude.

• ProCoS: Basically a discussionof the results (including theDuration Calculus) arising fromthe ESPRIT funded ProCoSproject.

• PVS: Recent uses of PVS andplanned enhancements.

• 6th SPIN InternationalWorkshop: Focusing on thepractical aspects of usingautomata-based model checkingin the systems engineeringprocess.

• TLA+ (Temporal Logic ofActions): Discussion of theapplication of TLA+ and varioustools, e.g VSE-II).

• VDM: Language andmethodology issues motivated byindustrial experiences and recent

industrial applications.

• Z: Educational issues, Z/EVESand an update on the proposedISO standard.

The two SPIN Workshops [SPIN99]provide a sense of the active work ongoingin model checking. The 5th InternationalSPIN Workshop on “Theoretical” aspectsof model checking was held in July 1999 aspart of the FLoC 1999 conference in Italy.The 6th International Workshop on“practical” aspects of model checking washeld in September 1999 as part of the FM99conference (see above).

The 5th International Workshop looked atsuch issues as runtime efficient statecompaction in SPIN; distributed-memorymodel checking; partial order reduction inthe presence of rendezvouscommunications, adding active objects toSPIN; model checking of manifoldapplications; and divide, abstract andmodel-check. An overview of a new releaseof SPIN was also presented. The 6thInternational Workshop had presentationson model checking for managers, integratedvalidation management for XSPIN,analysing mode confusion, analysingfeature interactions, the JAVA PathFinder,a visual interface for Promela. It includedan invited presentation of the work atLucent on applying model checking to thedevelopment of commercial systemswritten in C.

The International Workshop on CurrentTrends in Applied Formal Methods[FMTrends] aimed at focusing on keytechnologies that might broaden theapplication of formal methods in industry.In addition there were discussions ondeveloping a “roadmap for formalmethods.” From an assessment of thecurrent status of the technology, the aimwas “to identify intermediate goals in orderto obtain industrial strength formal methodsand a routine application in the softwareengineering process.” Both industrial and

Page 19: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 19 of 71

Version 1.0 September 2000

academic representatives gavepresentations. The invited talks coveredsubstantial territory with Wolfram Buttner’stalk being particularly noteworthy for itsfocus on formal methods adoption (mainlymodel checking and equivalence checking)at Siemens.

Verification tools from North America andEurope were demonstrated at the tools fair(including Uniform, Z/EVES, FDR2, VSE-II and the IFAD Toolset).

The TPHOL (Theorem Proving in HigherOrder Logics) [TPHOL] [TPHOL99]conference series focuses on all reasoningtools for higher order logics. Theconference series evolved from the informalHOL users’ meetings. According to theTPHOL web page [TPHOL] the maintheorem proving systems supporting higherorder logics that have formed the subjectmatter of the conferences are Coq, HOL,Imps, Isabelle, Lambda, Lego, Nuprl,ProofPower, PVS and TPS. Pointers to allthese systems are available at [TPHOL].Though the affiliations of the authors arenot listed (on the TPHOL website), itappears that only two papers have authorswith industrial backgrounds (with Intel):Harrison’s paper on “A Machine-CheckedTheory of Floating Point Arithmetic,” andCarl-Johan Seger’s co-authoring of “Lifted-FL: A Pragmatic Implementation ofCombined Model Checking and TheoremProving.”

One of the significant notations of formalmethods is Z. Z has been broadlydisseminated and supported by numerousbooks and tools. It is used in education,research and industry. A series ofconferences has been held on Z, with the11th International Conference of Z Users[ZUM98] held in Berlin, Germany, inSeptember 1998. In general, [ZUM98]primarily consisted of presentations from Zresearchers, with only a limited applicationand industrial perspective. Topic areasincluded concurrency, tools, Z and HOL,safety-critical and real-time systems,

semantic theory, theory and standards,reasoning and consistency issues,refinement and object orientation. The nextconference will expand from a purely Zperspective to include presentations relatingto the B tool.

3.3 Key industrial areas andapplications

3.3.1 Introduction

In this section we review the application offormal methods in three key industrialareas: safety related systems, securitycritical systems and hardware development.The review does not attempt to beexhaustive but does hope to cover the majorapplications with an emphasis on realprojects.

3.3.2 Safety related systems

In this section we review some of theapplications of formal methods to safetyrelated systems. We consider nuclear,railways, defence, aerospace and avionicsapplications.

One should perhaps note the scale of safetyrelated systems. The French SACEM traincontrol system has about 20,000 lines ofcode, the primary protection system (PPS)for the Sizewell B nuclear reactor has100,000 lines of code (plus about the sameamount of configuration data). Whilechallenging for formal methods they aremodest in size when compared withthe10M lines of code in a typical telephoneswitch.

Nuclear industry

The software-based safety system wereview is the primary protection system(PPS) of the Sizewell B nuclear reactor, asystem which aroused widespread interestin the UK scientific and engineeringcommunities, as well as in the world atlarge. The independent analysis and testing

Page 20: Formal Methods Diffusion

Page 20 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

of the software involved a “searching andcomplete” examination of the software andrequired that any errors found should beinsignificant. For Sizewell B this involvedan extensive retrospective application ofstatic analysis using Malpas of the PLMsource code, involving about 200 personyears of effort [Ward93], as well as acomparison of the object and source code[Pavey]. While the sponsors of this workare careful not to present it as “formalverification” it comes within our broaddefinition of formal methods. There wasalso some inspired unofficial analysis ofone of the protocols used [Anderson].While the effort to do the initial analysiswas considerable, the costs of maintainingand modifying the analysis in themaintenance phase is reported to be veryfavourable when compared to the costs ofretesting. This is due to the modularity ofthe analytical evidence and the cost of theelapsed time required for testing.

In Canada there has been considerablefocus on the issue of safety critical softwarefollowing the licensing delays arising fromthe assessment of the Darlington shut downsystems. In 1987 the Atomic EnergyControl Board (AECB) identified theshutdown system software as a potentiallicensing impediment and it concluded thatit could not be demonstrated that thesoftware met its requirements. Despite anintense and largely manual effort devoted todemonstrating that the code satisfied itstable based specifications the system wasjudged to be unmaintainable and thesystems are being reimplemented. To dothis, considerable effort has been expendedin developing a coherent set of softwareengineering standards and using these intrial applications. These standards set clearprinciples for the development of safetycritical software based on:

• good software engineeringprocess

• statistically valid testing

• mathematically formalverification

One of the motivations for moving towardsmathematical methods is that they provide amore rigorous notation for review[Joannou90]. More recently Ontario Hydrohave developed PVS theorem provingsupport for this work (winning an internalprize) and the table based approach hasdiffused into other areas.

Railway applications

Although in many railway projects thesoftware is developed to good industrypractice, using structured design andlimited use of static analysis for control anddata flow analysis, there is a growing bodyof examples of the use of formal methods.For example, GEC-Alsthom in France hasmade extensive use of formal methods andmachine assisted mathematical verification.The use of formal methods originated in1988 and since then has seen an increasingscope in their use, greater machine supportachieved by the development of the BMethod and the Atelier B toolset, and morerecently the integration of the formalmethods with the requirements engineeringprocess. Formal methods have been appliedto a number of train protection products(e.g. those used on the Cairo and CalcuttaMetros and recently to the new Paris metroline [Behm99]). The costs of applyingformal methods have decreased by abouttwo orders of magnitude from the earlyprojects reported in [Survey] and from 0.5to 2 lines per hour using B today[Chapront99]. The development of the Bprocess and tools has been supported by theFrench Railways and RATP. Interestingly,the regulators require the delivery of theproof file from Atelier B. Also the use ofAtelier B obviates the need for moduletesting. This use of machine-checkedformal methods for verification is some ofthe most advanced in the world in terms ofrigour and scale of application.

There have also been examples whereformal methods have been applied to the

Page 21: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 21 of 71

Version 1.0 September 2000

modelling and validation of signallingrequirements and specifications. Forexample special purpose theorem provershave been used in Sweden to modelspecifications [Erikson96], and someprotocols have been analysed in detail withthe HOL theorem prover as part ofacademic research [Morley93]. NetworkSouthEast also commissioned a largeformal specification (~300 pages of Z) of asignalling control system. The ability tomodel the railway signalling philosophyand demonstrate with high confidence thatchanges to it are consistent and safe may bean important tool in facing the futurechallenges of new and hybrid controlregimes. There are examples of modelchecking using SPIN [Cimatti98] andProver [Stalmark]. There are also somesignificant applications of RAISE asreported in [Haxthausen99]. Developmentsin the nuclear industry in France follow adifferent paradigm. The tool SAGA, basedon the language LUSTRE, is an industrialtool used in France by Merlin-Gerin. Itcarries out the familiar type checkingactivity but also carries out the automaticcode generation.

Defence (UK)

In the UK the Ministry of Defence standardDef Stan 00-55 sets out requirements for aformal methods based approach. Theapplication of this standard has beenprogressive. One of the first applications isreported in [King99]. This system checkswhether it is safe for helicopters to take offor land on a naval ship. Praxis CriticalSystems is responsible for developing theapplication software and this includesformal specification, implementation inSPARK Ada, flow analysis and proof.Praxis Critical Systems has also workedwith Lockheed on the verification of thesafety critical software used in the HerculesC130J aircraft. This includesimplementation in SPARK, flow analysisand proof [Croxford96]. This project wasoriginally seen as a retrospective additionof assurance but the techniques were found

to be so effective that Lockheed introducedthem into the forward development path.

While the UK MoD sponsored thedevelopment of early static analysis toolsmore recent policy, at least from theDefence Evaluation and Research Agency(DERA), has been to concentrate on the useof CSP and the FDR toolset as theirstrategic technology. (Note that FDR2evolved from interaction with missioncritical groups within the defence industry.)There is also work continuing on extendingstatic analysis to deal with pointers.

Avionics

The avionics industry approach isrepresented by the requirements ofDO178B: formal methods are not mandatedin DO-178B, but there are static analysisrequirements. On the commercial sideGEC-Marconi Avionics are convinced ofthe benefits of both static and dynamicanalysis. They applied formal methods tocertain critical algorithms in the Boeing 777Primary Flight Computer. Formalspecification was beneficial in some of thesimpler algorithms, and problems wereidentified and resolved. However, formalproof was less beneficial in the samealgorithms and the task was labourintensive of the highest grade of effort.Attempts to apply formal methods to morecomplex algorithms also required high-grade effort and produced no useful result(as reported in [SOCS98]). Their generalconclusion based on this experience wasthat formal methods have limitedapplicability. However there are strongeconomic drivers from the avionics industryto reduce the test effort required fromDO178B and this is driving both R&D andalso the revisions to DO178B.

Moreover work on other flight criticalaspects has been sponsored by NASA andis described in [Butler98, Owre99].Rushby’s work on formal methods andassurance is widely read in the safetycritical community [Rushby93]. Theparallel made between proof and

Page 22: Formal Methods Diffusion

Page 22 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

calculation is an important insightunderwriting the need for capable tools.The development of PVS and its associatedapplication in requirements, hardwareverification and fault tolerance is asignificant step in the maturing of formalmethods: over 1200 sites have installedPVS. However most of the work reported iseither research or initial industrialapplications. It has yet to be seen whetherthe use of PVS will take hold after thesefirst projects.

Earlier work in the aerospace sector [SIFT]provides an example of over claiming, or atleast a lesson in the care needed whenpresenting the results of this type of work.An independent investigation into theproject provided some detailed criticismand later work showed flaws in the proofsthat were undertaken.

Other applications

There is also considerable work in Europeon the development of drive by wireautomotive systems. These are leading tothe use of deterministic time triggeredarchitectures with formally verifiedhardware and protocols.

The automotive industry has also sponsoredthe development of a verified compiler forPLCs [DaimlerBenz]. There is also relatedwork in the defence industry [Stepney98].

There are also emerging new applicationsof model checking to operator proceduresin aerospace, air traffic control and nuclearapplications [Lüttgen 99, Zhang99].Work is being carried in both air trafficcontrol and nuclear plant control in whichthe complex procedures that have to befollowed by the operator are formallymodelled and desirable properties proved.

While the process industry does not appearto innovative in its use of formal methodsthere is an example of a small verifiedhardware design, motivated by thedifficulties in exhaustive testing. Theapplication of model checking to PLC

programs also seems a promising area andone that has been picked up by theSACRES project led by Siemens.

There is also perhaps the world’s onlyshrink-wrapped formally specified product,DUST-EXPERT , an advisory system onavoiding dust explosions [Froome99]. Thiswas developed by Adelard and involved a20k line VDM specification. The productwas sponsored and approved by the UKHealth and Safety Executive.

3.3.3 Security applications

Since the 1970s there has been significantinvestment into formal methods by agenciesconcerned with national security.Investment by these agencies has helped toaccelerate certain aspects of formalmethods, but has also retarded evolutionthrough, for example, restrictions on thedissemination of research results and on thedistribution of leading-edge formal methodssystems. As of the late 90s, many of theserestrictions have been significantly eased.

In North America, early security-relatedformal methods research was aimed at tool-supported processes, with definite emphasison relating formal security models, throughtop level specifications, to code. Systemssuch as the Gypsy VerificationEnvironment, HDM, Ina Jo, and m-EVESall aimed at handling the above, withvarying degrees of success. Early researchemphasised processes and tools that led to"code verification." Some of these systems(or their ilk) were used, again with varyingdegrees of success, on security products ofsome importance to national governments.Deployed security-related systems exist thathave benefited from these tool-supportedanalyses.

Assurance concerns about security-relatedproducts were not only manifested by R&Din formal methods, security modelling, etc.,but also in significant initiatives to produceeffective information technology securitycriteria and to assess commercial security

Page 23: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 23 of 71

Version 1.0 September 2000

products against these criteria. In the U.S.,this led to early work on the TrustedComputer System Evaluation Criteria(TCSEC) and the Federal Criteria forInformation Technology Security. Similarcriteria (ITSEC) were also developed in anumber of other countries, includingCanada, Germany, the Netherlands and theUnited Kingdom. Recognising that theburgeoning number of national criteriawould impede the acceptance of securityproducts in other countries because of thenumerous evaluations that would berequired, a number of countries (inparticular, Canada, France, Germany, theNetherlands, the U.K. and the U.S.)harmonised their criteria into aninternationally, standards-based "CommonCriteria."

Though at the time of writing this report itappears that the number of products thathave been evaluated against the CommonCriteria is small, one should notunderestimate the importance somegovernments are placing on the CommonCriteria and, in general, on evaluation. Forexample, the formal signing of theinternational agreement, in October 1998,was given rather extensive publicity.Furthermore, in the U.S., NSA and NISTjointly formed the "National InformationAssurance Partnership," (NIAP) which is aninitiative designed to meet the securitytesting needs of both informationtechnology producers and users. Asappears to be the case in other signatorycountries, the U.S. is privatising theevaluation process by setting upcommercial testing laboratories.

Though specific to the U.S., the goals of theNIAP partnership (extracted fromniap.nist.gov/howabout.html) appears togeneralise to other signatory countries:

• Promote the development anduse of security-enhanced ITproducts and systems.

• Demonstrate and increase thevalue of independent testing andcertification as a measure ofsecurity and trust in informationtechnology.

• Foster research and developmentto advance the state-of-the-art insecurity test methods and metrics.

• Move current government-conducted evaluation and testingefforts to accredited, privatesector laboratories.

• Help establish the elements of arobust commercial securitytesting industry.

• Establish the basis forinternational mutual recognitionand acceptance of securityproducts test results.

As of October 1999, NIAP is initiallyfocusing on three initiatives:

• Security Requirements: NIAPservices to aid interested partiesin specifying robust, testablesecurity requirements.

• Security Product Testing: Effortsfocusing on the evaluation ofproducts against specified securityrequirements.

• Security Testing Research andDevelopment.

The dearth of evaluated products appears tobe a result of the preliminary nature of theCommon Criteria and the ongoing processof establishing evaluation laboratories. Forexample, in Canada, there are twoevaluation labs. According to a theCanadian Communications SecurityEstablishment web pages(http://www.cse.dnd.ca/) there are 13

Page 24: Formal Methods Diffusion

Page 24 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

certified products ranging from evaluationlevel EAL1 to EAL4. Note that therequirement for a formal model of aproduct's security policy is required only atlevel EAL5 or above.

From a formal methods perspective, thereappears to have been a change on theapplication of the technology from the70s/80s to the current day. While much ofthe early focus (at least in North America)was on tools and processes that handled theentire development process (from securitymodels to code), much of the current workappears to be at a security modelling level.A review of the literature suggestssubstantial work on cryptographic/securityprotocols and pointwise applicationselsewhere.

As an example of the current work withformal methods on security protocols,consider a July 1999 workshop on thesubject [FMSP99]. Papers andpresentations included:

• A survey of formal analysis ofcryptographic protocols.

• The CAPSL intermediatelanguage, supporting a securityprotocol front-end to FDR.

• Analysis of a library of securityprotocols using Casper and FDR.

• Undecidability of boundedsecurity protocols.

• Towards the formal verificationof ciphers, logical cryptanalysis ofDES.

FM99 [FM99] contained a track on securityand formal methods. Seven papers wereaccepted for publication including:

• Secure interoperation of securedistributed databases.

• A formal security model formicroprocessor hardware.

• Abstraction and testing.

• Formal analysis of a securecommunication channel: Securecore-email protocol.

• A uniform approach for thedefinition of security properties.

• Group principals and theformalisation of anonymity.

In addition to the formal papers, there weretwo tutorials:

• Formal methods for smart cardsbased systems.

• Automated validation andverification of security protocols.

Other conferences of note are the IEEESymposium on Security and Privacy (heldannually in Berkeley, California), the IEEEComputer Security Foundations Workshop(last held in Italy, June 1999), and theNational Information Systems SecurityConference (Arlington, Virginia, October1999) (NISSC). These conferences cover abroad range of topics. For example, NISSCtracks included the assurance, criteria andtesting; electronic commerce; networkingand the internet (with substantial interest inPKI); and R&D.

It appears that most efforts at evaluatingsecurity products are occurring at the EAL4Common Criteria level or below and,consequently, limiting the application offormal methods technology. It also appearsthat there is a much improvedunderstanding within the securitycommunity on how to make use of formalmethods technology in a manner that ismuch more effective than the earlier effortsin the 70s/80s, when formal methods toolswere prematurely applied to significantly

Page 25: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 25 of 71

Version 1.0 September 2000

complex tasks. The use of formal methodsby the security community seems consistentwith the use of the technology by othercommunities: formal modelling is used toimprove understanding of the artefactsbeing developed, while formal analysis isused both to "formally debug" formalmodels and to provide assurance ofconsistency (e.g., between a security policyand a top level specification).

In conclusion, the broad range of formalmethods usage (electronic commerce,internet, security protocols, operatingsystems) by international R&D andcommercial groups, strongly suggestsmaturation. However, the lack of formalmethods in endorsed products also suggeststhe existence of adoption barriers.

3.3.4 Hardware and microcodeverification

There is an enormous amount of work onthe application of formal methods to thedesign and verification of hardwaredesigns. Formal methods appear to be

having a (potentially) major impact in thehardware industry. Some of the largehardware vendors are either developing in-house capabilities in formal verification(e.g., Intel) or are adopting externallydeveloped technologies (e.g., AMD withACL2). The forthcoming Merced chip fromIntel may be the first mass producedartefact to have formally verified parts ofthe hardware and microcode design(verified using HOL) although Intel havealso retrospectively verified the floatingpoint arithmetic of the Pentium Proprocessor [Intel99]. In doing so they havedeveloped a framework that combinesadvanced model-checking technology, user-guided theorem-proving software, anddecision procedures that facilitatearithmetic reasoning.

Formal methods in hardware design areused at a number of levels of abstraction inthe design and manufacture as shown in thediagram below (adapted fromhttp://www.chrysalis.com/products.html).

.

Theoremproving

Figure 1 Levels of hardware verification

Model checking and related automaticdecision procedures are the mostwidespread verification technology used in

Page 26: Formal Methods Diffusion

Page 26 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

hardware verification. A significant amountof research work on Binary DecisionDiagrams (BDDs), their optimisation andapplications has been reported at FMCAD.Theorem proving, however, was not absentas indicated by papers on the application ofPVS and ACL2. There were also researchpapers discussing how an industry standardhardware description language (VHDL)could be used formally.

Wolfram Buttner, Siemens AG, gave anoutline of their work on hardwareverification at the FM-Trends 98conference in Boppard [FMTrends].Equivalence Checking (EC) is used tovalidate VHDL refinements and canroutinely handle up to a million gates (seefor example Chrysalis Symbolic Designwho have "checked the formal equivalenceof nine, extremely large, complex ASICs.Each of these ASICs comprises betweenone and two million gates. The equivalencechecker typically takes between 10 minutesand 3 hours to formally verify each chip".).At least 800 Siemens engineers are requiredto use EC. One attraction of EC is that thereis a substantial ease-of-use factor. ModelChecking (MC) is used to validate circuitsand software (in SDL). In the latter case,MC is used at all levels of the developmentprocess. Buttner noted that MC is still in themissionary selling stages and that there aretechnical issues outstanding. In particular,while MC can handle a few hundred statebits, ASICs often have a few thousand statebits. Buttner concluded that the technologyis difficult and has high adoption barriers.However, Siemens has had encouragingexperiences and feels that the commercialprospects are good.

At higher levels of abstraction there aremany examples of the application oftheorem proving.1 The work at CLINC on

1 Of the 386 papers in the HOL bibliography(http://www.dcs.glasgow.ac.uk/~tfm/hol-bib.html) about 30% relate to hardwareverification.

the FM9001 is notable as part of theirapproach to developing a verified "stack" ofproducts, and the verification of aspects ofthe Viper processor in HOL [Cohn89]resulted in a high profile for the claims forassurance that can be made following aproof of correctness. More recently thework with PVS at SRI and RockwellCollins shows the continuing benefits ofthis form of semantic modelling and proof[Butler98].

3.4 Preliminary conclusions

The application of formal methods has along history but they have not beensubstantially adopted by the softwareengineering community at large. Failure toadopt has arisen for numerous reasons:

• Research that was successful butrequired heroic efforts to getresults (e.g., from lack of theorybase, tools not usable).

• Tools being developed but nottransferred to other groups (e.g.,due to a combination of platform,and people issues).

• Large investments in toolswithout a corresponding advancein theory or method, prematureattempts to increase rigour orincrease scale.

• Results that did not scale fromcase studies: a common problemin the applied area.

• Not meeting customerrequirements, e.g., sponsors’concerns shift but the researchcommunity not listening. Therehave been some notable shifts inemphasis by government agenciesthat in the past were the primesponsors of formal methods.

Page 27: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 27 of 71

Version 1.0 September 2000

• Over ambition and failure tomeet expectations (e.g., fromoverselling by the researchcommunity or from failing tomeet unreasonable expectations ofthe sponsors).

Yet in some respects these could be arguedto be the normal turbulence in any area oftechnological research and deployment. Theoverall perception of the current formalmethods landscape is that it consists of:

• Techniques and tools mostly witha long history of usage and acommunity of use behind them.They have not been the productof relatively short term (3 year)R&D programmes.

• Smallish organisations (primarilyuniversity research groups andsmall commercial companies)with little in the way ofwidespread commercial adoptionof core concepts and tools.

• Generally commercial companiesmaking money from services nottools.

• Stable core groups but no majorbreak out, some fission.

However there is significant take up offormal methods in critical industries. Largehardware vendors are either developing in-house capabilities in formal verification(e.g. Intel, Siemens) or are adoptingexternally developed technologies and inthe area of safety there is active research,case studies and serious application offormal methods throughout the safetylifecycle. The key points to emerge are:

• There are significant applicationsof formal verification in therailway and nuclear industry withvariable degree of rigour. Use ofstatic analyses, interactive

theorem proving and modelchecking with the use of domainspecific solutions.

• There is a need for integration offormal methods with the existingsystem and software engineeringprocess. This has lead to the useof engineer friendly approach tonotations – variants of tablebased notations, existing PLCnotations — tradingexpressiveness with ease ofanalysis.

• There are examples of verifiedcompilers for special purposelanguages and rigorous backtranslation. There is significantwork on verified hardware inflight and drive-criticalapplications.

• The use of model checking andequivalence checking has beenadopted by hardware chipmanufacturers. The forthcomingMerced chip from Intel may bethe first mass produced artefactto have formally verified parts ofthe hardware and microcodedesign.

There are also examples of "over claiming",and the need for careful presentation ofresults – not an easy balance to maintain incommercially and academically competitiveenvironments.

While not particularly scientific, it isinstructive to identify the present limits ofvarious FM-based technologies. Note thatwith judgement in abstraction andmodularization, larger systems could bepiece-wise analysed. Furthermore, evensmall systems may have substantialintellectual depth and challengetechnological limits. With these caveats thepresent limits appear to be:

Page 28: Formal Methods Diffusion

Page 28 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

Equivalence checking of 1 Milliongate ASICS.

Model checking of 1000 latches ata time, using techniques formodularising and looking atcomponents, e.g. there are claims ofverifying large (1020 ) state spaces

Software verification from designto code of ~80Kloc.

Simple formally verified compilersfor special purpose languages.

Static analysis of >150Kloc.

Specification and modelling of>30,000 lines of specification.

Theorem proving is a specialised activity,and there is some evidence of a skillsshortage in hardware verification and prooftechniques. Also despite some evidence thatengineers can read and write formalspecifications, mass uptake will be onderived partially automated solutions andthe use and adaptation of familiar notations(e.g. tables, diagrams).

One should also note that investment informal methods tools needs to be continual:they need to keep pace with the generalindustry rate of change in theimplementation technology. While thesechanges often facilitate new functionalitythey also drive obsolescence. We havealready mentioned hardware and operatingsystems but it also applies to databases,GUI builders, languages (Lisp, Smalltalk).

4 Analysis

4.1 The formal methods market

4.1.1 Characteristics of themarket

As indicated in the preliminary conclusionsabove, the overall perception is that the

formal methods landscape consists of avariety techniques and tools. While thetools often tackle different problem areastheir number would normally be taken for asign of an immature market. However theformal methods market is not really a singlemarket, more a particular technicalviewpoint on a host of disparate activities.The landscape primarily consists ofsmallish organisations (university researchgroups and commercial companies) with nowidespread commercial adoption of coreconcepts and tools. There are a number ofsmall service companies using formalmethods as a leading edge service. They aregenerally technically motivated to grapplewith new tools and put up with being thefirst to use the theories or tools on industrialscale projects.

Presently, the only places where formalmethods appear to be having a (potentially)major impact is with the hardware industryand parts of the safety critical industry.Some of the large hardware vendors areeither developing in-house capabilities informal verification (e.g., Intel) or areadopting externally developed technologies(e.g., AMD with ACL2). There areindications that the telecommunicationsindustry may also be adapting thetechnology. However, the areas of securitycritical systems show very little adoptionalthough of course this work is notparticularly visible even though securitywas originally one of the prime motivationsfor significant R&D funding.

It is also instructive to speculate on theextent of the formal methods activitiesworld-wide. Taking the number ofattendees at FM99 as indicative of 10% ofthose involved in the software relatedactivities, adding a similar number in thehardware fields and calculating the salariesand overheads of those involved we arriveat a figure of $1-2B per annum.

Page 29: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 29 of 71

Version 1.0 September 2000

4.1.2 Economic and other drivers

Both the hardware and telecommunicationsindustries have a compelling reason toadopt the technology. Intel’s Pentium Flaw(FDIV) cost the company hundreds ofmillions of dollars and was a wake up callthat the then existing process fordeveloping increasingly complex chips wasinadequate. With the only option being torecall the chips (unlike the softwareindustry’s general— but not ubiquitous—capability for releasing software patches),new scalable assurance arguments werenecessary. Model checking was a good fit.(Model checking is discussed furtherbelow.)

Both the hardware and telecommunicationindustries are spending substantialresources on simulation and testing toachieve (or approximate) relevant coveragecriteria. On some projects, the cost isnearing 50% of overall project funding.Increasing the efficiency of the analysis(through either increased coverage or moreefficient and cheaper exploration) is acompelling economic argument. This isalso true in the avionics and parts of thesafety critical industry where the directcosts and time of achieving test coverage isdriving the adoption or investigation offormal methods.

There are similar economic drivers in theaerospace industry. Lucas Aerospace reportin their justification for their ESSI projectPCFM that V&V accounts for typically41% of total software production costs andcode corrections for typically 32% of totalmaintenance costs. We have already noted(Section 3.3) that the use of B reduced theneed for module testing in a railwayapplication.

4.1.3 The impact of developmentprocess failure

The Intel Pentium Flaw demonstrated thatthe complexity of contemporary chips wasexceeding the engineering processes

currently in place. The failure of Ariane 5,because of a software fault, indicated afailure in the software architecture andprocess. The failure of an U.S. Titan rocketwas traced to a decimal placement error inits software.

Thomas Kuhn in his book “The Structure ofScientific Revolutions” [Kuhn] defines anddiscusses why paradigm shifts occur. Ineffect, a technical community will buy into(or start developing) a new theory/paradigmif the old theory is refuted or if the newtheory extends scientific prediction inmaterial ways. Often, the shift will occurwhen a significant infelicity with thecurrent belief system is found. The PentiumFlaw was an indicator of such infelicities inthe chip design processes.

The inability of hardware vendors andtelecommunications companies to achievean appropriate scope of validation withcurrent technology may very well beanother crisis point requiring a paradigmshift. Indeed the continual pressure on allcompanies to reduce costs and developmenttimes keeps new technologies underscrutiny. Companies are searching for orderof magnitude improvements in cost and notjust minor polishing of their processes:process improvement runs out of steamwithout a technology shift.

There also appears to be a lack of businessmodels to assess impact of newtechnologies on process. There needs to besupporting, investment-oriented, risk typemodels of the engineering process that willallow the business case for new methods ortechniques to be properly assessed.

4.1.4 The impact of communities

As suggested by the sociology group ofDonald Mackenzie at the University ofEdinburgh, community belief systems playa role in the adoption of new technologies.Hardware developers generally come with astrong engineering ethos. Consequently,they are receptive to new technologies that

Page 30: Formal Methods Diffusion

Page 30 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

are strongly grounded technically anddemonstrate economic potential. Thiscommunity appears to be less accepting ofhigh failure rates than the softwarecommunity.

However, even a solid engineeringcommunity may be slow to adopt. Forexample, the avionics/astrionics communityis highly self-referential (i.e., closed). Evenhigh-cost software-related failures (e.g., atleast one of the recent Titan failures at acost of over $1B) has not resulted in anapparent change in perspective. Over thedecades of aviation they have developedparticular processes with which they arecomfortable. Breaking into such acommunity not only requires a compellingreason to buy/adopt, but also requires thevendors to commit substantial resources toparticipate in the domain and to modifytheir technology accordingly.

Sometimes, the belief system of the R&Dcommunity differs from that of the sponsorsand potential consumers. Perhaps acautionary tale along these lines can be toldwith respect to a well-respected U.S.company Computational LogicIncorporated (CLI). CLI receivedsubstantial funding from various security-related agencies and was well respected forthe work they performed in thedevelopment of NQTHM, ACL2, and theapplication of these tools (especially tohardware) and other FM-related activities.One of the driving tenets at CLI was the so-called “CLI Stack” in which R&D targetedrigorous mathematical development andproof for application programs, throughcompilation down to verified chips. TheCLI vision was certainly comprehensive.However, ultimately, this was not a visionshared by a major customer, as it wasperceived not to meet the customer’smission requirements. Apparently, CLI didnot adapt to the new market reality andR&D funding diminished.

Though some other R&D organisationshave been somewhat more reactive to

market realities, in general terms onewonders whether the use of R&D servicecompanies and university research groupsare an effective means for technologytransfer and industrial adoption. Many ofthe organisations involved in such R&D donot have a true commercial productperspective nor, in fact, are they interestedin developing such a perspective. For someorganisations, providing R&D services istheir main source of revenues and technicalenjoyment.

Instead of investigating means of takingsolid R&D results and adapting them toniche markets (through whole productdevelopment) and then aiming forcommoditisation, the R&D stays in an earlymarket in which only highly giftedindividuals can participate. Consequently, aquestion that sponsors must ask is whetherthey are expecting an R&D group toperform, essentially, pure R&D, or whetherthey are expecting the group to take a moremarket oriented view. Either option isreasonable, but it is crucial that there is aclear statement as to intentions.

We also note that the self-referencing, whotalks to whom defines what we mean by amarket in formal methods: it is not just acase of using the same technology. Ofcourse markets have fuzzy edges but thetechnical communities considered here arequite closed with respect to other sectors.Avionics suppliers look towards othersimilar suppliers and to organisations likeNASA for leadership. Process industryplayers look to key multinationals withsafety records (Shell, Dupont) and processindustry working groups (such as ISASP84). Individuals in these groups do ofcourse keep abreast with the wider pictureand regulatory pressures can lead to somediffusion between sectors (it is an obviousquestion for the regulator to ask whetheryou have considered technique X that isused in sector Y). There are examples ofsome diffusion between sectors: normallyfrom those with development fundingoutwards. The use of the static analysis

Page 31: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 31 of 71

Version 1.0 September 2000

tools and their successors (Spade, Malpasand Spark) have migrated from their originsin the defence sector. Some would alsoclaim that the whole area of formal methodshas migrated out of the needs of nationalsecurity.

Another important aspect of communities isthe separation between the researchcommunities. For example the self-appointed main formal methodsconferences, even as late as 1998, had norepresentation from the model checkers. Itis only with FM99 that these have beenbrought together in some form. Thecommunities of model checkers, theoremprovers, abstract state machines, protocolverification are often too distinct.

Different communities tend to havedifferent research agendas and this can leadto a self-perpetuating separation. Forexample the perception by NASAsponsored work in [Rushby93] that it is thefront end of the lifecycle that is importantinfluenced a large body of work andimportantly the tools. Hence the lack ofcode verification features in PVS. This lackof code verification has meant that thisaspect has been downplayed. Yet there aresignificant economic drivers in theaerospace industry for the replacement ofthe lengthy and costly test coverage that isrequired by the relevant sector standardDO178B. Another example is the railwayindustry and the use of B where theemphasis on code verification has led to aquite different toolset, Atelier B.

In Moore’s terms [Moore1], there is a needto present solutions as a “whole product”for that community, e.g., a new andefficient method for solving propositionallogic might accurately describe thetechnical basis, but as a product it wouldneed to be seen as a “tool for checking thatthe shutdown logic meets its specification”or in the railways “a tool for checking thatinterlock specifications satisfy thesignalling requirements”. It would need tointerface to the tools, notations and

processes in use in these differentapplication areas.

The community viewpoint highlights anumber of important points:

1. Referencing important, differentkey players in different sectors.

2. Perceived needs andopportunities reflect communityvalues.

3. Need to establish credentialswithin a community – entry costscan be expensive.

4. Research agendas can shift byexternal factors.

5. Technology and companies mustbe sensitive to shifts incommunity values. As alwaysthere is a need to listen to thecustomer.

4.1.5 Impact of government andstandardisation bodies

Government and other large organisationscan influence the adoption of technologythrough their impact on the regulatoryprocess, in regulated industries, and throughshaping the market with their significantpurchasing power. The development by theUK Ministry of Defence of a standard DefStan 00-55 caused considerableinternational comment as a major purchaserof safety critical systems set out itsrequirements for a formal methods basedapproach. The standard was updated andissued as a full UK Defence Standard in1996 and is notable for its emphasis onformal methods. One might also note thatthe emerging generic international standardfor safety related systems, IEC 61508,requires the non-use of formal methods tobe justified for the systems in the highersafety categories.

In the security area the agencies in the USAand the UK have had a far reaching

Page 32: Formal Methods Diffusion

Page 32 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

influence on the development of formalmethods. Although they have beensignificant sponsors of formal methodsR&D and applications they have alsodistorted the market through the earlier USembargo on verification technology andtheir prescriptive approach to the methodsand tools that should be used. Thedevelopment of PVS was in part a responseto the potential problems of the technologyembargo. However the development ofnational and security requirements fordifferent techniques for different securityevaluation levels has had a stabilising effecton the market, and there is some evidencethat this is now providing a driver to formalmethods application as more higher levelsecurity evaluations are coming through.This is partly a result of the demands ofelectronic cash and e-commerceapplications.

NASA is currently an important sponsor offormal methods. The plans of Nasa andother future US programs are detailed inAppendix B.

4.2 Technology adoptionmodels

4.2.1 Introduction

In this section we introduce two technologyadoption models. The purpose of presentingthese models, within a formal methodscontext, is to provide a systematic means ofestimating the likely adoption trajectories ofnew formal methods technologies.Consideration of these models may suggesthow various research programmes andprojects could be altered to heighten thelikelihood of success.

We will discuss the technology diffusionmodel of Everett Rogers [Rogers] and ahigh technology model developed byMoore [Moore1, Moore2]. If one is tostimulate the adoption of formal methodsinto the developmental organisations of

critical products (e.g., within electroniccommerce) then one must not only focus ona strong R&D agenda but also on acomprehensive technology adoptionagenda. Our discussion of Rogers andMoore is meant to provide a backdrop forconsiderations relating to adoption. It is ourbelief these models will improve the oddsof adoption; however, if the developedtechnology is inadequate to the problems athand, an ideal adoption plan will still fail.The reader should note that, in effect, weare taking a business perspective onadoption and view successful adoption asbeing business success.

In this report, we can only provide a briefintroduction to the underlying concepts forthe Rogers and Moore models. Readers aredirected to [Rogers, Moore1, Moore2] forin depth discussion. The work of Rogers isquite general and concerned withtechnology diffusion in a number ofcultures (e.g. of health care in SouthAmerica), whereas that of Moore isprimarily focused on mass market adoptionand the associated marketing of hightechnology products. For the Rogers model,we provide two examples of its application.The first, example demonstrates how theRogers model was successfully used inORA Canada's EVES project, resulting insubstantially enhanced adoption of theEVES technology. The second exampleconcerns the Swedish company ProverTechnology. We do not provide examplesof the Moore model. However, [Moore1,Moore2] are replete with high-techexamples of both successful and failedadoptions.

4.2.2 Technology diffusion

In this section we apply the model ofRogers, as modified in [CGR] to gaininsights relevant to formal methodsadoption. The key criteria affectingadoption are:

Relative advantage: An analysis ofthe technical and business

Page 33: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 33 of 71

Version 1.0 September 2000

superiority of the innovation overtechnology it might replace.

Compatibility: An analysis of howwell the innovation meshes withexisting practice.

Complexity: An analysis of howeasy the innovation is to use andunderstand.

Trialability: An analysis of thetype, scope and duration offeasibility experiments and pilotprojects.

Observability: An analysis of howeasily and widely the results andbenefits of the innovation arecommunicated.

Transferability: An analysis of theeconomic, psychological andsociological factors influencingadoption and achieving criticalmass. Principal factors are priortechnology drag (the presence oflarge and mature installed bases forprior technologies), irreversibleinvestments (the cost of investingin the new technology),sponsorship (champions), andexpectations.

Early adopters are also at risk because of“transient incompatibility” (the adopter ismoving ahead of their community/market)and “risks of stranding” (the adopter isstranded because the community/market didnot adopt the technology).

In an earlier international survey ofindustrial applications of formal methods[Survey], Craigen et al., used the abovecriteria to identify likely formal methodsadoption trajectories. In summary, it wasconcluded that formal methods were on alow adoption trajectory. For completeness,one should note that such a trajectory is notinherent to formal methods: enhancedadoption could accrue, for example,through innovations on how to use the

technology effectively, the development ofnew capabilities (e.g., faster processors),and new markets.

As noted in [Cra99], until recently, “...formal methods did not compellingly passthe economic profitability test. However,especially with the use of model checkingin hardware verification, the balance isapparently shifting. Achieving reliabilitylevels through testing solely is running intoproblems with complexity and size. Thelack of a compelling economic argumentoften relegated formal methods to (aperceived ghetto of) security- and safety-critical systems, where the cost of systemfailure is inordinate. But, even here,adoption was minimal.”

The papers [CGR, Cra99] further discussformal methods in terms of the abovecriteria. Considerations along the abovelines had an effect, for example, on thework at ORA Canada in the distribution ofits EVES [EVES] technology. While, atleast from the ORA Canada perspective,EVES had substantial technical merit, forvarious reasons adoption was low (seediscussion in [Cra99]). To enhanceadoption, a strategic decision was made tolink Z and EVES resulting in Z/EVES. Thisdecision has been particularly positive inthat Z/EVES [Z/EVES] has now beendistributed to over forty countries. In termsof the Rogers model, ORA Canada felt itwas well placed for a successful transferinto organisations that were using Z. Forexample:

Relative Advantage: Z/EVES couldprovide substantially more analysisof Z specifications than mostexisting Z tools.

Compatability: Z/EVES processedZ as described by Spivey andaugmented the Spivey LaTeXmarkup language. Most existing Zspecifications could be processedby Z/EVES.

Complexity: Z/EVES supported a

Page 34: Formal Methods Diffusion

Page 34 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

spectrum of analyses, from simpletype checking through to fullautomated deduction. The first setof analyses (e.g., type checking,schema expansion, domainchecking and preconditioncalculation) do not require a fullunderstanding of the underlyingproof engine.

Trialability: The four analysesdescribed above require little efforton the part of the user. One needonly install Z/EVES (reasonablyeasy) and enter the specifications.

We could further discuss this example, butthe general tenor of the adoption analysisshould be clear.

Perhaps a more comprehensive example isprovided by the Swedish company ProverTechnology. This company was founded in1989 and has built its formal methodsbusiness on top of a patented proofprocedure developed by Gunnar Stalmarck[Stalmarck]. As described in [Railways],the company’s formal methods tools “arebased on an efficient theorem-prover forpropositional logic with compilers fromvarious design and specification languagesdown to the level of propositional logic.”Even though the dependency uponpropositional logic constrains the kinds ofproperties that can be analysed, manyindustrial problems fall within thepropositional context. A principal designobjective has been to hide as much aspossible the underlying analytic techniquesfrom the designer.

[Railways] describes how ProverTechnology’s technology has been adaptedfor use by railway signalling engineers. Inparticular, the technology was tailored tothe development of software-basedinterlocking systems from ABB Daimler-Benz Transportation Signal (ADtranz), andincludes:

1. Tool support for formal verificationof safety requirements in the

generic interlocking software usedby ADtranz.

2. Automatic formal modelling ofcomplete interlocking systems in acommercial formal verificationtool.

3. Implementation independent formalspecification of system-level safetyrequirements.

4. Automatic translation ofimplementation independentspecifications into specification ofthe system level safetyrequirements for a specificinterlocking implementation.

Of particular note is that all of the above areintegrated into the ADtranz developmentenvironment and uses their Sternolprogramming language for genericinterlocking systems. Apparently, the use ofthe tool has reduced the ADtranz softwaredevelopment costs by at least 30%: Hence,this provides a compelling reason foradopting the technology. [Railways] goeson to describe the overall process fordeveloping and certifying Sternol-basedrailyard interlockings as follows:

1. A formal verification tool is used inthe development of the genericSternol program in order to provethat the Sternol program satisfies aset of safety requirements, and anequivalence checking feature isused for inspection of newrevisions of Sternol code and alsofor strengthening the confidence inthe Sternol compiler.

2. Formal methods consultants andrailway interlocking expertsproduce an implementation-independent specification of thesafety requirements for arbitraryinterlockings.

3. A tool generates a formal model ofan interlocking implementation,

Page 35: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 35 of 71

Version 1.0 September 2000

based on the Sternol program andthe information in the site file.

4. The implementation-independentspecification is automaticallyinstantiated using the interlockinginformation in the site file,producing the specification for theinterlocking instance.

5. The formal model of theinterlocking implementation isautomatically proved to fulfil itsspecification and the proof islogged onto a file.

6. The proof is checked separately bya proof checker in order tostrengthen the confidence in thecorrectness of the proof.

Step 2 requires expert knowledge in formalmethods and railways. All other steps, froma formal methods perspective, can belargely automated. It is claimed that theextra effort in Step 2 is worthwhile sincethe general specification allows theautomatic generation of specifications foreach specific interlocking instance.ADtranz reports that the above process hasreduced the costs associated with theverification phase by 90%.

So, from a Rogers perspective:

Relative advantage: Cleareconomic justification.

Compatability: Formal methodslinked into the Sternol language.Formal analysis algorithms hidden.

Complexity: A bit with regards tothe use of formal methods in Step2. However, most complexityhidden with regards to automatedanalysis. Results presented in termsof domain expertise (Sternol).

Trialability: Not particularlydiscussed in [Railways], but shouldbe eased through compatibility and

complexity issues. Would appearthat for some basic signallingproperties, should be very easy toexperiment with.

Observability: The economicresults are highly visible.

Transferability: Linked intoinstalled base (Sternol). Apparentlyhas transferred very well.

4.2.3 Developing high technologymarkets

While the Rogers technology adoptionmodel is useful it does not explain failuresto market nor provide a strategy forsuccessful adoption. Geoffrey Moore hastaken the analysis further in two books[Moore1, Moore2], where he discussestechnology adoption of discontinuousinnovations and, in particular, uses themodel to discuss appropriate strategies andtactics for increasing the likelihood ofsuccess. While we do not envisage formalmethods penetrating mass markets eventheir adoption in small niche markets can beinformed by Moore’s work. The additionalperspective that Moore brings is in:

• describing a TechnologyAdoption Life Cycle (TALC) andparticularly the chasm betweentwo stages of this lifecycle

• introducing the importance of acomplete product described inuser terms and aimed at anidentified and characterisedbuyer

• the need to define the market —this depends on communities ofpeople not just on technicalprofile

• the extreme importance of

Page 36: Formal Methods Diffusion

Page 36 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

referencing and word of mouth indeveloping sales

• the problems of selling $10–50kproducts

• the need to segment, segment andsegment again until one is asignificant player in the market

In Moore’s model of the TechnologyAdoption Life Cycle (TALC), he defines anumber of players: Innovators, EarlyAdopters, Early Majority, Late Majorityand Laggards. Understanding these playersis crucial for the successful marketing(technology transfer) of a new product.

Innovators pursue new technology productsaggressively. They are crucial to anymarketing campaign as their endorsementreassures the other marketplace players.Early adopters are not technologists, butindividuals who find it easy to imagine,understand, and appreciate the benefits of anew technology and to relate the benefits totheir concerns. They do not rely on well-established reference points to make theirdecisions, preferring their intuition andvision. The Early Majority shares the earlyadopter’s ability to relate to technology, butare driven by a strong sense of practicality.They want well-established referencesbefore adopting. It is with these folks thatsubstantial profits are available for the first

time. The Late Majority has the sameconcerns as the early majority but areuncomfortable with technology products.They want a well-established standard, lotsof support, and buy from large and well-established companies. Laggards don’twant anything to do with new technology.It has to be buried deep inside anotherproduct.

According to Moore, one can view thedistribution of these players as a BellCurve, with the early and late majority eachconsisting of one-third of the population.However, Moore’s Bell Curve is actuallydiscontinuous, having three gaps of note:between innovators and early adopters;between the early majority and the latemajority; and, most significantly, betweenthe early adopters and the early majority. Itis this last gap that is of particularconsequence and, in Moore’s view, is moreaptly described as a Chasm. Moore is moredetailed in his second book [Moore2] andcharacterises the landscape of the TALCinto six zones:

1. The early market, a time of greatexcitement.

2. The chasm, a time of greatdespair, when the early-market’sinterest wanes but themainstream market is still notcomfortable with the immaturity

Earlymajority

Latemajority

Laggards

Earlyadopters

Innovators

Page 37: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 37 of 71

Version 1.0 September 2000

of the solutions available.

3. A period of niche-based adoptionin advance of the general marketplace, driven by compellingcustomer needs and thewillingness of vendors to craftniche-specific whole products.The bowling alley metaphor isused to describe this phase whereadjacent market sectors are"knocked down" with the samebowl.

4. The tornado, a period of mass-market adoption, when thegeneral marketplace switchesover to the new infrastructureparadigm.

5. Main street, a period of aftermarket development, when thebase infrastructure has beendeployed and the goal now is toflesh out its potential.

6. End of life.

Moore goes on to argue that the basis for asale between early adopters and the earlymajority is quite different. Early adoptersare looking for a change agent while theearly majority wants a productivityimprovement for existing operations.Basically, when promoters of high-techproducts try to make the transition from amarket base made up of visionary earlyadopters to the pragmatist early majority,they are effectively operating without areference base and without a support basewithin a market that is highly referenceoriented and highly support oriented. Ingeneral, it is our view that formal methodsis still in the early market (consisting ofinnovators and early adopters) and only to ahighly limited extent has their been anyeffort to cross the chasm into niche-basedadoption. More generally, Moore notes thatbusiness strategies must dramaticallychange throughout the TALC:

• The forces that operate in the"bowling alley" argue for aspecialised niche-basedmarketing strategy that iscustomer-centric.

• Those in the "tornado" push inthe opposite direction towards amass-market strategy fordeploying a common standardinfrastructure.

• On "Main Street", market forcespush back again toward acustomer-centric approach,focusing on specific adaptationsof this infrastructure for addedvalue through masscustomisation.

Some of the issues that must be faced oncrossing the chasm and aiming for entryinto the niche-based "bowling alley" are:

• Is the target customer wellfunded and are they readilyaccessible to our sales force?

• Do they have a compellingreason to buy?

• Can we today, with the help ofpartners, deliver a whole productto fulfil that reason to buy?

• Is there no entrenchedcompetition that could prevent usfrom getting a fair shot at thisbusiness?

• If we win this segment, can weleverage it to enter additionalsegments?

Perhaps a good example of market focus onformal methods was given by WolframButtner, Siemens AG, at the FM-Trends 98conference in Boppard [FMTrends]. Theenunciated vision was based on the

Page 38: Formal Methods Diffusion

Page 38 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

observation that mathematics has highquality standards and formal verificationwill “prove” requisite properties, whereasstandard quality assurance techniquesprovide “hope” that the requisite propertiesare valid. From his perspective, there werethree analytic options: equivalencechecking, model checking and theoremproving. The predominant success factorsfor Siemens were

• that only fully automatic proofswere cost effective

• that formal verification mustwork within existing designprocesses

• that formal verification mustaddress fully the debugging gap

• that formal verification musthandle the complexities of therelevant application domains

If the first point is given priority it leads toan emphasis on model checking or decisionprocedures rather than theorem proving.But as noted in Section 3.3.4 theoremproving is used in many hardwareverifications.

4.2.4 Adoption of the TALCapproach

Adopting the TALC approach will involve:

• Taking a “whole product” view,that is all the aspects of theproduct that the user may need tomeets his or her goal. Forexample, if the concern is theassurance of systems, the wholeproduct may be dealing withformal based analysis,development history, andoperating experience.Additionally, less technicalissues such as comprehensivedocumentation and free-phone

support are important wholeproduct concerns.

• Characterising the buyer forthese products and analysing thescenarios of use. This may bringto the fore the necessaryinterfaces to systems andprocesses. For example, it mightbring to light quite mundaneissues from an R&D point ofview of platforms. Most inindustry have windows or NTbased systems not Sunworkstations. The costs anddiscontinuity in adoption ofchanging platforms might be toogreat especially for the earlymajority. Visionaries are likely tobe more robust to technicalidiosyncrasies.

• Identifying the competition forthe whole product and the reasonfor buying.

The characterisation of the buyer shouldinclude their place on the TALC (e.g.,visionary or early majority) and the market.Bear in mind that technical similarity ofproblem and solution does not constitute amarket. Community and self-referencingare defining aspects of a market.

Medium term strategies should seek tobring potential users together into acommon market. This might include thestimulation of workshops, standardsactivities and general awareness anddissemination actions.

4.2.5 Success and failure factors

We can summarise the technology diffusionand product marketing discussion into amodel that is shown below. The model hasfive main stages: evidence of past use offormal methods, formal methods are on theagenda for a project, they are selected, usedsuccessfully, and that experience fed intothe evidence for past use. The main classes

Page 39: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 39 of 71

Version 1.0 September 2000

of success and failure factors are shown in the diagram below:

CLAIM: FM on the agenda

CLAIM: Project Risk

Factors addressed

CLAIM: Used successfully on

this project

CLAIM: Supports use on next project

CLAIM: Evidence of

past use

CLAIM: FM Selected on this

project

CLAIM: Tools

CLAIM: People

CLAIM: Process

CLAIM: Compelling business

case

CLAIM: Whole product solution

CLAIM: Market

penetration

CLAIM: Regulatory and legal

drivers

CLAIM: Technology

push

CLAIM: Observability

CLAIM: Transferability

CLAIM: R&D results

CLAIM: Other

projects

supports

supports

supports

supports

supports

supports

supports

supports

supports

supports

supports

supportssupports

supports

supportssupports

supports

supports

If any one of these stages fails then formalmethods will not be adopted or they mightbe adopted on one project but fail todiffuse. From each of these nodesquestions can be generated to assess a

project. The underlying questions willdepend on the stage in the technologyadoption lifecycle of the user. The tablebelow summarises the factors influencingdiffusion.

Page 40: Formal Methods Diffusion

Page 40 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

Table 2: Summary of factors influencing diffusion

Evidence of past use

• Other projects, case studies

• R&D results

• Evaluations by others / ourselves

• How communicated by thetechnical literature, trade press,peers

FM on the agenda

• Regulatory and legal drivers(liability, standards)

• Customer requirement

• Technology push (it exists so try it,problems with existing approach))

• Market penetration (follow theleaders)

• Competitive pressure (reduce costs)

• Liability or costs of failure (recall,liability, consequential damages)

• Process failures and disasters(Arianne )

FM Selected on this project

• Project risk factors addressed

• Whole product solution

• Compelling business case

Used successfully on a project

• Tools

• People

• Process

Supports use on next project

• Transferability

• Observability

• Other barriers to diffusion

Makes the transition to a differentcommunity or market

See Section 4.2.3

The five stages of technology diffusionform a spiral of adoption when we considerthe impact of the TALC of Moore. Not onlyhas the technology to successfully address

these stages of diffusion but it has cross thechasm that separates different communitiesand groups of users. This is illustrated bythe figure below.

Page 41: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 41 of 71

Version 1.0 September 2000

Agenda

Projectselection

Projectsuccess

Transfers tonext project

Evidence ofpast use

Agenda

Projectselection

Projectsuccess

Transfers tonext project

Evidence ofpast use

Agenda

Projectselection

Projectsuccess

Transfers tonext project

Evidence ofpast use

Innovators

Early adopters

Early majority

Technologyand diffusionwithin a stagein the TALC

Chasmbetweenstages in theTALC

TALC phases

Figure 2 Combined TALC and diffusion model

5 Recommendations andconclusions

The objective of this study is to identifycrucial factors leading to the success orfailure of the application of formal methodsand in doing so provide a perspective on thecurrent formal methods landscape. Theoverall aim is to inform future formalmethods dissemination activities and otherinitiatives.

5.1 Past failures

The application of formal methods has along history but they have not beensubstantially adopted by the softwareengineering community at large. Failure toadopt has arisen for numerous reasons:

• Research that was successful butrequired heroic efforts to getresults (e.g., from lack of theorybase, tools not usable).

• Tools being developed but nottransferred to other groups (e.g.,due to a combination of platformand people issues).

• Large investments in toolswithout a corresponding advancein theory or method, prematureattempts to increase rigour orincrease scale.

• Results that did not scale fromcase studies: a common problemin the applied area.

• Not meeting customerrequirements, e.g., sponsors’concerns shift but the researchcommunity not listening. Therehave been some notable shifts inemphasis by government agenciesthat in the past were the primesponsors of formal methods.

• Over ambition and failure tomeet expectations (e.g., fromoverselling by the research

Page 42: Formal Methods Diffusion

Page 42 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

community or from failing tomeet unreasonable expectations ofthe sponsors).

Yet in some respects these could be arguedto be the normal turbulence in any area oftechnological research and deployment.

5.2 The current landscape

The overall perception of the current formalmethods landscape is that it consists of:

• Techniques and tools mostly witha long history of usage and acommunity of use behind them.They have not been the productof relatively short term (3 year)R&D programmes.

• Smallish organisations (primarilyuniversity research groups andsmall commercial companies)with little in the way ofwidespread commercial adoptionof core concepts and tools.

• Generally commercial companiesmaking money from services nottools.

• Stable core groups but no majorbreak out, some fission.

However there is significant take up offormal methods in critical industries. Largehardware vendors are either developing in-house capabilities in formal verification(e.g. Intel, Siemens) or are adoptingexternally developed technologies and inthe area of safety there is active research,case studies and serious application offormal methods throughout the safetylifecycle. The key points to emerge are:

• There are significant applicationsof formal verification in therailway and nuclear industry withvariable degree of rigour. Thereis use of static analyses,

interactive theorem proving andmodel checking with the use ofdomain specific solutions.

• There is some integration offormal methods with the existingsystem and software engineeringprocesses. This has lead to theuse of engineer friendly approachto notations – variants of tablebased notations, existing PLCnotations — tradingexpressiveness with ease ofanalysis.

• There are examples of verifiedcompilers for special purposelanguages and rigorous backtranslation. There is significantwork on verified hardware inflight and drive-criticalapplications.

• The use of model checking andequivalence checking has beenadopted by hardware chipmanufacturers. The forthcomingMerced chip from Intel may bethe first mass produced artefactto have formally verified parts ofthe hardware and microcodedesign.

There are also examples of "over claiming",and the need for careful presentation ofresults – not an easy balance to maintain incommercially and academically competitiveenvironments.

We calculate that about $1-1.5B is spentannually on formal methods activitiesworld-wide.

5.3 Present limits

While not particularly scientific, it isinstructive to identify the present limits ofvarious FM-based technologies. Note thatwith judgement in abstraction andmodularization, larger systems could bepiece-wise analysed. Furthermore, even

Page 43: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 43 of 71

Version 1.0 September 2000

small systems may have substantialintellectual depth and challengetechnological limits. With these caveats thepresent limits appear to be:

Equivalence checking of 1 Milliongate ASICS.

Model checking of 1000 latches ata time, using techniques formodularising and looking atcomponents, e.g. there are claims ofverifying large (1020 ) state spaces

Software verification from designto code of ~80Kloc.

Simple formally verified compilersfor special purpose languages.

Static analysis of >150Kloc.

Specification and modelling of>30,000 lines of specification.

Theorem proving is a specialised activity,and there is some evidence of a skillsshortage in hardware verification and prooftechniques. Also despite some evidence thatengineers can read and write formalspecifications, mass uptake will be onderived partially automated solutions andthe use and adaptation of familiar notations(e.g. tables, diagrams).

One should also note that investment informal methods tools needs to be continual:they need to keep pace with the generalindustry rate of change in theimplementation technology. While thesechanges often facilitate new functionalitythey also drive obsolescence. We havealready mentioned hardware and operatingsystems but it also applies to databases,GUI builders, languages (Lisp, Smalltalk).

5.4 Increasing adoption offormal methods

If the objective is to increase the use offormal methods in industry then this can bedone in broadly two ways:

• increasing the use of formalmethods based services (eitherexternal or as a part of largecompanies) that influencesignificant projects and products(i.e. not just a case study culture)

• increasing the use of formalmethods based tools and productswithin software, hardware andsystems engineering

To achieve this it is necessary to addressthe following.

5.4.1 Apply technology adoptionmodels

Proven technology adoption models havenot been comprehensively applied to formalmethods technologies, by either sponsors ortechnology developers. The haphazardapproach has reduced the adoptiontrajectory of formal methods and haveresulted in unintended adoption barriers.

Therefor a major recommendation is thatunlike other R&D and disseminationprogrammes we have investigated, thestrategy of any future investmentprogramme should adopt models such asthe Moore TALC “chasm crossing”approach and take a more explicit view ofhow the market in high technology productsactually develops. We consider this to bethe single most likely factor to increases thechance of successful adoption. Theadoption of the Moore TALC should takeinto account that formal methods is not amarket as such but really a particulartechnical viewpoint on a host of disparateactivities.

Page 44: Formal Methods Diffusion

Page 44 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

5.4.2 Sustained investment intools

A realistic view is needed of the largeinvestment needed to develop and sustaintools, and the long time it may take for atool and associated experience and theorybase to develop. For specific projects thiswill involve the building on top of existingtools, and the importance and impact offreely available tools is immense (e.g.,HOL, PVS, SMV, SPIN, Z/EVES). Also ofimportance is the extent to which APIs orsource code is available.

However in critical applications there is aneed to trust the authenticity of the tool andits correct operation. There is scope forintentional and unintentional flaws in thetools giving misleading results (e.g. hidinga subtle deadlock, not reporting dead code)and there is a need for innovation to see ifthe benefits of widespread use and scrutinycan be achieved without compromisingtrust.

The formal methods strategy should addresspricing and the commercial viability ofproducts, given the general problems posedby the investment needed to develop tools,the continuing investment needed tomaintain their usability as platforms andoperating system change, and the small sizeof the tools market.

5.4.3 Address differences in targetusers

Most of the formal methods industrial workis done by service groups and smallcompanies. The encouragement of thissector will be quite different from the largermarket. This sector is more akin to Moore’svisionaries, it can quickly apply newtechnical solutions and is not put off by themore intellectually challenging aspects oftheorem proving or the heroism needed tograpple with immature tools and theories.

Increased formal methods uptake amongother engineering disciplines will involve

packing specific analyses into easier to usebut more restricted components: the priceof success is integration [Sifakis99].

There is a need for guidance on how theexisting tools and techniques can worktogether and how potential users shouldselect those to invest in.

5.4.4 Focus on critical applicationareas

The key drivers for adoption come fromapplications in areas with a large impact ofdevelopment failure such as hardware (e.g.,mass-market products, security or e-commerce products), telecommunications,safety and critical infrastructure. There areopportunities for formal methodsthroughout the system lifecycle. Thestrategy should take into account theincreasing use of COTS in criticalapplications.

5.4.5 Continue R&D

A strong R&D agenda is needed tocomplement a comprehensive technologyadoption agenda. To date the funding forformal methods has been a very smallcomponent in the overall IT R&D budget.While there appears a scientific consensusthat the technical future is to combinemodel checking with theorem proving thereis much work still to do.

There should be a clear distinction betweenR&D projects which are being pursued forpure knowledge purposes and those beingpursued to have an impact on industry orgovernmental processes: both arenecessary. In the former case, technologyadoption criterion may be of limitedconcern. The "scientific market" will siftthe importance and relevance of researchresults.

Page 45: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 45 of 71

Version 1.0 September 2000

6 AcknowledgementsNumerous individuals have provided inputinto this report primarily through informalinterviews or discussions. As many of these

individuals provided commentary on abackground basis, we will not specificallyidentify individuals. They know who theyare and we thank them for their input.

7 References

[AFM] Applied Formal Methods, A study into the feasibility, rationale, scope and content of apossible UK strategic programme, University Edinburgh, 1995.

[Anderson] The Formalization and Analysis of a Commmunications ProtocolG. Bruns and S. Anderson, LFCS report ECS-LFCS-91-137. This report was also published inFormal Aspects of Computing, 6:92-112, 1994.

[Behm99] Patrick Behm, Paul Benoit, Alain Faivre, & Jean-Marc Meynadier, Meteor: ASuccessful Application of B in a Large Project, in [FM99].

[Butler98] Ricky W Butler et al, NASA Langley’s Research and Technology Transferprogramme in formal methods, Nov 1998.

[CAV99] Nicolas Halbwachs, Doron Peled (Editors). Computer Aided Verification. Volume1633, Lecture Notes in Computer Science, Springer, 1999

[CGR] Dan Craigen, Susan Gerhart and Ted Ralston. Formal Methods Technology Transfer:Impediments and Innovation. In “Applications of Formal Methods,” M.G. Hinchey and J.P.Bowen (editors). Prentice-Hall International Series in Computer Science, September 1995.

[Chapront99] P Chapront, Alstom: 10 years using B, interview in Lettre B fromhttp://www.atelierb.societe.com/LETTRE_B/index_uk.html.

[Cimatti98] Model checking safety critical software with SPIN: an application to a railwayinterlocking system, in Computer Safety, Reliability and Security, Springer LNCS 1516.

[Clement99] Tim Clement, Ian Cottam, Peter Froome and Claire Jones, "The Development of aCommercial 'Shrink-Wrapped Application' to Safety Integrity Level 2: The DUST-EXPERT(tm) Story". In Computer Safety, Reliability and Security (Proceedings of Safecomp'99), Massimo Felici, Karama Kanoun and Alberto Pasquini (eds), Lecture Notes in ComputerScience 1698, Springer 1999. ISBN 3-540-66488-2.

[Cohn89] Avra Cohn, The Notion of Proof in Hardware Verification, Journal of AutomatedReasoning 5: 127-139, 1989. 1989 Kluwer Academic Publishers.

[Cra99] Dan Craigen. Formal Methods: What’s working; What’s not! Invited presentation forthe 6th International SPIN Workshop on Practical Aspects of Model Checking, in [SPIN99].

[Croxford96] Martin Croxford (Praxis Critical Systems), James Sutton (Lockheed), BreakingThrough the V and V Bottleneck, Paper presented at Ada in Europe 1995. Published bySpringer-Verlag, LNCS Volume 1031, 1996. See alsohttp://www.ddci.dk/programs/high_integ6.html

Page 46: Formal Methods Diffusion

Page 46 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

[DaimlerBenz] G. Egger, A. Fett, P. Peppert, Formal Specification of a Safe PLC Language andits Compiler, Proc. Safecomp94.

[Erikson 1996] L.-H. Erikson. “Specifying railway interlocking requirements for practical use,”in 15th International Conference on Computer Safety, Reliability and Security (Safecomp96),Springer, 1996.

[EVES] Sentot Kromodimoeljo, et al. The EVES System. In Proceedings of the InternationalLecture Series on “Functional Programming, Concurrency, Simulation and AutomatedReasoning (FPCSAR),” Peter Lauer (Editor). Lecture Notes in Computer Science, Volume 693,Springer-Verlag 1993.

[FASE98] Egidio Astesiano (Editor). Fundamental Approaches to Software Engineering.Volume 1382, Lecture Notes in Computer Science, Springer 1998.

[FM89] Dan Craigen (Editor). Formal Methods for Trustworthy Computer Systems (FM89).Workshops in Computing, Springer-Verlag 1990.

[FM99] Proceedings of "FM'99: World Congress on Formal Methods in the Development ofComputing Systems," Toulouse, France, September 1999. Jeannette Wing, Jim Woodcock andJim Davies (Editors).

[FMCAD96] Mandayam Srivas, Albert Camilleri (Editors). Formal Methods in Computer-Aided Design. Volume 1166, Lecture Notes in Computer Science, Springer, 1996.

[FMCAD98] Ganesh Gopalakrishnan, Philip Windley (Editors). Formal Methods in Computer-Aided Design. Volume 1522, Lecture Notes in Computer Science, Springer, 1998.

[FMSP99] Proceedings of the "Formal Methods and Security Protocols Workshop," Trento,Italy, July 1999. Nevin Heintze and Edmund Clarke (Editors).

[FMTrends] Dieter Hutter, et al. (Editors), Applied Formal Methods: FM-Trends 98.International Workshop on Current Trends in Applied Formal Methods, Boppard, Germany,October 7-9 1998. Volume 1641, Lecture Notes in Computer Science, Springer-Verlag, 1999.

[FMVL] http://www.comlab.ox.ac.uk/archive/formal-methods.html

[Haxthausen99] Anne Haxthausen & Jan Peleska, Formal Development and Verification of aDistributed Railway Control System, in [FM99].

[Intel98] Formal Software Verification, Intel Research Symposium, Santa Clara, November1998.

[Intel99] John O'Leary, Xudong Zhao, Rob Gerth, Carl-Johan H. Seger, Formally Verifying IEEECompliance of Floating-Point Hardware, Intel Technology Journal, 1999.

[Joannou90] P K Joannou, et al, "Software engineering and standards for real-time software",in Proc Safecomp 1990, Pergamon

[King99] Steve King, Jonathan Hammond, Rod Chapman, & Andy Pryor “The Value ofVerification: Positive Experience of Industrial Proof”, in [FM99].

Page 47: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 47 of 71

Version 1.0 September 2000

[Kuhn] T Kuhn, “The Structure of Scientific Revolutions”, University of Chicago Press, 1970.

[Lüttgen 99] Gerald Lüttgen and Victor Carreño, Analyzing Mode Confusion via ModelChecking, in Proceedings of the 5th and 6th SPIN Workshops. D. Dams, et al., editors, Volume1680, Lecture Notes in Computer Science, 1999.

[Mol93] A H Molina “The formal methods constituency: diffusion of an emerging technologyinto a high-volume industrial environment,”Research Centre for Social Science, Edinburgh1993.

[Moore1] Geoffrey A Moore. Crossing the Chasm. Harper Business, 1991. See also 2nd edition1999.

[Moore2] Geoffrey Moore. Inside the Tornado: Marketing Strategies from Silicon Valley’sCutting Edge. Harper Business, 1995.

[Morley93] M.J. Morley. “Safety in Railway signalling data: a behavioural analysis,” in HigherOrder Logic Theorem Proving and its Applications, Springer, 1993.

[NASA] Ricky Butler, et al. NASA Langley’s Research and Technology-Transfer Program inFormal Methods. Available from http://shemesh.larc.nasa.gov/fm.html.

[Owre99] S Owre et al, PVS: An Experience Report, in Applied Formal Methods— FM Trends98, LNCS 1641, 1999.

[Pavey] D.J Pavey and L.A Winsborrow, “Formal Demonstration of the equivalence of sourcecode and PROM contents: an industrial example”, Conference on The Mathematics ofDependable Systems, Institute of Mathematics and its Applications, Royal Holloway College,University of London, September 1993.

[PCCIP] “Critical Foundations: Protecting Amerca’s Infrastructures.” The Report of thePresident’s Commission on Critical Infrastructure Protection, October 1997. Available fromhttp://www.pccip.gov.

[PCCIPR&D] “Research and Development: Recommendations for Protecting and AssuringCritical National Infrastructures.” Report of the President’s Commission on CriticalInfrastructure Protection, October 1997. Available from http://www.pccip.gov.

[PDD63a] “Protecting America’s Critical Infrastructures: Presidential Decision Directive 63”Available through http://www.pub.whitehouse.gov.

[PDD63b] “The Clinton Administration’s Policy on Critical Infrastructure Protection:Presidential Decision Directive 63.” Dated May 22, 1998. Available throughhttp://www.pub.whitehouse.gov.

[PITAC99] "Information Technology Research: Investing in Our Future." President'sInformation Technology Advisory Committee, Report to the President, February 1999. Reportavailable at www.ccic.gov/ac/report.

[Railways] Arne Boralv, Gunnar Stalmarck. Prover Technology in Railways. Available atwww.prover.com.

Page 48: Formal Methods Diffusion

Page 48 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

[Rogers] Everett Rogers. Diffusion of Innovations. Free Press, New York, 1983.

[Rushby99] John Rushby. Presented at The 3rd Workshop on Human Error, Safety, and SystemDevelopment (HESSD’99), Liege, Belgium, 7–8 June 1999.

[Rushby95] John Rushby, Formal Methods and their Role in the Certification of CriticalSystems by SRI-CSL-95-01, March 1995.

[Rushby93], John Rushby, Formal Methods and Digital Systems Validation for AirborneSystems. NASA Contractor Report 4551, December 1993.

[Sifakis99] J Sifakis, Integration the price of success, in [FM99].

[SIFT] NASA Conference Publication 2377, Peer Review of a Formal Verification/DesignProof Methodology, July 1983.

[SOCS98] The Safety of Operational Computer Systems, HMSO 1998.

[SPIN99] Theoretical and Practical Aspects of Automata-based Model Checking. Proceedingsof the 5th and 6th SPIN Workshops. D. Dams, et al., editors, Volume 1680, Lecture Notes inComputer Science, 1999.

[SPRU91] Evaluation of the Alvey Programme for Advanced Information Technology, SciencePolicy Research Unit, London HMSO, 1991.

[Stalmarck] Gunnar Stalmarck. A System for Determining Propositional Logic Theorems byApplying Values and Rules to Triplets that are Generated from a Formula, 1989. SwedishPatent Number 467076. U.S. Patent Number 5276897, European Patent Number 0403454.

[Stepney98] Susan Stepney, Incremental Development of a High Integrity Compiler: experiencefrom an industrial development. Third IEEE High-Assurance Systems Engineering Symposium(HASE'98), Washington DC, November 1998.

[Survey] Susan Gerhart, Dan Craigen and Ted Ralston. Experience with Formal Methods inCritical Systems. IEEE Software, January 1994. Reprinted in High-Integrity SystemSpecification and Design, J.P Bowen and M.G Hinchey (Editors). Formal Approaches toComputing and Information Technology Series (FACIT), Springer-Verlag, April 1999.

[TiC] “Trust in Cyberspace.” Fred B. Schneider, Editor. Committee on Information SystemsTrustworthiness, Coputer Science and Telecommunications Board, commission on PhysicalSciences, Mathematics, and Applications, National Research Council. National Academy Press,Washington, D.C., 1999. ISBN 0-309-06558-5.

[TPHOL] www-sop.inria.fr/croap/TPHOLs99/history.html.

[TPHOL99] Theorem Proving in Higher Order Logics. Proceedings of the 12th InternationalConference, TPHOLs’99, University of Nice-Sophia-Antipolis, Nice, France, September 1999.To be published in Lecture Notes in Computer Science, Springer, 1999.

[Ward 93] N.J Ward, “Rigorous Retrospective Static Analysis of the Sizewell B PrimaryProtection System Software”, Proceedings of SAFECOMP ‘93, ISBN 3-540-19838-5, SpringerVerlag, 1993.

Page 49: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 49 of 71

Version 1.0 September 2000

[Z/EVES] Dan Craigen, Irwin Meisels, Mark Saaltink. Analysing Z Specifications withZ/EVES. In Industrial Strength Formal Methods in Practice, J.P Bowen and M.G Hinchey(Editors). Formal Approaches to Computing and Information Technology Series (FACIT),Springer-Verlag, September 1999.

[Zhang99] Wenhui Zhang, Model Checking Operator Procedures, in Proceedings of the 5th and6th SPIN Workshops. D Dams, et al., editors, Volume 1680, Lecture Notes in ComputerScience, 1999.

[ZUM98] Jonathan Bowen, Andreas Fett, Michael Hinchey (Editors). ZUM’98: The Z FormalSpecification Notation. Volume 1493, Lecture Notes in Computer Science, Springer 1998.

Page 50: Formal Methods Diffusion
Page 51: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 51 of 71

Version 1.0 September 2000

Appendix A What areFormal Methods?

This is adapted from "Guidance on the useof Formal Methods in the Development andAssurance of High Integrity IndustrialComputer Systems" from EWICS TC7 atwww.ewics.org.

The term “Formal Methods” (FM) hascome into common use (and abuse) duringthe past decade. In this guideline we take afairly liberal interpretation of the term. Weinclude, for example, not only“mainstream” FMs such as Z, VDM, CSPand CCS, but also other programming andsystem design paradigms which areunderpinned by discrete mathematics, forexample code generation andtransformation, techniques and tools forstatic analysis of programs, andprogramming languages with soundsemantics. Not included are structuredmethods, such as JSD or Yourdon, dataflowapproaches, or object-orientedprogramming although we are interested inthe integration of formal methods withthese methods. The UK MoD Interimdefence Standard on Safety-CriticalSoftware [40] defines a formal method as

A software specification andproduction method, based on amathematical system, that comprises:a collection of mathematicalnotations addressing thespecification, design anddevelopment phases of softwareproduction; a well-founded logicalsystem in which formal verificationand proofs of other properties can beformulated; and methodologicalframework within which softwaremay be verified from the specificationin a formally verifiable manner.

This is a rather ambitious definition. As wedescribe below such methods are attractive,but in practice most formal methods incommon use do not address the full

spectrum of design, some supportingspecification phases, some the constructionphases, and others the analysis of systems.

Typically, formal methods have threecomponents:

• a notion of “program”2,

• a means of expressing propertiesof the computation of programs,and

• some method for establishingwhether a program has someproperty.

Each of these is rigorously mathematicallydefined. This is a fairly liberal descriptionof a formal method. It covers things like thetype inference system of some modernprogramming languages which “prove”programs have a well-formed type, toentirely general purpose theorem provingsystems or proof assistants, or paper andpencil methods. The quality of evidenceone gets from a formal method variesaccording to the method but generally it ishighly accurate, is convincing to trainedworkers, and often has a rather narrowcoverage because it abstracts from manydetailed features of systems.

There is also a wide range of formality andrigour applied in the use of formal methods.A formal proof might consist of justifying aconjecture by deriving it from the basicaxioms of the mathematics upon which thelogical system in use is based. A rigorousargument looks more like an outline of howa proof might proceed, but would notsupply all the intervening detail.Additionally, rigorous argument may draw

2 The notion of program should be interpretedvery liberally. It need not conform closely to“conventional” programs. Along with the notionof program will come a well defined notion ofwhat it means to compute with that program.

Page 52: Formal Methods Diffusion

Page 52 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

upon a rich body of known results, butwithout the need for formally integratingthem into a proof. Note that this is howengineers and mathematicians usually work– they customarily have a commonlyunderstood context which obviates the needto descend into detail which obscures themain subject. However computer basedproof tools do not have the insight ofmathematicians, and cannot interpolateunstated detail between steps. Computerbased proofs are therefore usually verydetailed, and can be extremely obscure,although their validity is potentially lesscontentious than a rigorous argument.

The mathematical basis of formal methodsmay be an existing part of mathematics—for example the Z specification languagehas set theory at its heart— or can bedeveloped anew for the method— theCalculus of Communicating Systems has atheory presented in an algebraic style, butspecific to the Calculus.

Model checking is a method for formallyverifying finite-state concurrent systems.Specifications about the system areexpressed as temporal logic formulas, andefficient symbolic algorithms are used totraverse the model defined by the systemand check if the specification holds or not.Extremely large state-spaces can often betraversed in minutes. The technique hasbeen applied to several complex industrialsystems such as the Futurebus+ and the PCIlocal bus protocols. (seehttp://www.cs.cmu.edu/~modelcheck/index.html)

There are also techniques for showing thatmodels of finite state systems (e.g. thosedefined by a process calculus or transitionsystem) have the same behaviour. This istermed "equivalence checking" and thereare a variety of technical definitions andapproaches depending on the notion of"equivalent".

There is a large and growing variety offormal methods, of varying age and

maturity. It must be realised that there is no“standard” formal method. Each techniquehas particular strengths and weaknesses andit may be that in the course of systemsdevelopment it is appropriate to use anumber of methods at different stages of theprocess. Just as in traditional engineering,no single theory encompasses all aspects ofdesign and development. Many theoreticalapproaches are used— either explicitly orimplicitly.

.

Page 53: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 53 of 71

Version 1.0 September 2000

Page 54: Formal Methods Diffusion
Page 55: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 55 of 71

Version 1.0 September 2000

Appendix B Future Programmes in US

B.1 NASA

With regard to future programmes in formal methods, there is NASA Research Announcement(NRA), which calls for proposals relating to the research, development, prototyping andimplementation of flight critical systems design and validation technologies. There are twosections to the NRA. Within the first section is flight critical systems design and validation andthe NRA describes the following three topic areas:

1. Design Correctness & Certification

Proposals are sought to create requirements specification techniques that enable rigoroushazard/safety analyses and formal verification, and that are useful to all partners(customer, developers, regulators, and maintainers) in the development process, and todevelop objective measures for digital systems.

2. Fault-Tolerant Integrated Modular Avionics

Proposals are sought to develop enabling technologies for fault-tolerant integratedmodular avionics architectures that are scalable to transport class aircraft, business jets,and low cost general aviation aircraft. Rigorous, innovative verification methods aresought that can serve as a basis for certifying reusable integrated modular avionicsplatforms.

3. Operational Malfunction Mitigation

Proposals are sought for development of technologies to ensure the safety andperformance of the closed-loop flight system in the operational environment.Technologies that provide digital upset immunity of processing components, and augmentcurrent capabilities to provide transparent containment, accommodation, and recoveryfrom single-mode and common-mode malfunctions, faults, and failures in the flightsystem are needed. Innovative validation and certification methods that combinesimulation, hardware-in-the-loop testing, and rigorous analysis are of particular interest.

The second section of the NRA specifically relates to formal methods and describes four topicareas:

1. Formal Analysis Tools and Methods

In this area proposals are sought to develop hardware, software, and system design toolsand methods that are mathematically rigorous and application-oriented, and that have astrong potential to reduce certification costs. This topic area is also seeking advances inautomated deduction such as new decision procedures for mathematical domains relevantto avionics applications and their seamless integration into existing verification tools.

2. Demonstration Projects

These concern the demonstrations of the economic feasibility and technical validity ofapplying formal methods to real or realistic systems. Proposals that are closely alignedwith industrial applications are of particular interest.

Page 56: Formal Methods Diffusion

Page 56 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

3. Generic Avionics Components

This is seeking proposals to develop reusable, formally verified avionics components.

4. Accident/Incident Data Research

This is concerned with the development of rigorous methods and tools to assist in theanalysis of the contributions of digital systems to aircraft accidents and incidents

B.2 Critical infrastructure

Over the past few years there have been increasing concerns in the U.S. about potentialvulnerabilities in the nation’s critical infrastructure. The President’s Commission on CriticalInfrastructure Protection (PCCIP) defined an “Infrastructure” to be a network of independent,mostly privately-owned, manmade systems and processes that function collaboratively andsynergistically to produce and distribute a continuous flow of essential goods and services.Critical infrastructures are those that are so vital that their incapacity or destruction would havea debilitating impact on U.S. defence and economic security. The full report can be found in[PCCIP]. Recommended R&D is presented in [PCCIPR&D]. The PCCIP made three key R&Drecommendations [PCCIPR&D]:

1. Conduct research and development in six areas:

• information assurance

• monitoring and threat detection

• vulnerability assessment and systems analysis

• risk management and decision support

• protection and mitigation

• incident response and recovery

2. Increase the federal investment in infrastructure assurance research to $500 million inFY99 and incrementally increase the investment over a five-year period to $1 billion inFY04.

3. Establish a focal point for national infrastructure assurance R&D efforts and build apublic/private-sector partnership to foster technology development and technologytransfer.

As a result of the R&D recommendations and the recommendations made in the main PCCIPreport, the Clinton administration executed Presidential Decision Directive 63 (PDD 63)[PDD63a,PDD63b] on “Protecting America’s Critical Infrastructures.” PDD 63 is summarisedby a White House fact sheet which is included in Appendix C below. In January 1999, theClinton administration announced further initiatives to defend the U.S. critical infrastructurefrom cyber-terrorism. These initiatives are summarised by a White House fact sheet which isincluded in Appendix D below. The initiative appears to further build upon the report of the

Page 57: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 57 of 71

Version 1.0 September 2000

PCCIP by focusing on critical infrastructure applied research, focus on computer intrusiondetection networks, information sharing and analysis centres, and the creation of a cyber corps.

As of the writing of this report, there is still a substantial state-of-flux in the U.S. government asto how to react to PDD 63 and the various Clinton initiatives. There are substantial interagencydiscussions and some indications that the U.S. Congress is becoming somewhat unsettled as tothe evolving plans. We expect that as the budgetary debate for FY00 proceeds, clarifications onhow the government will actually respond to PDD 63 will be forthcoming. There are variouscommittees that have investigated the Critical Infrastructure issue. Of note, and publiclyavailable, is the report from the U.S. National Research Council [TiC] entitled “Trust inCyberspace.” This report provides some technical substance to the issue and, we expect, willhelp to focus some of the interagency discussions. The key conclusions and researchrecommendations of the NRC committee is reported in Appendix E.

However it is interesting to note that the two themes identified earlier, that of COTS and formalmethods, recur here, see Appendix E:

“It is clear that networked information systems will include COTS components into theforeseeable future. However, the relationship between the use of COTS components andNIS trustworthiness is unclear. Greater attention must be directed toward improving ourunderstanding of this relationship.”

“Formal methods are being used with success in commercial and industrial settings forhardware development and requirements analysis and with some success for softwaredevelopment. Increased support for both fundamental research and demonstrationexercises is warranted.”

B.3 Other US programmes

Other U.S. programmes are likely to have formal methods components. For example, the AirForce Office of Scientific Research (AFOSR) BAA 2000-1 which includes formal methodsunder “Software and Systems,” and DARPA’s BAA 99-33 on information assurance andsurvivability. All three programmes are security- or defence-related.

Page 58: Formal Methods Diffusion
Page 59: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 59 of 71

Version 1.0 September 2000

Appendix C Protecting America’s critical infrastructures:PDD 63

Dated: May 22, 1998

This Presidential Directive builds on the recommendations of the President’s Commission onCritical Infrastructure Protection. In October 1997, the Commission issued its report calling fora national effort to assure the security of the United States’ increasingly vulnerable andinterconnected infrastructures, such as telecommunications, banking and finance, energy,transportation, and essential government services.

Presidential Decision Directive 63 is the culmination of an intense, interagency effort toevaluate those recommendations and produce a workable and innovative framework for criticalinfrastructure protection. The President’s policy:

Sets a goal of a reliable, interconnected, and secure information system infrastructure by theyear 2003, and significantly increased security to government systems by the year 2000, by:

Immediately establishing a national center to warn of and respond to attacks.

Ensuring the capability to protect critical infrastructures from intentional acts by 2003.

Addresses the cyber and physical infrastructure vulnerabilities of the Federal government byrequiring each department and agency to work to reduce its exposure to new threats;

Requires the Federal government to serve as a model to the rest of the country for howinfrastructure protection is to be attained;

Seeks the voluntary participation of private industry to meet common goals for protecting ourcritical systems through public-private partnerships;

Protects privacy rights and seeks to utilize market forces. It is meant to strengthen and protectthe nation’s economic power, not to stifle it.

Seeks full participation and input from the Congress.

PDD-63 sets up a new structure to deal with this important challenge:

a National Coordinator whose scope will include not only critical infrastructure but also foreignterrorism and threats of domestic mass destruction (including biological weapons) becauseattacks on the US may not come labeled in neat jurisdictional boxes;

The National Infrastructure Protection Center (NIPC) at the FBI which will fuse representativesfrom FBI, DOD, USSS, Energy, Transportation, the Intelligence Community, and the privatesector in an unprecedented attempt at information sharing among agencies in collaboration withthe private sector. The NIPC will also provide the principal means of facilitating andcoordinating the Federal Government’s response to an incident, mitigating attacks, investigatingthreats and monitoring reconstitution efforts;

Information Sharing and Analysis Centers (ISACs) are encouraged to be set up by the private

Page 60: Formal Methods Diffusion

Page 60 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

sector in cooperation with the Federal government and modeled on the Centers for DiseaseControl and Prevention;

A National Infrastructure Assurance Council drawn from private sector leaders and state/localofficials to provide guidance to the policy formulation of a National Plan;

The Critical Infrastructure Assurance Office will provide support to the National Coordinator’swork with government agencies and the private sector in developing a national plan. The officewill also help coordinate a national education and awareness program, and legislative and publicaffairs.

Page 61: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 61 of 71

Version 1.0 September 2000

Appendix D Keeping America secure for the 21st century:Computer Security and Critical Infrastructure

Dated: January 22, 1999

Today, President Clinton will announce new initiatives to defend the nation’s computer systemsand critical infrastructure from cyber-terrorism. The most critical sectors of our economy –power-generation, telecommunications, banking, transportation and emergency services – arepotentially vulnerable to disruptions from computer attack.

President Clinton will propose in the Fiscal Year 2000 budget spending $1.46 billion to defendagainst this emerging threat. This represents an increase of $400 million from the FY 1999budget proposal.

It will include funding for the following initiatives:

Critical Infrastructure Applied Research Initiative: The President’s budget proposal includes$500 million for research and development efforts. A portion of these funds will be spent onnew initiatives to improve information assurance by safeguarding networks. Funds will also bededicated to developing tools that can identify anomalous activities and “Trojan Horses”(malicious codes installed by unauthorized users).

Computer Intrusion Detection Networks: The Defense Department has already begun to installintrusion detection systems and create a network to warn key computers of an attack. Under thePresident’s initiative, a similar system for other Federal agencies will be evaluated anddesigned. These networks will ensure that when one computer system is attacked, others in thenetwork will be instantly warned of the intrusion, informed of the mode of attack used, andprovided with methods to stop it.

Information Sharing and Analysis Centers (ISACs): As part of thepublic-private partnership, wewill support the initial establishment of ISACs to foster private sector development of bestpractices and standards for computer security, to encourage the sharing of vulnerability analysis,and to provide outreach and training programs. These ISACs will enable the Federalgovernment to provide private industry with threat information without compromising privacy,civil liberties or proprietary data.

Cyber Corps: This program will address the shortage of highly skilled computer scienceexpertise in the government and enable agencies to recruit a cadre of experts to respond toattacks on computer networks. It will use existing personnel flexibilities, scholarship andfinancial assistance programs, and examine new scholarship programs to retrain, retain andrecruit computer science students.

Page 62: Formal Methods Diffusion

Page 62 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

Page 63: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 63 of 71

Version 1.0 September 2000

Appendix E Trust in cyberspaceThese conclusions and research recommendations are drawn from Chapter 7 of the “Trust inCyberspace” report. The report should be read to provide the context for these conclusions. Wehave not included all the conclusions, but have provided a selection so as to provide anindication of the issues discussed.

1. The public telephone network is increasingly dependent on software and databases thatconstitute new points of vulnerability. Business decisions are also creating new points ofvulnerability. Protective measures need to be developed and implemented.

2. The Internet is too susceptible to attacks and outages to be a viable basis for controllingcritical infrastructures. Existing technologies could be deployed to improve the trustworthinessof the Internet, although many questions about what measures would suffice do not currentlyhave answers because good basic data is scant.

3. The design of trustworthy networked information systems presents profound challenges forsystem architecture and project planning. Little is understood, and this lack of understandingultimately compromises trustworthiness.

4. It is clear that networked information systems will include COTS components into theforeseeable future. However, the relationship between the use of COTS components and NIStrustworthiness is unclear. Greater attention must be directed toward improving ourunderstanding of this relationship.

5. Formal methods are being used with success in commercial and industrial settings forhardware development and requirements analysis and with some success for softwaredevelopment. Increased support for both fundamental research and demonstration exercises iswarranted.

6. Cryptographic authentication and the use of hardware tokens are promising avenues forimplementing authentication.

7. Foreign code is increasingly being used in NISs. However, NIS trustworthiness willdeteriorate unless effective security mechanisms are developed and implemented to defendagainst attacks by foreign code.

8. Improved trustworthiness may be achieved by the careful organization of untrustworthycomponents. There are a number of promising ideas, but few have been vigorously pursued.

9. As a truly multidimensional concept, trustworthiness is dependent on all of its dimensions.however, in some sense, the problems of security are more challenging and therefore deservespecial attention.

10. The NSA R2 organization must increase its efforts devoted to outreach and recruitment andretention issues.

11. DARPA is generally effective in its interactions with the research community, but DARPAneeds to increase its focus on information security and NIS trustworthiness research, especialywith regard to long-term research efforts.

Page 64: Formal Methods Diffusion
Page 65: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 65 of 71

Version 1.0 September 2000

Appendix F European R&D projects

F.1 Esprit projects

The following table describes the Esprit projects who mention formal methods in theirdescriptions.

METEOR An Integrated Formal Approach to Industrial SoftwareDevelopment

ESPRIT 1

DESCARTES Debugging and Specification of Ada Real-Time EmbeddedSystems

ESPRIT 1

GENESIS An Environment for Formal Systems Development ESPRIT 1

FOR-ME-TOO Formalisms, Methods and Tools ESPRIT 1

FORMAST Formal Methods for Asynchronous System Technology ESPRIT 1

GRASPIN Personal Workstation for Incremental GraphicalSpecification and Formal Implementation of Non-Sequential Systems

ESPRIT 1

VIP VDM Interfaces for PCTE ESPRIT 1

DRAGON Distribution and Reusability of Ada Real-TimeApplications through Graceful and On-Line Operations

ESPRIT 1

PANGLOSS Parallel Architecture for Networking Gateways LinkingOSI Systems

ESPRIT 1

DEMON Design Methods Based on Nets ESPRIT 2

SPEC Formal Methods and Tools for the Development ofDistributed and Real-Time Systems

ESPRIT 2

FIDE Formally Integrated Data Environment ESPRIT 2

DYANA Dynamic Interpretation of Natural Language ESPRIT 2

IS-CORE Information Systems: Correctness and Reusability ESPRIT 2

COMPASS A Comprehensive Algebraic Approach to SystemSpecification and Development

ESPRIT 2

REDO Maintenance, Validation and Documentation of SoftwareSystems

ESPRIT 2

ICARUS Incremental Construction and Reuse of RequirementsSpecifications

ESPRIT 2

PATRICIA Proving and Testability for Reliability Improvement of ESPRIT 2

Page 66: Formal Methods Diffusion

Page 66 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

Complex Integrated Architectures

LACOS Large-Scale Correct Systems Using Formal Methods ESPRIT 2

PROOFS Promotion of Formal Methods in the European SoftwareIndustry

ESPRIT 2

AFRODITE Applying Formal Methods to Real-Size Object-OrientedDesigns in Technical Environments

ESPRIT 3

FORMAT Formal Methods in Hardware Verification ESPRIT 3

CHARME-2 Formal Design and Correctness Verification ofSynchronous and Asynchronous Digital VLSI Systems

ESPRIT 3

CLICS II Categorical Logic in Computer Science II ESPRIT 3

AMODEUS 2 Assaying Means of Design Expression for Users andSystems

ESPRIT 3

ACID-WG Asynchronous Circuit Design ESPRIT 3

COMPASS A Comprehensive Algebraic Approach to SystemSpecification and Development

ESPRIT 3

CASCADE Certification and Assessment of Safety-Critical ApplicationDevelopment

ESPRIT 3

NADA New Hardware Design Methods ESPRIT 3

USEGAT Use of integrated graphical and textual formal specificationlanguages in industry

ESPRIT 4

SACRES Safety critical embedded systems: from requirements tosystem architecture

ESPRIT 4

ARTS Application of real time specification ESPRIT 4

2RARE 2 real applications for requirements engineering ESPRIT 4

LAW LEGACY ASSESSMENT WORKBENCH ESPRIT 4

FORSITE FORMAT SOFTWARE IN AN INDUSTRIAL DESIGNENVIRONMENT

ESPRIT 4

APPLIGRAPH Applications of Graph Transformation ESPRIT 4

V-FORMAT VERIFYING SYSTEM DESIGNS USING FORMALMETHODS

ESPRIT 4

INFORMA INTEGRATED FORMAL APPROACHES FOREMBEDDED REAL-TIME SYSTEMS

ESPRIT 4

SPECTRUM IMPROVED COMPETITIVENESS THROUGHINTEGRATION AND AUTOMATION OF

ESPRIT 4

Page 67: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 67 of 71

Version 1.0 September 2000

COMPLEMENTARY FORMAL APPROACHES IN THEDEVELOPMENT PROCESS

VIRES Verifying Industrial Reactive Systems ESPRIT 4

MEFISTO Modelling, Evaluating and Formalising Interactive Systemsusing Tasks and interaction Objects

ESPRIT 4

FMERAIL FORMAL METHODS EUROPE— RAILWAYS ESPRIT 4

F.2 ESSI

QUAFOMET Improving Quality and Efficiency of Safety CriticalEmbedded Computing Systems by the Use of FormalMethods

ESSI 1

MAFMETH Application, Measurement and Dissemination of aMathematically Formal Methodology

ESSI 1

CONFORM A Comparison of Conventional and Formal Methods in theDevelopment of a Secure System

ESSI 1

FMEINFRES Formal methods Europe: information resources ESSI 2

FMEINDSEM Industrial seminars on formal methods ESSI 2

FM-GUIDES Guide-books and videos for managers on formal methods ESSI 2

PICGAL Process improvement experiment of a code generator to theAriane launcher

ESSI 2

PCFM PROOF BY CONSTRUCT USING FORMAL METHODS ESSI 2

EUREX EUROPEAN EXPERIENCE EXCHANGE ESSI 2

Page 68: Formal Methods Diffusion

Page 68 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

Appendix E Selected German R&D Programmes

Priority research programme 'Deduction' of the German Research Foundation.The German Research Foundation brang together almost all research groups engaged inthe field of automated reasoning within Germany. The program had started in 1992, andthe final phase lasted from 1996 to 1998.

The primary goal of this programme was the converging of deduction methods andsystems. An immediate subgoal was to achieve by comprehensive comparisons a certainclarification and unification of the various logics, formalisms, methods, strategies,representational structures and implementation techniques. The results of these studiesshould serve as an orientation and basis for the development of adequate deductionmethods.

The following list contains respective research topics which were addressed within theprogramme; it is structured according to classical dimensions of a deduction system:

- Calculi like Resolution, Tableaux calculi, Model Elimination and many more, aswell as their variants. Investigations of relations amongst them and complexityissues.

- Mechanisms of concentration, like integrating sorts, theories, constraints, etc.,unification theory, term rewriting systems and equality. Comparison andimprovement of successful strategies coming from different methods, such asdefinitions, lemmas, proof schemata, proof planning, tactics, etc; incorporation ofthese into proof search in order to avoid redundancy.

- Representational aspects of knowledge, control structure, strategies; indexing andintegration of different methods and systems. Study of these questions in differentlogics, such as Horn logics, first and higher order predicate logics (with sorts),(restricted) induction, aiming at an optimal balance between expressivity andcomplexity.

- Implementational problems, compilation techniques, high-performance provers;interaction and combination of different systems.

Connected with theoretical research there was the strong need for the development ofsome deduction systems, expected to outperform existing systems.

Reference:Wolfgang Bibel, Peter H. Schmitt, Automated Deduction – A Basis for Applications,Volume I-III, Kluwer Academic Publishers

Page 69: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 69 of 71

Version 1.0 September 2000

KORSO (Correct Software) was funded by the German Ministry of Research andTechnology as a compound project from 1991 to 1994. The aim of this research projectwas to produce a framework for developing correct software. This included a rigorousmethodology, a language for specification, and a powerful system to support thedevelopment process. The system had to include a comprehensive library ofstandardized specifications, program modules, transformations, proofs, anddevelopment methods, as a knowledge base for program technology.

Project Partners

- GMD Birlinghoven, GMD Berlin, GMD Karlsruhe

- Technische Universität Braunschweig

- Technische Universität Berlin

- Universität Karlsruhe

- Universität Ulm

- Ludwig-Maximilians-Universität München

- Technische Universität München

- Universität Oldenburg

- Deutsches Forschungszentrum für künstliche Intelligenz

- Universität des Saarlandes at Saarbrücken

- Siemens FZE München

Reference:URL: http://www.informatik.uni-bremen.de/~inform/forschung/korso/korso.html

In the years 1996 – 1999 the development program 'Software Technology' wassponsored by the German Federal Ministry of Research and Technology in cooperationwith the associations of the german electronic and mechanical engineering companies.One research area was the further development of methods and tools to increase thesecurity/safety and reliability of complex software systems. Three projects in that areawere funded: ESPRESS, KorSys, UniForM

ESPRESS aimed to increase productivity in developing complex, safety-critical,embedded systems and enhance the reliability of such systems by the development of amethodological tool-supported software technology for specific application areas

Page 70: Formal Methods Diffusion

Page 70 of 71 Formal Methods Diffusion: Past Lessons and Future Prospects

September 2000 Version 1.0

covering the whole life-cycle. The project focused on the application area of automobileelectronics and traffic light control.

Essential features were the explicit separation of specifications into functional andsafety relevant parts, the combination of graphical (statecharts) and formal methods (Z)as well as verification, code-generation, systematic testing and automatic testevaluation.

The soundness of the software technology and its orientation towards practical needswas ensured by conducting and taking into account concrete case studies of industrialrelevance. This approach additionally lead to a gradual improvement of its maincharacteristics, concepts and support tools.

Reference:URL: http://www.first.gmd.de/~espress

KorSys: The aim of the KorSys project was to improve the applicability of formalmethods for the specification and verification of complex software systems in practice.

The activities at Technical University of Munich were part of the KorSys projecttogether with BMW, ESG, SIEMENS AG and OFFIS.

The project was divided into four specific areas:

- Methods and tools for reactive systems

- Methods and tools for finite state systems

- Industrial applications

- Quality of tools

Reference:URL: http://www4.informatik.tu-muenchen.de/proj/korsys/all/

UniForM: The development of reliable software for industrially relevant tasksdemanded a suitable tool-supported combination of formal methods.

The UniForM Workbench was a generic framework, instantiated with specific tools formethods to handle communicating distributed systems and real-time requirements. Thecombination of methods in a logically consistent way and the development of correcttransformation tools, a basic aspect of quality assurance, were demonstrated.

Page 71: Formal Methods Diffusion

Formal Methods Diffusion: Past Lessons and Future Prospects Page 71 of 71

Version 1.0 September 2000

The industrial potential of the UniForM Workbench was illustrated by a case study fromindustry, the development of a decentralized control unit for single-track tramwaynetworks.

Reference:URL: http://www.informatik.uni-bremen.de/~agbkb/UniForM/Title.html