Certus - Glimpses of a Centre for Research-based Innovation in Software Verification\&Validation

8/12/2019 Certus - Glimpses of a Centre for Research-based Innovation in Software Verification\&Validation

1/4

Certus - Glimpses of a Centre for Research-basedInnovation in Software Verification&Validation

Sagar SenCertus V&V Centre

Simula Research Laboratory1325 Lysaker, Norway{sagar}@simula.no

ABSTRACT

What is gratifying to a software engineering researcher? Threeof many possible answers to this question are (a) the in-tellectual exercise in developing/disseminating approachesthat address novel challenges (b) recognition from impactin a community of researchers (c) widespread use of novelideas, including software, in the society at large leading to

enhancement of human ability and job creation. Aculmina-tionof these sources requires an organizational effort. Thisarticle presents glimpses of a research-based innovation cen-tre, Certus, to facilitate such a culmination for software engi-neering researchers. Certus has established a body of knowl-edge, methods and tools for the validation and verification(V&V) of software systems in the local Norwegian indus-try. Certus works in close cooperation with five foundinguser partnersand is hosted by the Simula Research Labo-ratory. We present the organizational structure of Certusand describe how Certuss life and health is planned andevaluated on a regular basis as a research-based innovationcentre. We hope that this document will serve as a ba-sis to encourage national/international funding schemes tocreate call for proposals for long-term research-based inno-

vation centres. This, we believe, is one way to justify use oftax payers resources in creating a win-win situation for thetriple helix: government, researchers and industry.

Keywords

Certus, industry-driven software engineering research, soft-ware verification and validation

1. INTRODUCTIONThe well-known american author Henry Jackson van Dyke

once said, In the progress of personality, first comes a dec-laration of independence, then a recognition of interdepen-dence.. From a software engineering researchers perspec-

tive this entails making the transition from being satisfied by

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$15.00.

results known to his/her research community to widespreadimpact in the software industry and the society at large.This is a non-substantial effort and often has several barrierssuch as attitudes and communication between researchersand practitioners, strain from the quintessential drive ofpublish-or-perish, technique and tools not very well deployedto be useful in industry, and lack of an organizational struc-ture for serious commitment and dialogue between industry

and academia. In this article, we present an organizationalstructure to facilitate fruitful industry-academia research.

This article presents the model of a centre for softwareverification&validation research called Certus. The CertusCentre has established a body of methods and tools forthe validation and verification (V&V) of software-intensivesystems. Working in close cooperation with Certuss fivefounding user partners (software companies), the centresresearchers have analyzed and produced innovative meth-ods to validate the following types of systems: (a) real-timeembedded software systems (b) highly-configurable softwaresystems and (c) data-intensive software systems. We presentCertus as an example to motivate the creation of similar cen-tres in conducive environments. We envisage a huge poten-

tial for such centres in city areas with universities and a largenumber of software companies. A centre such as Certus canhelp train both undergraduate and graduate students bet-ter for both the workforce of the local software industry andeffective industry-academia research.

The article is organized as follows. In Section 2, we presenthow Certus is funded under a governmental scheme calledSFI. We present the architectural overview of Certus in Sec-tion 3. We briefly describe the evaluation of the centreshealth in Section 6. In Section 7, we present some relatedwork on the subject of research-based innovation. We con-clude in Section 8, with future perspectives.

2. FUNDING THE CENTREA legally binding agreement among the triple helix [3]:

government, academia, and industry is a key requirementto fund and establish centers such as Certus. Some of thetext in this section is a re-production of content from theNorwegian research councils website that provides severaloptions for sharing the same information.

In 2006, the Minister of Education and Research announcedthe creation of fourteen centers for research-based innova-tion or in Norwegian Senter for forskningsdrevet innovasjon(SFI) 1. These SFIs were appointed by the Executive Board1http://www.forskningsradet.no/prognett-sfi/About_


2/4

of the Research Council and started in the spring 2007 andwill be financed by the Research Council for eight years un-til the spring 2015. The centers include CAST for Stem-cell based Innovation, SIMLAB or Structural Impact Labo-ratory, MABCENT-Marine bioactives&drug discovery, andMI Lab-Medical Imaging Laboratory, NTNU and so on.

The call for proposals of such centers is renewed everyfour years similar to the model of the olympic games. Sevennew centres were appointed by the Research Council Board

on 16th December 2010. They have started their activitiesin the autumn 2011 and are expected to operate until 2019.The Certus Center is one of them. It was established byProf. Lionel Briand (who was also the first centre leader,while Arnaud Gotlieb is the current leader) in collaborationwith Simula Research Laboratory. The centre leader playsan important role in attracting funding owing to his/heroutstanding scientific credentials.

The SFI scheme aims to :

Encourage companies to innovate through a greaterfocus on long-term research, and make it attractive forinternational companies, to establish their operationsin Norway.

Facilitate active cooperation between innovative com-panies and prominent research groups.

Promote the development of industry-oriented researchthat is in the forefront of international research and in-cluded in dynamic international network.

Stimulate research in areas that are important for in-dustry and research-based knowledge and technology.

SFI scheme has higher goals, greater longevity and a moreconcentrated effort than other means of innovation. Theinitiative is aimed specifically at the R&D active part ofthe Norwegian sector. SFI scheme provides companies withthe opportunity for greater persistence, continuity and risk

reduction in funding for research. For the research opens theSFI opportunity for long-term human resources developmentthrough research of high scientific level in close cooperationwith companies.

SFI has two overarching criteria that form the basis forselection of centers:

Potential for innovation and value creation

Scientific quality

The SFI is related to a research institution or companythat is responsible as the host institution for business. HostInstitutions for the centers could be universities, colleges, re-search institutes, R&D-performing companies and researchperforming public service which is well placed to realize SFIconcept. The host institution and a number of partnersconstitute Centre (CRI consortium). Partners can be enter-prises, public enterprises and research institutions. The SFIis a time-limited research that is not regarded as a separatelegal entity.

SFI centers shall have a duration of eight years, but ex-tension beyond five years is dependent on a positive resultof the evaluation will be conducted after 3.5 years. Indica-tive financial framework for Research Council funding of an

the_SFI_scheme

SFI are 9.12 million per. year for up to eight years. Atleast 50 percent of the agencys annual budget is funded bySFI consortium. It is expected that the business partnersin the consortium alone should cover at least 25 percent ofthe agencys annual budget. This means that a center wouldnormally have an annual budget of 20-30 million. A largepart of this contribution from business partners of Certus isaccomplished by involvement in person-weeks by employees

on the business partner company sites.

3. ARCHITECTURAL OVERVIEWThe Certus centre for research-based innovation (Certus

SFI) is a 5-8 year research centre within the area of soft-ware validation and verification established in October 2011.The centre engages Simula Research Laboratory (Simula orSRL) as host institution and primary research partner, andseveral user partners forming a consortium. An user part-ner is involved in various projects prescribed in collabora-tively developed annual work plansas described in Section3.1. The current user partners of Certus include KongsbergMaritime, FMC Kongsberg Subsea, Norwegian Customs andExcise, Cisco and Esito, and ABB Robotics. The role of thecentre in this architectural setup is to test and transfer re-

search results from software verification and validation intothe workflow of its user partners.

3.1 Annual Work PlanThe Annual Work Plan (AWP) refines and develops the

Research and Innovation actions of the centre. It is an agree-ment between the Certus centre, its user partners, and theprimary funding body (The Norwegian Research Council)about the actions to be taken in the current year. The AWPis organized in Projectsand Tasks which correspond respec-tively to long-term scientific areas and short-terms dedicatedactions. Each task has one or more deliverables. The tem-plate for the projects in the AWP is presented in Figure 1.For instance, the plan for 2014 contains 8 active Projects and

24 active Tasks. The production of the Centre is expected tobe of 68 deliverables, for a total effort of 173 Person Months.The deliverables in each task are produced in four categories:

1. Administrative documents. These deliverables cor-respond to documents used for the administration ofthe centre. This includes plans, progress reports, min-utes, lists of participants, budgets, and so on.

2. Technical reports. These deliverables correspond totechnical documents containing scientific results, re-sults of experimental evaluation, domain and state-of-the-art analysis, partners specific case study, and soon.

3. Presentations. These deliverables correspond to tech-nical presentations made by or for the user partners ofthe Centre.

4. Software. These deliverables correspond to both ad-ministrative soft. designed for the needs of the centreadministration and scientific software designed for im-plementing and evaluating scientific results.

Projects define the various activities of the centre. A projectis managed by a responsible, and has a research type: indus-trial or fundamental. Each project has several tasks. Each


3/4

...

No. Time Type DescriptionDeliverables

Task i.1

Task LeaderStart MonthFinishMonth

Descripition

Partner Resource Allocation

Adjacent Activity


No. Time Type DescriptionDeliverables

Task i.n

Task LeaderStart MonthFinishMonth

Descripition


Adjacent Activity


Project i

Figure 1: Annual Work Plan Structure

task has a leader and he/she must ensure that deliverablesare submitted within the timeline. The different activitiesin projects include:

1. Management. actions involves tasks such as plan-ning and reporting (deliverables include budget, andreporting to the research council, and the annual workplan), governance (deliverables include minutes frommeetings, updated online repository of deliverables),mid-way evaluation (deliverables include mandate forself-evaluation process, self-evaluation report, docu-ment outlining improvements suggested).

2. Industrial Exploitation. aims to develop researchresults (a) into best practices guidelines in a form specif-ically adapted to the needs of the individual user part-ners (b) in the form of software prototypes into soft-

ware tools with stability and functionality adapted tothe needs of the individual user partners (c) in theform of software prototypes into software tools withstability and functionality suitable for general distri-bution without any form of warranties (d) To supportand maintain the developed best practices guidelinesand software tools for the benefit of the consortiumthroughout the term of the centre. The tasks in theproject include development of an exploitation pol-icy, exploitation case studies consisting of legal issuesand an exploitation plan, and exploitation of specificprojects such as our tool Depict to test data quality inthe Norwegian Customs and Excise department.

3. Training and Knowledge Transfer. This project

aims to train industry professional with the state ofthe art in software V&V. It is described in Section 5.

4. Dissemination and Communication. The Certusdissemination and communication activities are on-going efforts throughout the term of the SFI. In ac-cordance to the listed objectives, the SFI aims to dis-seminate information in popularized, scientific and in-dustrialized form. Certus produces an annual reportwith text, facts and figures summarizing the activi-ties and achievements of the centre during the year.

Certus also maintains a website 2 that publishes thedeliverables and news updates. Research scientists inCertus are also closely involved in supervising mastersstudents from the University of Oslo.

5. Scientific theme-specific projects. Certus researcherslead projects with a scientific theme, which also involveextensive interaction with user partners. These themesinclude model-based testing, testing data-intensive sys-tem, smart testing of evolving systems, and model-based requirements engineering. Scientific projects areled by postdocs and permanent researchers. They ac-tively undertake PhD student supervision under theirrespective themes.

The lifetime of projects will span the entire duration of

the existence of Certus. New projects can be added to theAWP depending on addition of a new form of competencein the centre. However, the tasks and their deliverables arebounded by time. They are preserved in the AWP, andadditional tasks or deliverable are added each year. Thenew tasks and deliverables are added based on a workshopheld with user partners ( Section 4).

4. USER PARTNER WORKSHOPSCertus organizes User Partner Workshops (UPW) to bring

all user partners together. We realize two workshops, onein the spring and one in the fall. These workshops providea platform for (a) Researchers to eliminate jargon and com-municate their scientific ideas to our industry partners in

the most conducive manner (b) Encourage our partners toexpress their problems with complete trust in Certus SFI.Trust with regard to aligning our research activities to solvethese problems to the very end (c) Identify commonalitiesacross problems in various user-partners (d) Find a mid-dle path to foster effective innovation and state-of-the-artresearch for the current year. We achieve these goals by or-ganizing the workshop as a combination of lectures, groupwork and plenum discussions to foster effective interaction

2www.certus-sfi.no


4/4

between Certus and our partners. Each workshop will b earranged as a 1.5 day overnight event at a conference hotel.

5. INDUSTRIAL COURSESCertus is involved in several dissemination activities. Giv-

ing courses to industry professionals is an important form ofdissemination to greatly improve communication betweenresearchers and industry professionals. These courses are

held one or two times a year over the period of two fulldays. It is offered to employees of user partners and alsoopens doors to new professionals. The low frequency of thecourses is due to conflicting commitments of full-time em-ployees. The courses are tailored to suit the needs of theindustry professionals. For instance, the training programin 2014 will disseminate scientific elements in Software V&Vthat is relevant to challenges identified after the 2013 col-laboration with our user partners. A feedback-driven brain-storming session helps Certus fine-tune its annual work plan.

6. EVALUATING CERTUSS HEALTHThe center is evaluated based on a number of factors.

Some factors include number of publications, innovation in-dex (spin-offs and tools successfully transferred to the in-dustry), international cooperation, gender equality to namea few. The research council and external evaluators (in amid-way evaluation) evaluate Certus on these factors. Wedescribe some of the factors below.

6.1 PublicationsScientific publications is the primary factor to evaluate the

health of Certus. Publication quality is evaluated based onrankings of conferences and journals in software engineering.Conference venues where Certus members normally pub-lish include MODELs, ICSE, ICST, ICSM, ICTSS, SPLC,CAISE, CP, QSIC to name a few. The journal venues in-clude SoSym, IEEE Trans. of Software Engineering, IEEE

Software, IST, ACM TOSEM, and so on.

6.2 International Cooperation and Standard-ization

International cooperation with researchers in software en-gineering and other domains is also of paramount impor-tance to evaluate the health of the centre. Certus officiallycollaborates with partners from China, Italy, Luxembourg,Canada, and France and is always looking to extend itsreach. Certus also contributes to standardization effortssuch as UML MARTE, and UML Testing Profile. Certusalso actively participates in European projects such as Open-coss for safety certification. As a results of its collaborationwith partners in EU, Certus actively seeks external funding

in European call for proposals.

6.3 Gender EqualitySimula is committed to recruiting women into software

engineering. As of 2014, Certus funds two PhD-students,one male and one female. Considering only 15 per centof masters-level software engineering graduates (prerequi-site for PhD study) are women, recruitment is a challenge.Through systematic work Certus has succeeded in achievinga gender balance. There are currently seven full-time scien-tists working in Certus. Of these, three are female and four

are male. The Certus Governing Board has a 50-50 genderdistribution.

7. RELATED WORKCapitatlization of knowledge is gaining importance in coun-

tries to spawn innovation and create jobs. Creating science-based industries from academic research is a common themeof industrial policy, whether made explicit as in France and

Mexico or implicit such as in the USA [3] [1]. When it comesto research in software engineering just before the turn ofthe 20th century Robert Glass wrote an influential article[2] . He pointed out that software research was too narrowandarrogantto address practical problems. There were sev-eral analytical approaches in controlled setups but a lack ofempirical evaluation of large systems. On the other hand[4], wrote about the emergence of the modern trend in soft-ware engeering to adhere to theindustry-as-laboratory modelleading to empirical definition of problems. This burgeon-ing is now leading to the culmination of organizational ef-forts such as Certus. There are several industry-led researchlabs in software engineering such as Microsft research, NokiaResearch, Ercisson research, IBM research to name a few.However, centres such as Certus accomplish a close collab-

oration within the triple helix of government, industry andacademia as idealized in editors Henry Etzkowitz et. al. [3].Examples of well-established centres for research-based inno-vation in software engineering include SnT, InterdisciplinaryCentre for Security, Reliability and Trust 3, the FraunhoferCentre for experimental software engineering, Maryland toname a few.

8. CONCLUSIONIn this paper, we present glimpses of the Certus center

for research-based innovation. Certus is a center for soft-ware verification and validation research that is driven bychallenges in the local Norwegian software industry. Thearchitecture of Certus is based on an annual work plan that

consists of projects, tasks and deliverables. It is an agree-ment between Certus and its user partners for the tasks tobe achieved over an year. The health of Certus is evalu-ated based on several criteria including publications, inter-national cooperation, standardization efforts, innovation in-dex, and gender equality. We believe that the paper presentssome elements of the Certus organizational model that canhelp play a role in motivating funding schemes and univer-sity research groups in countries with a flourishing softwareindustry.

9. REFERENCES[1] H. Etzkowitz. The norms of entrepreneurial science:

cognitive effects of the new universityindustry linkages.

Research Policy, 27(8):823 833, 1998.[2] R. L. Glass. The software-research crisis. Software,

IEEE, 11(6):4247, Nov 1994.

[3] P. H. Henry Etzkowitz, Andrew Webster, editor.Capitalizing Knowledge: New Intersections of Industryand Academia. SUNY Press, 1998.

[4] C. Potts. Software-engineering research revisited.Software, IEEE, 10(5):1928, Sept 1993.

3http://www.securityandtrust.lu

Certus - Glimpses of a Centre for Research-based Innovation in Software Verification\&Validation

Documents

Transcript of Certus - Glimpses of a Centre for Research-based Innovation in Software Verification\&Validation