Teaching software testing in a modern development environment1356814/FULLTEXT01.pdf ·...

INOM EXAMENSARBETE DATATEKNIK,GRUNDNIVÅ, 15 HP

, STOCKHOLM SVERIGE 2019

Teaching software testing in a modern development environment

Lära ut mjukvarutestning i en modern utvecklingsmiljö

ANDERS GAWELL

ANTON KALLIN

KTHSKOLAN FÖR ELEKTROTEKNIK OCH DATAVETENSKAP

i

AbstractAll developers understand the benefits of testing their code to ensure its func-tionality. Today’s market is moving further towards design principles wheretesting is a central or driving force during development. This puts a certainpressure on academia to supply these skills to their students.

Recently the course II1302 - Projects and project methods at the Royal In-stitute of Technology in Kista made a concerted effort to introduce the studentsof the course to these modern concepts. This thesis investigates how areas oftesting can effectively be introduced to the students in the course, utilizinga tailored example that takes the area of testing into particular considerationand how to automate it via DevOps-tools provided by a cloud-based service.Further, it also makes an attempt to provide additional material to be used forteaching testing in conjunction with the example provided.

The case study covers the development of an example application, meantto mirror a typical student project. It also covers how this was used for teach-ing the students about the testing areas considered. The covered testing areasinclude unit testing, integration testing and UI testing. With these given test-ing areas, the application and an associated learning module was developedfor each area in question. Relevant standards, strategies and approaches wasalso identified for each of these areas.

The thesis also presents important properties to take into considerationwhen developing similar examples in the future, based on the experiences ob-tained during the study. These include needs such as understandable by inex-perienced students, applicability outside the course, adherence to establishedstandards, tools that are simple to use and an architectural structure that al-lows for testing.

Some improvements are also recommended: the students would benefitfrom learning software testing from an early stage of their studies. The contentof the learning modules should also be brought to the students earlier in thecourse, so it can be applied in their projects at an early stage as well.

Further research is also recommended to evaluate the suitability of usingother cloud-based environments instead, and to evaluate the applicability ofthe learning modules for students of varying disabilities.

Keywords: Teaching, DevOps, Test automation, Node.js, Unit testing,Integration testing, UI testing, Continuous Delivery, Continuous Integration,Student project, Cloud services.

ii

SammanfattningAlla utvecklare förstår fördelarna med att testa kod för att garantera dess funk-tionalitet. Dagens industri går i en riktning där testning spelar en central delav design under utveckling av mjukvara. Denna tendens lägger en viss presspå högskolan att lära ut dessa erfarenheter till dess studenter.

På senare tid har kursen II1302 - Projekt och projektmetoder på KungligaTekniskaHögskolan i Kista tagit en stor ansats för att introducera sina studenterinför dessa moderna koncept. Denna uppsats undersöker hur testningsområdeteffektivt kan introduceras till studenterna inom denna kurs, genom att utnyttjaett egengjort exempel som har området i fokus, samt att automatisera dettavia DevOps-verktyg tillhandahållna av molnbaserade tjänster. Dessutom görsäven en ansats för att tilldela ytterligare material som kan användas för att läraut testning av mjukvara i samband med det givna exemplet.

Fallstudien omfattar utvecklingen av en exempelapplikation, som var av-sedd att likna ett typiskt studentprojekt. Den täcker även hur denna användesför att lära ut de betraktade testningsområdena till studenterna. De täckta om-rådena av testning inkluderar enhetstestning, integrationstestning och testningav användargränssnitt. Med dessa givna testningsområden utvecklades bådeapplikationen och dess associerade lärmoduler för vardera testningområde ifråga. Relevanta standarder, strategier och metoder var också identifierade förvardera av dessa områden.

Denna uppsats presenterar även ett antal viktiga egenskaper att hålla i åtan-ke vid utveckling av liknande exempel i framtiden, baserat på erfarenheternafrån studien. Detta inkluderar behov som tillgänglighet för mindre erfarna stu-denter, applicerbarhet utanför själva kursen, tillämpning av etablerade stan-darder, utnyttjande av lättanvända verktyg och en arkitektur som tillåter test-ning.

Några förbättringar föreslås även: studenterna skulle gynnas av att lära sigommjukvarutestning i ett tidigt skede av sina studier. Innehållet i lärmodulernabör även presenteras för studenterna tidigare i kursen för att kunna applicerasi deras projekt.

Ytterligare forskning rekommenderas även för att utvärdera andra lämpligamolnbaserade miljöer, samt för att utvärdera tillämpbarheten av lärmodulernahos studenter med inlärningssvårigheter.

Nyckelord: Undervisning, DevOps, testautomatisering, Node.js, enhets-testing, integrationstestning, UI-testning, ContinuousDelivery, Continuous In-tegration, studentprojekt, molntjänster.

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Research Questions . . . . . . . . . . . . . . . . . . . 31.3 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.5 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . 41.6 Document Outline . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Background Theory 72.1 The MERN-stack . . . . . . . . . . . . . . . . . . . . . . . . 72.2 The Internet of Things . . . . . . . . . . . . . . . . . . . . . 72.3 Cloud Computing . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3.1 Platform as a Service (PaaS) . . . . . . . . . . . . . . 82.4 DevOps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.4.1 Development . . . . . . . . . . . . . . . . . . . . . . 92.4.2 Operations . . . . . . . . . . . . . . . . . . . . . . . 92.4.3 Important DevOps Concepts . . . . . . . . . . . . . . 9

2.5 Software Testing . . . . . . . . . . . . . . . . . . . . . . . . 102.5.1 Testing Areas Considered . . . . . . . . . . . . . . . 102.5.2 Testing Models . . . . . . . . . . . . . . . . . . . . . 11

3 Methodology 153.1 Research Methodologies . . . . . . . . . . . . . . . . . . . . 15

3.1.1 Qualitative & Quantitative Methods . . . . . . . . . . 153.1.2 Inductive & Deductive Approaches . . . . . . . . . . 153.1.3 The Case Study . . . . . . . . . . . . . . . . . . . . . 163.1.4 Bunge’s Method . . . . . . . . . . . . . . . . . . . . 163.1.5 Applied Approach . . . . . . . . . . . . . . . . . . . 17

iii

iv CONTENTS

3.2 Research Method . . . . . . . . . . . . . . . . . . . . . . . . 173.2.1 Literature Study . . . . . . . . . . . . . . . . . . . . 183.2.2 Problem analysis . . . . . . . . . . . . . . . . . . . . 193.2.3 Development . . . . . . . . . . . . . . . . . . . . . . 193.2.4 Gathering Result . . . . . . . . . . . . . . . . . . . . 193.2.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . 19

3.3 Project Method . . . . . . . . . . . . . . . . . . . . . . . . . 193.3.1 Development Iterations . . . . . . . . . . . . . . . . . 233.3.2 Project Requirements . . . . . . . . . . . . . . . . . . 233.3.3 Focus Group . . . . . . . . . . . . . . . . . . . . . . 25

3.4 Technologies Used . . . . . . . . . . . . . . . . . . . . . . . 26

4 Results 274.1 Example Application . . . . . . . . . . . . . . . . . . . . . . 27

4.1.1 Web Application . . . . . . . . . . . . . . . . . . . . 274.1.2 DevOps toolchain . . . . . . . . . . . . . . . . . . . . 28

4.2 Course Modules . . . . . . . . . . . . . . . . . . . . . . . . . 304.2.1 Unit Testing . . . . . . . . . . . . . . . . . . . . . . . 304.2.2 Integration Testing . . . . . . . . . . . . . . . . . . . 314.2.3 User Interface Testing . . . . . . . . . . . . . . . . . 324.2.4 React & Front-end Unit testing . . . . . . . . . . . . . 33

5 Discussion & Conclusion 355.1 Course Observations . . . . . . . . . . . . . . . . . . . . . . 355.2 Answering the Research Questions . . . . . . . . . . . . . . . 36

5.2.1 RQ1: How to create an example application suitablefor learning aboutmodern development environments,and testing? . . . . . . . . . . . . . . . . . . . . . . . 36

5.2.2 RQ2: How to use the example application to teachabout development environments, and testing? . . . . 37

5.3 Portability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.3.1 The example application . . . . . . . . . . . . . . . . 395.3.2 The DevOps toolchain . . . . . . . . . . . . . . . . . 39

5.4 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.4.1 User Group . . . . . . . . . . . . . . . . . . . . . . . 40

5.5 Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.6 Different Aspects . . . . . . . . . . . . . . . . . . . . . . . . 41

5.6.1 Environmental Sustainability . . . . . . . . . . . . . . 415.6.2 Equality and Diversity . . . . . . . . . . . . . . . . . 41

CONTENTS v

5.6.3 Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . 425.7 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . 425.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Bibliography 45

A Additional Material 51A.1 Online Resources . . . . . . . . . . . . . . . . . . . . . . . . 51

Chapter 1

Introduction

An increased pace in competition amongst businesses, along with a steadilychanging need in the market has influenced the way enterprises operate in theIT business[1][2]. Besides not being able to have customers wait for monthsfor a new release anymore, customers also expect to be able to engage with thedevelopers via continuous feedback [2].

This drive towards agile development and pushing the development-sideinto the same space as the operations-side, i.e. DevOps, brings distinct require-ments to the development process. An important part of this process revolvesaround continuously testing the software delivered[2]. In order to optimizethis, the need for automation has also become an essential part of agile devel-opment[3].

While academia is working to meet the market demands of theses compe-tences, there are gaps in knowledge to be filled here, specifically in the areaof testing. How can academia hope to match the market demand for compe-tence of developing tested code, as well as apply this in a modern developmentenvironment which also demands its automation?

1.1 BackgroundThe course Projects and project methods, II1302 is held for second year stu-dents in the Degree Programme in Computer Engineering, TIDAB, and theDegree Programme in Electronics and Computer Engineering, TIEDB, at theRoyal Institute of Technology (or KTH). One of the course’s intended learningoutcomes is to “Apply an appropriate project processes appropriate to the artInformation Technology (IT)”[4]. The participating students are working inproject groups of 3 to 5 people. Each project group is assigned an IT project

1

2 CHAPTER 1. INTRODUCTION

where each project group member is expected to contribute both in the soft-ware development process and project management.

All students are also assigned at least one formal role. For these, a studenthas themain responsibility of project management, stakeholder representation,software development, software architecture, or software testing. The studentsare trained in these roles at seminars during the course, as well as how to applytheir knowledge in their respective project group. In addition to the seminars,the course material is hosted onCanvas, the learningmanagement system usedby KTH.

From spring 2019, it is recommended that the student project is beinghosted on the cloud as well as be designed as an Internet of Things appli-cation, to incentivize learning about these modern concepts. It is because ofthis that the students are being introduced to DevOps and its methods duringthe course, via both the cloud service itself, and by designing applications thattake Internet of Things into account.

1.2 ProblemSince the projects are hosted on a cloud service, new demands of the studentshave emerged. The course’s ambition is that the development process is atleast partly automated, and that the project groups use some of the DevOpsfacilities provided by the cloud platform. Another problem is that the TIDABand TIEDB programmes have no course on the subject of testing [5]. Thismeans that no prior experience of testing can be expected from the students.Because of the new requirements and the fact that the students have very littleexperience in software testing, the teaching of the students in charge of testinghas to be enhanced and adapted to these circumstances. For this, new learningmaterial needs to be developed. The learningmaterial will be a Canvasmoduleconsisting of the following:

• Written material

• Video material

• Seminars with the students in charge of testing

• An example application demonstrating how to implement testing in aproject similar to those in the course.

CHAPTER 1. INTRODUCTION 3

1.2.1 Research Questions1. How to create an example application suitable for learning aboutmodern

development environments, and testing?

2. How to use the example application to teach about development envi-ronments, and testing?

1.3 PurposeThe purpose is to examine how to assist students in increasing their knowledgeabout software testing, automation of software tests, and how to use these in aDevOps context.

This thesis is primarily in the interest of KTH itself, to be able to improvethe quality of the education provided by the institution. It is also beneficialfor the students themselves in several ways, to get them better prepared forworking in large IT projects, and to develop software of higher productionquality.

For employers in the IT industry it would reduce the need of introductionfor newly graduated employees. These new engineers can also in many casescontribute with new knowledge about modern software development practices.

1.4 GoalThe goal is to create an example application and associated learning modulesfor teaching in the course II1302. Each learning module should cover differ-ent important aspects of testing as part of software development. The exampleapplication is aimed to cover these relevant aspects taught in the learning mod-ules.

The concepts taught and the examples provided should be relevant to thecourse II1302, and applicable to the students studies.

Also of importance is that the material should be at an adequate difficultylevel for the students. Again, testing is only part of the course for the students,and the students cannot be expected to have extended knowledge in this.

This thesis addresses these problems directly by identifying relevant areasof knowledge, creating an application and associated learning modules thatcan be used to learn about these relevant concepts.


1.5 DelimitationsThis thesis is a case study, which was conducted within the following primaryconstraints:

• The study will focus on the requirements for the course Projects andProject Methods at KTH.

• The example application will utilize aMERN-like stack via JavaScript.

– CouchDB in particular will be used instead of MongoDB.– Express.– React framework.

– Node.

• The project will be a built-in implementation, using IBM’sCloud Foundryservice.

• The project will use the DevOps tools provided IBM Cloud’s servicespecifically.

– The items must be included in IBM:s Lite plan, so as to not spendany money on the service.

• The study is only concerned with the following areas of software testing:

– Unit testing.

– Integration testing.

– User Interface testing.

1.6 Document OutlineBackground Theory describes the fields and terms that the thesis concerns.

This is to prepare the reader for the remainder of the report.

Methodology details how the thesis was conducted, as well as how the prob-lem and issues were approached. The purpose behind this is to providetransparency behind how the studies in the thesis were conducted.

Results describes the complete process of development, and summarizes theresult achieved.

CHAPTER 1. INTRODUCTION 5

Discussion & Conclusion provides summarized answers to the research ques-tions, and contains a discussion regarding the results arrived at, and themethods used to produce these.

Chapter 2

Background Theory

2.1 The MERN-stackAll web applications are built upon multiple technologies, referred to as a“stack”, as popularized by the LAMP stack1. The MEAN-stack was an earlyalternative that emphasized a shift towards single page applications, that usedlightweight calls to only update parts of a website, rather than the whole page.The stack was composed ofMongoDB for persistent storage, Express as a web-server built on Node, Angular for the front-end framework, and Node for theserver-side runtime environment.[6] Besides the appeal of shifting towardssingle page applications, it also allowed for a single underlying language acrossthe entire web application, i.e. JavaScript[7].

Since theMEAN stack was first introduced, React has since been launchedas an alternative front-end technology, hence the name MERN stack.

The difference between these is that while Angular is an MVC framework,while React is merely a library for building a user interface.[6]

2.2 The Internet of ThingsThe Internet of Things, or IoT, is not a concept with a single clear definition2,since as it became more universal, so too did the definition evolve to moregeneral terms[8].

One of the earliest definitions of IoT defines the term as a concept where a1LAMP is an acronym for Linux, Apache, MySQL and PHP, which are the technologies

that form the bundle, or stack.2See reference [8] for a more complete collection of definitions of Internet of Things.

7

8 CHAPTER 2. BACKGROUND THEORY

variety of things and objects in the environment are able to interact and coop-erate to reach a common goal. These objects can, for example, be RFID-tags,sensors, mobile phones, among other things[9].

Another definition provided by the IEEE Internet Initiativemakes a distinc-tion between small and large environment scenarios, in terms of complexityand management of the “things”. For the smallest scenario, an IoT is said tohave “things” of actuating and programmable capabilities, where their statescan be collected or changed from anywhere at any time[10]. In a larger envi-ronments, the “things” in question are characterized as being owned by differ-ent administrative or managing domains without explicit relationships, withthe issue of scalability becoming a dominant point[10].

While both, and maybe even most, of these definitions vary in terms ofword, they do include several common elements. As defined by Hassan, Khan,and Madani [8], an IoT is made of uniquely identifiable objects, that can bothbe physical or virtual. Further, these objects can capture their surroundingcontext (via sensors) and send this data over the Internet, or even change theirown state, or the state of their surroundings. Also, all of the above, shouldideally function without much, or any at all, human intervention.

2.3 Cloud Computing

2.3.1 Platform as a Service (PaaS)Platform as a Service (or just PaaS) is an abstracted and integrated cloud-based computing environment that supports the development, running, andmanagement of applications. Application components may exist in a cloudenvironment or may integrate with applications managed in private clouds orin data centers[11]. Working in a Paas cloud environment, programmers canfocus on providing proper source code, since operating system, middlewareand runtime usually is provided by the service[12]. Some popular PaaS solu-tions are AWS Elastic Beanstalk, Google App Engine, and Cloud Foundry.

2.4 DevOpsThe term DevOps is a portmanteau of the words Development and Opera-tions [13][1]. It describes several practices to simplify cooperation betweenthose departments.

CHAPTER 2. BACKGROUND THEORY 9

2.4.1 DevelopmentDev area refers to the development of an application or service, via say a soft-ware company. Besides the coding of a product, it also entails the planning ofhow problems are supposed to be solved, in what ways they’re aiming to testthem and building the code itself. For this to be possible, it usually includessome automation, via say testing frameworks for unit and integration testing,to allow for such continuous integration.

2.4.2 OperationsThe Ops area is concerned with the actual deployment of an application orservice, to maybe a customer. Not only does software have to be deployed toa given end-point, say a company server, but it also presumably requires someoperation on the user end. This operation is also monitored in some capac-ity to provide some feedback how the end users are using the software. Thismonitoring can range from anything from a simple crash report to extensivelydetailed user statistics via trackers. The important part here is that some feed-back about the software is fed back to the Dev area so some improvement canbe made.

2.4.3 Important DevOps Concepts

Continuous IntegrationThe principle of continuous integration revolves around integration not oc-curring in a single phase of development, but throughout the entire process.Having developers work in independent branches for long periods of time con-sequently increases the likelihood of introducing breaking changes when thebranches are merged. In large change-sets it becomes hard to identify what’scaused the error to occur in the first place. Continuous integration demandssmaller change-sets on more frequent periods to alleviate these issues, andsystems that implement this usually runs a series of automated tests beforemerging code[3].

Continuous TestingTo continuously integrate, there’s also a demand for ongoing testing and verifi-cation of the code. This naturally results in shorter phases for testing, and alsogives a certain quality guarantee very early in development. Automated testing


is the primary way to allow for this, but it may also simulate an environmentsimilar to the production environment being fed various use cases[13].

Continuous DeliveryClosely related to continuous integration, continuous delivery is what allowsfor continuous integration of software. It is a set of general software engineer-ing principles that allows any changes to a new release to be deployed after thetests for integration have passed[3].

Continuous DeploymentLinked closely to continuous delivery, continuous deployment refers to puttingthat delivery into production. Essentially, continuous delivery ensures that anew change can be deployed, while continuous deployment ensures that theyare deployed into production[3][14].

Continuous Monitoring & FeedbackThese concepts revolves around the feedback Ops returns to Dev. Continuousmonitoring providesmetrics to operations and various stakeholders at differentstages of the delivery of an application[13]. Continuous feedback refers to howthe end user is using the software, which can be either raw data or elaboratefeedback[13].

2.5 Software Testing

2.5.1 Testing Areas Considered

Unit TestingTo test a completed unit it must be proven to satisfy its intended purpose. Thesekinds of tests include some documentation, an objective and an expected result,and are written by the programmer and are executed on a completed unit (ormodule).[15] The units under test are isolated during unit testing. To ensureisolation, external units are simulated. These simulated units are referred to asmocks.[16] In essence, the modules are verified that they do perform the taskexpected of it. In unit testing it is desirable that all possible scenarios for theunit are tested.


Figure 2.1: The test automation pyramid, with tools used in this thesis catego-rized by layer.

Integration TestingWhen individual units have been verified for their intended behaviour, thesetests are held to ensure their behaviour when combined. The purpose behindthis is to identify potential failures when integrating different components,since this is not guaranteed when they work as intended when isolated.[15]

UI TestingAs the user provides a system with input, they expect changes in the interfaceprovided. The testing process for this area verify that the system is respondingto the user in an expected manner.[17]

Acceptance TestingThis testing is carried out by the customer to verify that the software matchesthe customer’s expectations prior to acceptance.[15]

2.5.2 Testing Models

The Test PyramidThe test pyramid[18] (see figure 2.1) is a guideline model describing how toset up and distribute automated tests. As defined by Martin Fowler, softwaredeveloper and co-author of the agile manifesto[19], the model suggests that


Figure 2.2: The V-model illustrated.

more lower level unit tests should run through an application than higher levelGUI-tests[20].

Going upwards in the pyramid, the cost and time consumption a test re-quires increases. The size of each section reflects the proportions of tests thatshould be written. The model indicates that most of the written tests shouldbe unit tests, less tests should be integration tests, and the least amount of testsshould be UI tests [20][21].

Note that the names and amount of layers in the pyramid can vary fromsource to source. Some call what’s identified as Integration in figure 2.1 asthe API layer[22], or the Acceptance or Service layer[20][23]. Others evenidentify several intermediary layers like a Component layer, or even separatelayers like Integration and API as different types of tests[22]. Usually thisdifference comes down to the focus of the application being developed, butregardless of the naming convention, the principle in the pyramid remains thesame. Tests down at the lower levels of functionality are more cost-effectivethan ones at the higher levels.

The V-ModelThe V-Model is a representation of the development process, and how thephases of developing code relates to testing it[24]. This model is illustrated infigure 2.2, where the downward slope represents verification and planning andthe upwards represents validation via testing. At each level a verification phasecorresponds to a validation phase. The horizontal distance between phasesrepresents the time required for successful validation, and the vertical levelbetween phases represents the difference in functionality.

For example, the unit tests validate the detailed design of individual com-ponents. These unit tests are of a lower level of functionality than the integra-


tion tests.

Chapter 3

Methodology

3.1 Research Methodologies

3.1.1 Qualitative & Quantitative MethodsThe choice between a qualitative or a quantitative research approach dependson if a project is numerical in nature or not. Quantitative research requiresmeasurable data to verify the hypothesis in question, and as such requires aquantifiable nature[25]. This kind of research also carries an additional re-quirement of a non-vague hypothesis that’s answerable. Qualitative researchconcerns understanding of meanings and behaviours to reach tentative hypoth-esizes and theories[25]. Despite this added vagueness however, the researchstill may answer the hypothesis sufficiently.

3.1.2 Inductive & Deductive ApproachesThe research approach is what establishes what is true and false[25]. Thereare primarily two methods to determine this, which are inductive and deduc-tive (barring the abductive approach, which uses both of the aforementionedapproaches).

Inductive reasoning formulates theories and propositions with alternate ex-planations from observed patterns. Commonly data is collected via a quali-tative method and afterwards analyzed to arrive at these explanations. Thesepropositions are outcomes based on the behaviour observed, but does carrythe added demand that there must be enough data to be able to derive an ex-planation of the behaviour in question[25].

Deductive reasoning builds upon the testing of verified theories, to either

15

16 CHAPTER 3. METHODOLOGY

verify or falsify a hypothesis. Usually this is done by rigorously testing theseagainst huge data sets, which results in an outcome that’s a generalizationbased on the collected data (i.e. does the proposition hold for our data?)[25].

3.1.3 The Case StudyCase studies are a research strategy that involves using an empirical investiga-tion of a phenomenon in a real life context.[25] In essence, it is research into aspecified area by observing the outcome of the case in question. A case studycan be based on either quantitative or qualitative evidence, as well as a mix ofboth[25].

3.1.4 Bunge’s MethodMario Bunge defines the general scientific strategy as the following sequenceof ten steps:[26]

1. First, a problem has to be identified.

2. Afterwards a choice of approach must be decided upon. This includesmethods included, the goal to be striving towards and background knowl-edge.

3. With an approach given, the problem can be be formulated with thischosen approach. Essentially, the initial raw problem is defined into aproblem that is more defined.

4. With the problem defined, more relevant information can be gathered inpreparation for solving the problem.

5. With sufficient knowledge and a well defined problem, a research plancan be designed.

6. Now that a plan has been established, it’s time to implement it. The planthat’s been decided upon must be kept to during implementation.

7. At this point a given approach has been provided as well as an imple-mentation of it. All that is left to do is to check the outcome of theimplementation.

8. Now, with a result, the candidate solution can be evaluated with regardsto the background knowledge acquired.

CHAPTER 3. METHODOLOGY 17

9. If the solution is satisfactory, the background knowledge can be ex-tended or revised.

10. If the result isn’t satisfactory or found acceptable, the research processmust restart. This can either include replicating the same process or bymodifying a significant component, like the method or hypothesis.

3.1.5 Applied ApproachThis thesis has a number of understated factors that are crucial for the choiceof approach to answer the research questions. Firstly, the thesis is less con-cerned with pure data, rather than an interpretation of a description, since itaims to provide an answer to the “Hows” in the research questions. Becauseof this qualitative methods were used to allow room for descriptions, as wellas the research questions themselves. Secondly, the research questions do nothave an underlying hypothesis, besides the assumption that the problem can besolved. Since conclusions are instead derived from the experiences garneredduring the research process itself, an inductive approach was also deemed ap-propriate.

For these reasons, this thesis is a qualitative case study. The approach ofusing a case study was decided upon as the answers to the research questionsaren’t necessarily universal.

3.2 Research MethodThe problem is defined from the research questions defined in 1.2.1, with thedelimitations in 1.5 and course goals in regard. The research questions firstneed to be divided into sub-questions:

1. How to create an example application suitable for learning aboutmodern development environments, and testing?

(a) Which properties should the application have?

(b) Which tools and technologies should the application use?

2. How to use the example application to teach about development en-vironments, and testing?

(a) What content should be presented?

(b) How should the content of the learning modules be presented?


Figure 3.1: Outline of the research method

To answer the research questions, and its sub-questions, the research pro-cess goes through the following steps shown in 3.1.

3.2.1 Literature StudyInformation about the concerned areas is gathered in regard of the questionsin the previous step. Suitable methods and established standards needed tobe taught to the students, are identified. The resources in the literature studyis mainly scientific articles, books, technical documentation, and video walk-throughs.


3.2.2 Problem analysisThe questions from the problem statement are delimited and further clarifiedusing the information gathered in the literature study. If further questions needto be added, they are also defined here. Hypotheses are drawn from the ques-tions. Requirements for evaluation of the hypotheses are also defined.

3.2.3 DevelopmentIn the case study, the plan set up in the previous stage is implemented. Theplan is strictly followed. The planned functionality is implemented for theexample application, with the requirements from the planning stage in regard.The written material is created and a video walkthrough is recorded. Thevideo and the written material is published on Canvas, and the changes to theexample application are pushed to the GitHub repository. A seminar is beingheld for the students, where information gathered from the literature study ispresented, and put together and delimited in regards of the planning of the casestudy.

3.2.4 Gathering ResultThe resulting modules and the example application are gathered. The authors’observations from the development and creation of the example applicationand the learning modules are discussed and documented. Feedback from An-ders Sjögren, and the students is gathered and documented as well.

3.2.5 EvaluationThe hypotheses from the planning stage are evaluated against the gatheredresult and documentation from the previous stage. The gathered results andthe evaluation of the hypotheses are used to answer the research sub-questions,which in turn are used to answer the research questions.

3.3 Project MethodTo answer the research questions the case study takes an iterative approachbased on Bunge’s definition of the scientific method. The process is illustratedin figure 3.2, and goes through three distinct main phases:


Figure 3.2: Workflow of the development of the application and its associatedlearning modules. The gray elements indicate indicate how the method relatesto Bunge’s method.


Figure 3.3: Development sub-phase, which details the development of the ap-plication and its associated learning modules.


Pre-study This phase encapsulates the definition of the work before a planhas been set. Before the research plan is produced and adhered to, the studyremained in this state. This conforms to steps one to five in the sequencedescribed in section 3.1.4.

Method is a sub-phase that encapsulates the gathering of relevant informa-tion and the definition of the research plan. It is part of the pre-studyas the work hasn’t started yet, but requires that the problem itself hasbeen defined. This pertains to steps four and five described in Bunge’smethod. At the end of this phase we define a hypothesis for the researchquestions and its sub-questions.

Implementation This relates to the implementation of the provided planfrom the pre-study phase. The requirements imposed by each iteration, i.e.testing field, are prioritized and planned before development commences.

Development This sub-phase encapsulates the work of each iteration, whichis illustrated in figure 3.3. It includes the development of the exampleapplication and the learning module, which is done by setting up testsagainst the requirements put forth beforehand, and developing the appli-cation around these.

The application is validated by putting it through these pre-made tests,and development continues if they don’t pass. If they do pass, devel-opment shifts to the learning module, by documenting the results in alab compendium, and expanding this to include the relevant knowledgeneeded to understand the example and concerned testing field. In ad-dition to this, a video walkthrough is recorded as well. If there is timeavailable to extend the example, development on the app is resumed andthe sub-phase restarts. Otherwise, the module is presented at the oralseminar and feedback received from the students are documented as wellfor future work. The oral seminars are a final deadline and signifies theend of one iteration and the start of a new one.

Each iteration goes through the steps included in this sub-phase at leastonce to construct a result, evaluate it and revision it if necessary. Thisalso pertains to steps six to ten described in section 3.1.4.

Conclusion This details the discussion and conclusions drawn regarding theproject and its suitability. In this phase a thorough evaluation of the final resultis done, along with the process itself. Also, more thorough feedback from


the students of II1302 is also acquired to give a different perspective on theprocess. The final result is presented for peer-review and a final revision ismade before submission.

3.3.1 Development IterationsThe development of the test module and the example application is dividedinto four iterations:

1. Unit testing the Server.

2. Integration testing the Server.

3. Acceptance testing the Graphic interface.

4. Unit testing the Client.

In each of these iterations, the rigid structure described in the implementationphase is always followed, and is based on the MoSCoW method as well asBunge’s method. An iteration has a single field in software testing to imple-ment into the example application.

Throughout the iterations, the functionality of the application is expandedupon. In each iteration, tools pertinent to the area at hand are considered, suchas frameworks and other technologies, to form a conclusion on what’s mostsuitable for the user-group.

Each iteration also allows for feedback from the user-group by allowingthem to watch the example take form successfully and how the testing area inquestion is applied to it. This feedback primarily comes from the oral seminarwith the students, where the testing area is explained, the expanded function-ality of the application is described and how the area is implemented into theapplication.

3.3.2 Project Requirements

The MoSCoW methodThe MoScoW method is a technique to manage project needs by ordering therequirements based on their priority to the project as a whole.[27]. Thesepriorities are divided into four main categories:

Must These requirements are critical to the project and must be met. In asense, they are the core functionality that the project lives and dies on.


Figure 3.4: The project management triangle

Should These requirements are important, but not critical to the project. Sincethey aren’t entirely central to the project they have the possibility to beabsent when the project reaches its conclusion, but they still represent alarge chunk of what is expected of the project.

Could These requirements are of less importance, but will still be valuableto the project. Not necessarily neglected, but the project can survivewithout it.

Won’t Not in the scope of the project, but might be valuable in later works.These are important to note, to ensure that the project delivers what’sexpected of it, and more importantly what it will not.

Application of RequirementsAccording to Eklund [28] the Project Management triangle (fig 3.4) dividesthe project constraints in three categories:

• Time

• Cost

• Quality

At least one of the constraints needs to be flexible. In this project, the timeconstraint is fixed: the project needs to be finished in 10 weeks. The Costconstraint is also fixed: The two authors of this thesis has 40 hours per personand week to work on the project. This means that the quality constraint needsto be held flexible. Therefore some project requirements may not be fulfilled.


To ensure that the requirements are prioritized correctly, this project uses theMoSCoW method for requirement priority management (see section 3.3.2):

Must

• Deliver a suitable minimal example application with aDevOps environ-ment and IoT in mind.

• Provide a study on testing modules and the example application for astudent project (this report).

• Provide the students with a document detailing the testing area in ques-tion and how it is implemented in the example application.

• Deliver a video walkthrough explaining the implementation of each test-ing in more detail.

• Host a seminar walking through the testing area and the implementation,to allow for instant feedback from the user group.

Should

• Include a set of frameworks and/or libraries suitable for each testingfield.

• Detail how the frameworks and libraries used in question interacts witheach other.

Could

• Deliver a quiz testing the students knowledge for each testing area.

Won’t

• Deliver testing examples that assumes anything other than JavaScriptbeing used.

3.3.3 Focus GroupA group consisting of the head of testing for each project group will functionas a focus group. Members of the focus group will be interviewed regardingtheir group’s individual implementation of testing, and how they perceive saidimplementation. The information collected from the interviews will be a partof the evaluation of the testing methodologies and frameworks.


3.4 Technologies UsedIBM Cloud: The full stack cloud platform used by the course in question.

Provides the DevOps environment in particular via toolchains.

Toolchain: A set of tools used for development, deployment, and operationtasks[29]. Provides an interface for reading changes in the repository,along with a delivery pipeline.

GitHub: Hosting service for version control of code. Used to host the sourcecode of the example application, along with being a catalyst for thetoolchain via acting on new versions of source code being pushed tothe repository.

Node: JavaScript run-time environment, allowing JavaScript to be run outsideof a browser. Used as the basis for the example application, as well asfor server-side functionality.

Express: Node framework used for building web applications. Provides var-ious utilities for server-side functionality, like allowing HTTP requeststo be handled via API:s. Used for the example application’s back-end,by having an application script act as a webserver.

Jest: A testing framework for Node. It provides a test runner, a built-in asser-tion library, and code coverage functionality.

React: JavaScript library used for building user interfaces andmanaging client-side states. Used for the example application’s front-end.

Chapter 4

Results

4.1 Example ApplicationThe example of an IoT application selected, was a punchclock, and it wascontinuously used as an example in the course.

The system was intended to have two different types of users, employeesand administrators. Employees use an IoT device to allow employees to reg-ister their punch-in and -out times to which are then stored as timestamps ina non-relational database. The administrators however has a web interfacethat renders the registered (i.e. “punched”) timestamps of the employees. Ad-ministrators can either review the total worked hours of all employees, or theregistered timestamps of a single employee, by navigating the interface.

As the application wasn’t intended for commercial use, the focus on ex-tended functionality is not a primary concern, hence administrators only beingable to filter the registered timestamps for single employees. Issues that do notpertain to this functionality or the issue of testing were not considered duringdevelopment, as the end result was intended to be as lightweight as possible.In essence, the application is merely meant to reflect a typical student project,while also providing enough functionality to allow to be tested.

4.1.1 Web ApplicationThe web application runs in a Node runtime environment, with an Express ap-plication as the web server [30]. The server provides the user interface througha React front-end. Application data is fetched from theCloudant database, andis provided to the React front-end via a HTTP API. The server part of the ap-plication is implemented in a layered architecture. The IoT application, which

27

28 CHAPTER 4. RESULTS

Figure 4.1: Structure of the Punch clock example.

will continuously be used as an example in the course, will be a punch clock.This system will use an IoT device to allow employees to register their check-in and check-out time, which are then stored in a database. This system willalso have a web interface to allow employers to review the working periodsof their employees. This approach allows for a web application that incorpo-rates a typical stack of software while also incorporating IoT devices into thestructure (Fig 4.1).

4.1.2 DevOps toolchainThe DevOps tools used for the example application is built upon IBM Cloud’scloud-native toolchain[31]. The toolchain uses integrations for Github, theEclipse Orion Web IDE, the Slack team collaboration service, and a deploy-ment pipeline.

The deployment pipeline (fig 4.2) consists of the five different stages:

Build The source code is fetched from the GitHub repository. The build scriptin the package.json file is run, and dependencies are installed. Thebuild artifacts are compressed, and passed to subsequent stages.

Unit and integration tests Unpacks the artifacts from the Build stage. Theunit tests are run. If all tests passes, the next stage is run.

Deploy to test Runs a script passing the build to a Cloud foundry instancedesignated for running a test version of the application.

GUI tests Unpacks the artifacts from the Build stage. Runs GUI tests againstthe test application. If all tests passes, the final stage is run.

CHAPTER 4. RESULTS 29

Figure 4.2: The deployment pipeline for the Example application


Deploy to production Pushes the build to a newCloud foundry instance. Thisinstance runs the actual fully tested application.

One of the goals was to make the cloud services fit within the free Lite plan. Inthe Lite plan there is a limited amount of memory usage allowed. Therefore,the test application is stopped when the GUI testing is finished, right beforethat deployment of the production version of the application. This does notaffect the performance or quality of the application. Since the test applica-tion is stopped in the beginning of the Deploy to production stage, the testapplication could still be used for manual acceptance testing, if the Deploy toproduction is triggered manually.

4.2 Course ModulesEach of the considered testing fields was designated a course module. Thecontents of these modules are primarily composed of three components, alongwith an oral seminar. These components includes a lab compendium, a videowalkthrough and the slides used during the seminar.

The seminar and lab compendium serve the same purpose, which is to in-troduce the area in question, provide some examples, and show an implemen-tation of the idea through the example application. While the lab compendiumis intended to convey the same information as the oral presentation, albeit inwritten form, it does provide more opportunities for elaboration.

The video walkthrough also serves to introduce the testing area consid-ered, via examples and the settled implementation of the area for the exampleapplication. The walkthrough also visually show how the environment wasset up on the DevOps environment used, to allow the user group to retrace thesteps taken to allow the testing to occur.

While the presentations at the seminars were not recorded, they do stillfollow a set flow, via the presentation slides used. As the slides are an integralpart of the presentations as well, they were also included in the modules toallow a different mean of divulging the information given during the seminar.

4.2.1 Unit TestingFor the first coursemodule the students are given an introduction to the conceptof automated software testing, starting with the individual units. To this end, adescription of the application from both perspectives were given, along with asettled structure and functionality requirements, to establish the context behind


the example. As an additional primer of the application structure, the MERNstack, as well as the components included, was introduced to further motivatethe application structure.

The process of setting up automated tests was also elaborated upon by in-troducing the Test Pyramid, and stressing the importance of lower level testingover high level tests. The design of unit tests was also discussed, as the argu-ment type directly affects the range of cases a unit tests should include. To thisend, the examples given provided instances of “bad data” as well, to illustratethe further size of data to test against.

With the necessary background established, the server-side logic was pre-sented, along with the chosen representative cases used to test it. This devel-oped functionality corresponds to the Calculations unit in figure 4.1, whichincludes a functions that takes a list of timestamps in the form of JSON objectsand determines the total time worked from that list. To test this, an ordinarycase of several timestamps was provided, along with several different cases ofbad data. For the ordinary case, the test merely succeeded if the calculation re-turned the correct result. For the cases of bad data, the function was expectedto throw errors as a result to prove the expected behaviour of the function.These cases included punch-outs occurring before punch-ins, timestamps thatgo outside the boundaries of the 24-hour clock, and a timestamp including anon-representative type of the clocked time. These bad data cases exemplifyillegal data values and constructs, that go outside the boundaries of the func-tion’s purpose, which should be guarded against. Other examples of bad datacovered, but not exemplified include cases of no data, undefined values andcases of incomplete or overflowing data structures.

Finally, the incorporation of automating the unit tests via IBM Cloud:sDevOps Toolchain was described, more specifically via the environments im-plementation of a delivery pipeline. As the stages in the pipeline are primarilycomposed of BASH scripts, a stage could be included that both resolved theNode dependencies required, along with running the tests. A stage deployingthe application to a production environment, provided by the cloud service,was also included.

4.2.2 Integration TestingFor the second module the students were introduced to integration testing,along with the choice of integration strategies. As the application and MERNstructure had already been presented, the seminar moved directly towards thisnext level of testing, in accordance with the Testing Pyramid.


Since the design of the tests didn’t differ much from the unit tests, besidesthe importance of testing results rather than implementation, the students wereinstead introduced to some strategies associated with integration testing. Thestrategies of Big Bang, Top-Down, Bottom-Up and Sandwich were all intro-duced, along with their respective strengths and weaknesses. Alongside these,the use of drivers and stubs in each of the integration strategies was also madeapparent.

As the module was primarily focused on testing of internal components,external components and libraries was considered very lightly. In fact to pushthe performance of the tests, the students were encouraged to hide the interac-tion against external systems, like file systems and databases, during testing viamocks. This reason behind this was to emphasize the importance of keepingthe tests as lightweight as possible, but also to keep track of what’s importantto test. The students were however given advice on how to test against externalcomponents, by only testing via the code that interacts directly with these com-ponents (again, keep the tests as isolated and small as possible). An exampleof an external integration test was also given by testing the HTTP endpointsthrough SuperTest[32], a testing library for Node HTTP servers.

4.2.3 User Interface TestingFor the thirdmodule the students were introduced to the concept of GUI testingand the tools used to automate this. Again, since the seminars assume that thestudents took part in the previous ones, the seminar moved straight along tothe final level of the Testing Pyramid.

As a start a brief overview of the different types of GUI tests were pro-vided. Specifically, the distinctions between functional testing, visual testing,acceptance tests and automated GUI tests was highlighted and their applica-bility. While the focus of the module was on scripting automated GUI tests,alternatives like manual tests and record/playback automation was also as analternative. The reason for this was to further illustrate the strengths and weak-nesses of these approaches, specifically compared to scripted tests.

With this background the Selenium framework itself was presented to il-lustrate scripted tests. Utilizing the framework’s official JavaScript API, userinput was simulated by selecting HTML elements via CSS selectors and uti-lizing relevant functions provided by the API to “click” these. Since the stu-dents had been encouraged to use JavaScript from the start of the course, onlythe most common JavaScript implementations of Selenium was presented andused during the seminar.


Besides this, students were also informed about several best practices whentesting GUI, as well as common pitfalls when testing the GUI. The importanceof testing against “bad input” was especially stressed, along with a short recapof malicious input via common attack techniques such as SQL injection[33]and Cross-site scripting[33].

The video walkthrough, besides covering the aforementioned areas, alsoprovided an example of how to create Selenium tests from Use Cases, us-ing Jest. The incorporation of Selenium’s JavaScript API and Jest was alsocovered here, as Selenium itself doesn’t facilitate assertions when comparingduring tests (not a problem since Jest could fill this gap instead).

4.2.4 React & Front-end Unit testingThe fourth module covers the concept of front-end frameworks. For this mod-ule no seminar was held, nor was a compendium made. Instead the relevanttools was merely linked to along with their own tutorials. The main focus ofthis module revolved around the React framework and how to create unit testsfor a React module. Common tools for layout, building, and type checkingwere also referred to.

The reason behind the lack of a seminar came down to the fact that veryfew groups had implemented React into their applications. Also at this point intime in the course, the students were in the concluding phases of their projects,so the knowledge imparted wouldn’t be able to be applied. Because of this, itwas decided that React would only be mentioned, as another seminar wouldn’tserve much of a purpose for the projects.

Chapter 5

Discussion & Conclusion

5.1 Course ObservationsAs the student groups were subjected to the learning modules, several diffi-culties were encountered by them during the progress of the course. The fol-lowing observations are the results of informal sessions with the students andtheir projects, and are not the result of a quantitative analysis. Despite this, theconcerns raised are applicable to the course’s learning modules. The fourteengroups in question consisted of three to four members each.

• Unit tests were sparely used universally. The student groups simplydidn’t prioritize testing to the same extent as the rest of the course’scontents.

• While most of the student projects were written in entirely in JavaScriptusing a Node back-end, at least three groups didn’t. These resorted tothe languages Python, PHP, and Scala. As such, they had to resort totesting frameworks outside the authors recommendations.

• Some groups used back-end frameworks for their projects. Frameworksused were Flask, Play, and Laravel.

• Many of the projects featured architectures that complicated implemen-tation of testing. These flat unstructured projects didn’t facilitate unit-or integration testing, as they lacked components to test. For these, au-tomated UI tests were simply the only resort.

• Several student groups preferred using relational databases, such asDB2,over the non-relational alternative Cloudant.

35

36 CHAPTER 5. DISCUSSION & CONCLUSION

• Very few groups used React, or any front-end framework at all. As such,the final course module was fairly inapplicable for their projects.

5.2 Answering the Research Questions

5.2.1 RQ1: How to create an example application suit-able for learning about modern development en-vironments, and testing?

Which properties should the application have?The application should be similar to the group projects in the course Ourexample application is built on the MERN-like stack, consisting of Node.js,Express, React, and Cloudant, as this stack was considered to be the standardfor this course. A majority of the student projects used Node.js and Express.The choice of database and front-end solution did however differ more. A fewgroups used React, but other groups were using server-side rendered pages,using very little application logic on the client. Many groups also had difficul-ties working with Cloudant, and chose to work with IBM:s DB2, a relationalSQL database.

The application should be structured in away thatmakes the code testableTo make the example application easy to unit test, we chose a modular ap-proach. The application server follows an object oriented paradigm, and usesa layered Model-View-Controller (MVC) architecture. The Express front-endwas mainly used for handling HTTP requests, and making calls to a controllerobject. In the first seminar, one of the students asked about testing of the HTTPinterface. In this student’s case, their application was heavily coupled, and theonly provided interface for testing was the HTTP endpoints, which directly ledto the introduction of API testing in the second seminar.

The application should meet standards and best practices Despite thesmall nature of the application itself, there are still several standards in ef-fect. The application uses a layered MVC-like architecture[34][35], and ad-heres to object oriented standards such as separation of concerns [36], lowcoupling[35], and high cohesion[35].

CHAPTER 5. DISCUSSION & CONCLUSION 37

The application should be easy to understand for a second year studentat the TIDAB or TIEDB programme In the example application the focuslies on teaching rather than functionality. The module for time calculation, forexample, is using integers as a time unit. The reason behind this decision wassolely to avoid distraction from the main functionality.

Which tools and technologies should the applicationuse?The tools should be simple to use One of the main reasons that the Jestframework was chosen was its simplicity. An installation of Jest includes atesting framework, a test runner, and code coverage tools. Since Jest followsthe Convention over configuration, very little or no configuration is required.An Express server can be run with very few lines of code. The exampleapplication uses a MERN-like stack, and the entire application is written inJavaScript.

The application should use technologies that are useful outside of thescope of II1302 React is a very popular framework [37] and is widely usedby companies as Facebook, Instagram, Netflix, and the New York Times. [38]Also worth mentioning is that one can create Native mobile applications foriOS and Android with React syntax by using React Native [39]. The Jestframework is one of the most popular testing frameworks [40][41], and is aswell as React developed by Facebook [42]. In a survey conducted by Katalonand KMS in 2018, 84% of software testers answered that they were using, orhave used Selenium [43]. WebDriverJs is the official JavaScript implementa-tion of Selenium [44], and was therefore chosen for GUI testing of the exampleapplication.

5.2.2 RQ2: How to use the example application to teachabout development environments, and testing?

What content should be presented?Theoretical background The students need a proper introduction of thesubjects covered, to learn not only how, but also why, and when the testingtypes are being used. In each seminar the students are introduced to the typeof testing covered, and which problems that are being addressed. The intro-duction also covers the part of the example application that is concerned in


the particular seminar. The students are also taught how the covered test typesrelates to known models, such as the test pyramid and the V-model.

Tools used The Learning modules themselves cover all the tools used inthe example application, along with other similar tools intended for differentframeworks.

In the first module the students are introduced to the MERN stack, Jest,and the basics of test automation.

The second module introduces few new tools, but does include an intro-duction to API testing using Supertest.

The third module focuses on the Selenium framework, as well as intro-duces the students to WebDriverJs, and other popular GUI testing tools forJavaScript.

The fourth module provides links to information about React and otherfront-end frameworks. Links are also provided to some of the most commontools used for building and testing a client-side application, such as Webpack,Babel, and Enzyme.

Testing strategies The relevant strategies within each of the covered fieldsof testing needs to be presented in order for the students to understand howto apply the tests in their specific projects. The learning module mentions themost common strategies used in each of the covered test types.

Howshould the content of the learningmodules be pre-sented?Seminars At the seminars an oral presentation is given to the students. Anoverview of the course module is presented. The visual material mostly con-sisted of slides, but some live coding examples and demonstrations of relevantweb pages were also shown during the seminar. This also served as an oppor-tunity for the students to ask questions directly to the teachers. Demonstrationsof various strategies in automation of tests, both locally and in the cloud werealso given.

Source code Source code showing how to implement different testing strate-gies should be provided, to give the students a concrete view of the theorytaught in the modules. In the learning modules code examples are provided,not only in the example application, but also in the compendiums and videowalkthroughs, which are focused on implementing the taught material as code.


Code examples are also shown in the seminars, in the form of live demos.The source code for the example application and the recommended DevOpstoolchain are also available in public Github repositories.

Lab compendium The written compendium was both intended as a substi-tute for the ones who couldn’t attend the seminar, as well as a reminder ofthe many concepts introduced and their connection during the seminar. Moreimportantly, the written compendium was a chance for the students to haveready-made explained examples available, to be able to apply them in theirown projects.

Video walk-through The video walkthrough was intended as a different al-ternative to the compendiums and seminars. The spoken nature allowed fora less formal explanation of the concepts, and the recording of a desktop en-vironment could highlight the meaning behind individual lines of code moredirectly.

5.3 Portability

5.3.1 The example applicationThe application is designed with a Node runtime provided by a PaaS in mind.This does not however bind the application to IBM Cloud, since similar Noderuntimes are provided by several cloud service providers. While the exam-ple application is not designed for any specific back-end framework, it shouldbe easy to implement in most of the MVC frameworks due to the simple im-plementation of that design pattern in the application. The use of Cloudantdoes bind the application to IBM Cloud to a certain degree, since Cloudantis an IBM service, but the Node client library for Cloudant is a wrapper forNano, which is a library for Apache’s CouchDB. All the commands used inthe application are Nano commands, except for those used for establishing thedatabase connection. One configuration file, manifest.yml, contains theIBM Cloud-specific settings. The filename is quite generic and may conflictwith configuration files for other cloud services.

5.3.2 The DevOps toolchainThe source code for the toolchain is written according to Open Toolchain’sYAML standard, and needs to be modified if another cloud solution is used.


Most of the tasks performed in the deployment pipeline, however, are im-plemented as bash scripts, and should easily be ported to any cloud servicerunning a Linux environment. The deployment scripts are specific to CloudFoundry, and needs to be modified if another deployment method is used. Thememory optimization made in the deployment pipeline, is done with the Liteplan in mind. This does not bind the toolchain to IBM Cloud per se, but mightbe unnecessary if another cloud service is used, or if the conditions for the Liteplan are changed.

5.4 ReliabilityIn qualitative research, the subject of “reliability” refers to the extent that ameasurement will yield the same results for repeated attempts [45][25]. Inessence, will the same result be produced, if the same process is repeated byanother party at another time?

5.4.1 User GroupThere is a significant difference of knowledge and experience amongst thestudents themselves, which is difficult to account for. Besides this, there arealso big variations in the student projects regarding architecture, functionality,programming languages used, and the choice of tools and frameworks. In thisproject the student interactions has been mainly with a group of test leaders,consisting of fourteen students in total. This small group of students makes itdifficult to achieve a representative selection, as even the students in charge oftesting for their respective projects, may not be raising the concerns of theirpeers. Studies on future courses needs to be done to improve the reliability.

5.5 ValidityBesides the concern of reliability, the subject of validity is also crucial to es-tablishing a degree of objectivity in qualitative studies[45]. “Validity” refersto the extent that a given procedure has provided a correct answer. Essentially,in the context of the research questions, has the research method answered thequestions posed?

The creation of the example application itself, while threadbare, does pro-vide an example suitable for teaching about testing. While the application is


intended to work against IoT devices however, it doesn’t carry this functional-ity on its own.

As the example for teaching about testing was also incorporated into thestudy, as well as the course itself, this question was also resolved, because ofit was central component of the case study.

5.6 Different Aspects

5.6.1 Environmental SustainabilityAs this project was directly bound to various services outside the participantsdirect control, it’s difficult to gauge in what way the outcome will affect theenvironment as a whole. Utilizing cloud-based services like IBM Cloud re-moves the control over the hardware itself, leaving only the efficiency of thesoftware as a potential factor. Considering that the produced software wasintended to be as small as possible from the start for convenience, there arevery few other factors within control that can impact the environment, besidesthe cloud itself. It should be considered whether other cloud-based servicescan allow for smaller carbon footprints, than the one provided by IBM. If thereare more energy-efficient services available, the potential for utilizing theseinstead in the course should be considered.

5.6.2 Equality and DiversityWhile the learning modules are intended to be used by students of varyingbackgrounds, there is very little conscious effort to accommodate for any po-tential disabilities. The main reason behind this comes down to the fact thatthe authors in question lack the relevant knowledge in this area.

An argument can be made that the video walkthroughs themselves can beused as a substitute for the compendiums, as these videos can be rewound andrepeated at the student’s leisure. In this sense the material has the potential toaccommodate for people with learning disabilities, to an extent. The materialdoes however assume that any future student groups will have a participantthat understands Swedish. While the course is held in Swedish, this severelyhampers the ability to use the material for anyone that doesn’t speak the lan-guage.

The authors of this thesis therefore recommend that the material producedis evaluated in regards to how well it accommodates for students with learningand reading difficulties, if it is to be reused in the future.


5.6.3 EthicsThe project made a concerted effort to keep the results within the Lite plan ofthe IBM Cloud environment. Despite the possibility of signing up and usingthe relevant tools provided by IBM Cloud for free, it did put a sharp restrictionon the amount of web services that could be kept online at any point in time.As a result, the end result is able to be run without signing up a credit cardwith an IBM ID.

5.7 Future WorksAs this work only covers IBM Cloud Services, a study comparing other cloudservices more directly would also be of interest to the institution. Other ar-chitectures for the example application, for example microservices could alsobe used. In the CI/CD pipeline used in this work, the application is deployedto Cloud Foundry. Other deployment techniques, like Kubernetes and Dockercontainers could also be explored. The IoT part of the example application wasnever implemented. Implementation, testing, and deployment of IoT devices,is an interesting field to explore in future works. In this work only the codingand deployment part of DevOps were covered. Other aspects of DevOps, forexample those regarding continuous planning and feedback could be useful inthe II1302 course.

5.8 ConclusionAs the purpose of the thesis lied in teaching the students about software test-ing in concrete detail, the example and learning modules may be consideredsufficient. Despite the absence of equivalent material for the unit testing of thefront end, the areas covered encapsulated the whole of the test pyramid. De-spite this however, as detailed in the discussion, several student groups hadn’tapplied these principles in their own projects.

While this can be explained as the result of a very diverse user group, itdoes imply that the material may have been in conflict with the other require-ments of the course. The content of the learning modules was taught to the stu-dents in parallel to the development of the student projects. The students hadalready made critical design choices before the content was brought to them,which in some cases made it difficult to implement the techniques taught inthe modules. If the seminars were held earlier in the course, it would probablybe easier for the students to set up automated tests properly.


The students in general, as well as the authors had very little knowledge andexperience of software testing. Because of that, a lot of time had to be spenton covering the very basics of testing. The introduction of the area of softwaretesting may be ill suited for a course that implements so many other areas ofsoftware development. If these concepts were taught to students before II1302,either in the programming courses prior to II1302, or in a separate course, theII1302 course modules could dive deeper into the subject of test automationand DevOps, and the students might be more likely to include tests in theirprojects.

Bibliography

[1] B. S. Farroha and D. L. Farroha. “A Framework for Managing MissionNeeds, Compliance, and Trust in the DevOps Environment”. In: 2014IEEE Military Communications Conference. Oct. 2014, pp. 288–293.doi: 10.1109/MILCOM.2014.54.

[2] ManishVirmani. “UnderstandingDevOps&bridging the gap from con-tinuous integration to continuous delivery”. eng. In: Fifth InternationalConference on the Innovative Computing Technology (INTECH 2015).IEEE, 2015, pp. 78–82. isbn: 9781467375504.

[3] Jennifer Davis and Katherine Daniels. Effective DevOps : Building aCulture of Collaboration, Affinity, and Tooling at Scale. O’Reilly Me-dia, Incorporated, 2016.

[4] II1302 Projects & Project Methods 7,5 credits |KTH. [Online; accessed28.Mar. 2019].Mar. 2019.url:https://www.kth.se/student/kurser/kurs/II1302?l=en.

[5] Degree Programme inComputer Engineering (TIDAB), Programme syl-labus for studies starting in autumn 2017, Appendix 1: Course list |KTH. [Online; accessed 10. Apr. 2019]. Apr. 2019. url: https://www.kth.se/student/kurser/program/TIDAB/20172/kurslista?l=en.

[6] Vasan Subramanian. Pro MERN Stack: Full Stack Web App Develop-ment withMongo, Express, React, andNode. eng. Berkeley, CA:Apress,2017. isbn: 9781484226520.

[7] AmosQ. Haviv.MEANWebDevelopment. eng. Packt Publishing, 2014.isbn: 1783983280.

[8] Qusay F. Hassan, Atta ur Rehman Khan, and Sajjad A. Madani. Internetof Things. eng. 1st ed. Taylor & Francis, 2017. isbn: 1-4987-7853-4.

45

https://doi.org/10.1109/MILCOM.2014.54

https://www.kth.se/student/kurser/kurs/II1302?l=en

https://www.kth.se/student/kurser/kurs/II1302?l=en

https://www.kth.se/student/kurser/program/TIDAB/20172/kurslista?l=en



46 BIBLIOGRAPHY

[9] Luigi Atzori, Antonio Iera, and Giacomo Morabito. “The Internet ofThings: A survey”. eng. In:Computer Networks 54.15 (2010), pp. 2787–2805. issn: 1389-1286.

[10] Towards a Definition of the Internet of Things (IoT). [Online; accessed12. Apr. 2019]. May 2015. url: https : / / iot . ieee . org /images/files/pdf/IEEE_IoT_Towards_Definition_Internet_of_Things_Revision1_27MAY15.pdf.

[11] What Is Platform as a Service (PaaS) in Cloud Computing? - dum-mies. [Online; accessed 13. Aug. 2019]. Aug. 2019. url: https ://www.dummies.com/programming/cloud-computing/hybrid- cloud/what- is- platform- as- a- service-paas-in-cloud-computing.

[12] “Cloud-Based Development Environments”. eng. In: Encyclopedia ofCloudComputing. Chichester, UK: JohnWiley&Sons, Ltd, 2016. isbn:9781118821978.

[13] Sanjeev Sharma. DevOps for dummies. John Wiley & Sons, 2014.

[14] Understanding DevOps - Part 6: Continuous Deployment vs Continu-ousDelivery. [Online; accessed 03. Apr. 2019]. Oct. 2013.url:https://sdarchitect.blog/2013/10/16/understanding-devops-part-6-continuous-deployment/.

[15] Gerard O’Regan. Introduction to Software Quality. eng. 2014th ed. Un-dergraduate Topics in Computer Science. Cham: Springer InternationalPublishing, 2014. isbn: 9783319061054.

[16] Paul Hamill.Unit test frameworks. eng. 1st ed.. 2005. isbn: 1-306-81647-5.

[17] Daniel Galin. Software quality : concepts and practice. eng. 2017, pp. 255–317. isbn: 1-5231-2159-9.

[18] Mike Cohn. Succeeding with agile: software development using Scrum.Pearson Education, 2010.

[19] Mike Beedle et al.Manifesto for Agile Software Development. [Online;accessed 23. Aug. 2019]. 2001.url:https://agilemanifesto.org/.

[20] Martin Fowler. bliki: TestPyramid. [Online; accessed 27. May 2019].May 2019. url: https://www.martinfowler.com/bliki/TestPyramid.html.

https://iot.ieee.org/images/files/pdf/IEEE_IoT_Towards_Definition_Internet_of_Things_Revision1_27MAY15.pdf



https://www.dummies.com/programming/cloud-computing/hybrid-cloud/what-is-platform-as-a-service-paas-in-cloud-computing




https://sdarchitect.blog/2013/10/16/understanding-devops-part-6-continuous-deployment/



https://agilemanifesto.org/

https://agilemanifesto.org/

https://www.martinfowler.com/bliki/TestPyramid.html

https://www.martinfowler.com/bliki/TestPyramid.html

BIBLIOGRAPHY 47

[21] ChuckGreb. Test Pyramid. [Online; accessed 25. Apr. 2019]. Dec. 2016.url: https://medium.com/android- testing- daily/the-3-tiers-of-the-android-test-pyramid-c1211b359acd.

[22] Selenium Framework | Choose Automation Solution - 2. [Online; ac-cessed 04. Aug 2019]. 2017.url:http://www.seleniumframework.com/decision-models/choose-automation-solution-2/.

[23] Andrei Contan, Catalin Dehelean, and Liviu Miclea. “Test automationpyramid from theory to practice”. eng. In: 2018 IEEE InternationalConference on Automation, Quality and Testing, Robotics (AQTR). IEEE,2018, pp. 1–5. isbn: 9781538622049.

[24] Model-driven testing using the UML testing profile. eng. Berliln ; NewYork: Springer, 2008. isbn: 1-281-35283-7.

[25] Anne Håkansson. “Portal of Research Methods and Methodologies forResearch Projects andDegree Projects”. eng. In: KTH, Skolan för informations-och kommunikationsteknik (ICT), Programvaruteknik ochDatorsystem,SCS. Las Vegas USA: CSREA Press U.S.A, 2013, pp. 67–73. isbn: 1-60132-243-7.

[26] Mario Augusto Bunge. Epistemology & Methodology I: Exploring theWorld. eng. Bunge,Mario, 1919- Treatise on basic philosophy ; Volume5. 1983. isbn: 94-009-7027-7.

[27] TheDSDMAgile Project Framework (2014Onwards). [Online; accessed3. Apr. 2019]. Nov. 2015. url: https://www.agilebusiness.org/content/moscow-prioritisation.

[28] Sven Eklund. Arbeta i projekt: individen, gruppen, ledaren. Studentlit-teratur, 2010.

[29] DevOps toolchains - IBM Cloud Garage. [Online; accessed 20. May2019].May 2019.url:https://www.ibm.com/cloud/garage/toolchains.

[30] AndersGawell andAntonKallin. anga13/punchclock. [Online; accessed23. Aug. 2019]. Aug. 2019.url:https://github.com/anga13/punchclock.

[31] AndersGawell andAntonKallin. anga13/II1302-cloud-native-toolchain.[Online; accessed 23. Aug. 2019]. Aug. 2019.url:https://github.com/anga13/II1302-cloud-native-toolchain.

https://medium.com/android-testing-daily/the-3-tiers-of-the-android-test-pyramid-c1211b359acd

https://medium.com/android-testing-daily/the-3-tiers-of-the-android-test-pyramid-c1211b359acd

http://www.seleniumframework.com/decision-models/choose-automation-solution-2/



https://www.agilebusiness.org/content/moscow-prioritisation

https://www.agilebusiness.org/content/moscow-prioritisation

https://www.ibm.com/cloud/garage/toolchains

https://www.ibm.com/cloud/garage/toolchains

https://github.com/anga13/punchclock


https://github.com/anga13/II1302-cloud-native-toolchain


48 BIBLIOGRAPHY

[32] visionmedia/supertest. [Online; accessed 22. Aug. 2019]. Aug. 2019.url: https://github.com/visionmedia/supertest.

[33] Mike Shema. Seven deadliest web application attacks. eng. Syngressseven deadliest attacks series. Amsterdam ; Boston: Syngress, 2010.isbn: 1-282-54173-0.

[34] “The MVC Pattern”. eng. In: Pro ASP.NET MVC 3 Framework. ThirdEdition. Berkeley, CA: Apress, 2011, pp. 63–88. isbn: 9781430234043.

[35] Craig Larman. Applying UML and patterns : an introduction to object-oriented analysis and design and iterative development. eng. 3. ed.. Up-per Saddle River, N.J.: Prentice Hall PTR, 2005. isbn: 0-13-148906-2alk.

[36] Walter L. Hürsch and Cristina Videira Lopes. Separation of Concerns.Tech. rep. Northeastern University Boston, Feb. 1995.

[37] StackOverflowDeveloper Survey 2019. [Online; accessed 29.May 2019].May 2019. url: https://insights.stackoverflow.com/survey/2019.

[38] Matt Warcholinski. “10 Famous Apps Using ReactJS Nowadays”. In:Blog Brainhub.eu (May 2017). url: https : / / brainhub . eu/blog/10-famous-apps-using-reactjs-nowadays.

[39] React Native · A framework for building native apps using React. [On-line; accessed 29.May 2019].May 2019.url:https://facebook.github.io/react-native.

[40] jest. [Online; accessed 29. May 2019]. May 2019. url: https://www.npmjs.com/package/jest.

[41] facebook/jest. [Online; accessed 29.May 2019].May 2019.url:https://github.com/facebook/jest/network/dependents.

[42] Facebook Open Source. [Online; accessed 29. May 2019]. May 2019.url: https://opensource.facebook.com.

[43] [Infographic] Challenges in Test Automation: Survey Key Results |Kat-alon Studio. [Online; accessed 29.May 2019]. June 2018.url:https://www.katalon.com/resources-center/blog/infographic-challenges-test-automation.

[44] WebDriverJS -Web application Automation tutorials. [Online; accessed29. May 2019]. May 2019. url: http://www.webdriverjs.com.

https://github.com/visionmedia/supertest

https://insights.stackoverflow.com/survey/2019

https://insights.stackoverflow.com/survey/2019

https://brainhub.eu/blog/10-famous-apps-using-reactjs-nowadays

https://brainhub.eu/blog/10-famous-apps-using-reactjs-nowadays

https://facebook.github.io/react-native

https://facebook.github.io/react-native

https://www.npmjs.com/package/jest

https://www.npmjs.com/package/jest

https://github.com/facebook/jest/network/dependents

https://github.com/facebook/jest/network/dependents

https://opensource.facebook.com

https://www.katalon.com/resources-center/blog/infographic-challenges-test-automation



http://www.webdriverjs.com

http://www.webdriverjs.com

BIBLIOGRAPHY 49

[45] Jerome Kirk. Reliability and validity in qualitative research. eng. Qual-itative research methods ; v. 1. Newbury Park, CA: Sage Publications,1986. isbn: 1-4522-1227-9.

50 BIBLIOGRAPHY

Appendix A

Additional Material

A.1 Online Resources1. "anga13/II1302-cloud-native-toolchain". Code repository for customized

pipeline.https://github.com/anga13/II1302-cloud-native-toolchain

2. "anga13/punchclock". Code repository for the produced example appli-cation.https://github.com/anga13/punchclock

3. Canvas learning modules. Requires access to II1302 through the Can-vas learning system:

(a) "1 - Enhetstestning". Learningmodule for seminar concerning unittesting.https://kth.instructure.com/courses/7679/pages/1-enhetstestning?module_item_id=137456

(b) "2 - Integrationstestning". Learning module for seminar concern-ing integration testing.https://kth.instructure.com/courses/7679/pages/2-integrationstestning?module_item_id=138597

(c) "3 - GUI-testning". Learning module for seminar concerning userinterface testing.https://kth.instructure.com/courses/7679/pages/3-gui-testning?module_item_id=138599

(d) "4 - React och test av frontendramverk". Learning module for sem-inar concerning user interface testing.

51



https://kth.instructure.com/courses/7679/pages/1-enhetstestning?module_item_id=137456

https://kth.instructure.com/courses/7679/pages/1-enhetstestning?module_item_id=137456

https://kth.instructure.com/courses/7679/pages/2-integrationstestning?module_item_id=138597

https://kth.instructure.com/courses/7679/pages/2-integrationstestning?module_item_id=138597

https://kth.instructure.com/courses/7679/pages/3-gui-testning?module_item_id=138599

https://kth.instructure.com/courses/7679/pages/3-gui-testning?module_item_id=138599

52 APPENDIX A. ADDITIONAL MATERIAL

https://kth.instructure.com/courses/7679/pages/4-react-och-test-av-frontendramverk?module_item_id=138604




II1302 - Projekt & Projektmetoder Seminarie 1 - Att Enhetstesta en Server

Seminarie 1Att Enhetstesta en Server

Anders Gawell, Anton Kallin

30 augusti 2019

1 Introduktion

I denna labbkompendium kommer omradet testning inom mjukvaruutveckling att intro-duceras, samt delomradet enhetstestning. Till detta andamal har aven de forsta stegentagits i utvecklingen av en applikation for att illustrera principerna kring testning appli-ceras.

1.1 Krav & Begransningar

Lasaren forvantas ha grundlaggande kunskap i programmering, samt viss erfarenhetav att utveckla storre system. Det stalls inga forvantningar pa lasarens kunskap inomomradet testning i detta kompendium. Exempel i detta kompendium kommer halla sigtill en applikation som ar utvecklad med JavaScript. Exempelapplikationen kommer integenomgas i detalj, endast de delarna som ar vasentliga for detta seminarie. Lasaren kansatta sig in i funktionaliteten i mer detalj via den publika GitHub-repositoriet som ar bi-fogat[1] efter behov, men detta ar inte ett krav. Det ar dessutom inte ett krav att lasarenforstar applikationen i forvag. Principerna som beskrivs kring testning och utvecklingar dock universella och ar inte begransade till varken spraket eller applikationen sjalv.Lasaren forvantas ta till sig av principerna som gas igenom och anvanda exemplet somen startpunkt for att forsta de abstrakta koncepten och teknologierna.

1.2 Mal

• Introducera omradet testning inom mjukvara.

• Beskriva vad enhetstestning innebar och hur det gar till.

• Introducera exempelapplikationen som testmiljon kommer bildas i for i detta do-kument och de senare seminarierna.

• Detaljera den utvecklade applikationen pa serversidan.

• Visa en implementation av enhetstestning pa ovan namnda applikationen.

https://github.com/antonkal/II1302-S1 1


1.3 Teknologier

• Node.js

• Express.js

• Jest

• Bash-terminal

2 Bakgrund

2.1 Testning av Mjukvara

Testning handlar om att satta en produkt i bruk for att utvardera om den fungerar ratt.Testning av mjukvara gar ut pa att bade verifiera att koden i fraga fungerar ratt, medansman samtidigt vill hitta sa manga fel som mojligt.

2.1.1 Enhetstestning

Enhetstestning gar ut pa att testa individuella metoder och objekt tillhorande proble-met. Malet med denna typ av testning ar att visa att koden tjanar designen. Enhets-testning sjalv gors isolerat pa de minsta modulerna i programmet och skall idealt tackaupp alla olika scenarion modulen kan genomga. Testernas operationer ska ske pa ettsadant satt att den testade enhetens alla villkorssatser provas. Tag koden i figur 1 som

func t i on and ( b1 , b2 ) {i f ( b1 && b2 ) return true ;r e turn f a l s e ;

}

f unc t i on t e s t e d ( ) {conso l e . l og ( and ( true , t rue ) === true ) ;con so l e . l og ( and ( true , f a l s e ) === f a l s e ) ;c on so l e . l og ( and ( f a l s e , t rue ) === f a l s e ) ;c on so l e . l og ( and ( f a l s e , f a l s e ) === f a l s e ) ;c on so l e . l og ( and ( [ ] , f a l s e ) === f a l s e ) ;c on so l e . l og ( and(−1 , {}) === f a l s e ) ;

}

Figur 1: Ett alternativ for att gora en AND-jamforelse i JavaScript, samt att testa denna.

exempel. Det enda som gors ar en kontroll av tva varden, men den reagerar olika be-roende pa dess input. For att garantera funktionaliteten hos detta kodstycke jamforsdess producerade resultat emot dess avsedda. Dessutom gors ytterligare kontroller av



returvarde for oavsedda argumenttyper som tomma listor och objekt. Notera att dettaar inte nodvandigtvis ett onskat beteende for boolska operationer. Det kan vara merpassande att jamforelse med icke-boolska varden returnerar nagot annat kontrollvarde(som null) eller ett undantag, men detta ar en implementationsfraga.

2.1.2 Testfall

I figur 1 sa har koden testats efter alla antagbara varden i den avsedda domanen (d.v.s.true och false), samt nagra fall for oavsiktliga argument. Inom enhetstestning ar dettaforedraget, eftersom alla mojliga fall vill testas, men det ar ofta varken nodvandigt ellerens mojligt. Med storre domaner utanfor boolska varden ar det praktiskt omojligt atttesta alla varden, eftersom testfallet blir for stort och tar lang tid. Aven for en doman somalla positiva heltal blir antalet fall otympliga att testa (d.v.s.∞). Istallet for att testa allafall nojer man sig ofta med ett antal gransfall, vilket ar representativa situationer somkod kan tankas anropas med. En representativ situation beror framst pa funktionaliteteni fraga, men primart vill man testa efter god och felaktig data.

Fall med god data inkluderar situationer da datastrukturen ar den som avsetts forfunktionen. Detta kan vara fall som testar inom granserna av funktionen, typiska fallman kan forvantas fa och villkor for kontroll-strukturer. I alla dessa fall har funktionenmottagit data som den ar avsedd for och agerar pa ett typiskt satt. For en lista, t.ex.,sa ar granserna da listan ar tom eller bestar av enstaka element och det typiska fallet arnar den har mer an ett element. Villkoret for kontroll-strukturen utgar ocksa ifran attden mottagna datan ar en lista, vilket ocksa maste testas efter.

Fall med felaktig data ar situationer som funktionen inte skall hantera. Detta inklu-derar fall da datan ar otillaten, det finns ingen data alls eller oinitialiserade variabler,samt att det anges for lite eller for mycket data. For exemplet med listan sa ar null ochundefined exempel pa ingen data, vilket ger problem om vi forvantar oss att arbetamed en lista. Om man kraver en viss langd pa listan sa ar detta ocksa fall att testa efter(man kanske endast vill ha ett jamnt antal element). Aven att mata en funktion med endatastruktur som inte ar avsedd ar ett fall att testa efter (t.ex. ett tal istallet for listan).

Notera att for koden i figur 1 sa finns de olika gransfallen representaterade. Det finnsendast ett fall vid anrop med boolska varden som skall ge true som resultat, resterandeskall ge false. Daremot testas aven risken for otillaten data, som har antas ocksa gefalse som resultat.

2.1.3 Martins Principer

Dooley [2] beskriver ett antal karaktarsdrag for enhetstestning, beskrivna av RobertMartin:

Snabba: Snabba tester kors oftare an langsamma, sa hall dem enkla.

Isolerade: Tester skall inte vara beroende av varandra, och skall inte satta uppdata for andra tester. Ramverk for testning tilldelar dock ofta verktyg for att justundvika detta problem och halla testerna isolerade.



Upprepbara: Testerna bor koras i vilken ordning som helst och nar som helst,speciellt nar ny kod har tillagts.

Sjalv-validerande: Ett test kan antingen lyckas eller misslyckas, sa resultatetskall vara en boolsk datastruktur. Det skall vara sjalvklart om vad testens resultatar, utan att behova lasa en stor logg.

Lagliga: Skriv tester nar de behovs, sa de ar tillgangliga vid behov. Detta gar handi hand med testdriven utveckling for just agila programmeringsstil, sa testernafordefinieras innan arbetet utfors.

Det ar ocksa nyttigt att tester inte testar mer an ett koncept samtidigt, eftersom ett testbor endast testa efter ett fall i taget. Detta kan aven gora felsokning av koden enklare,eftersom fel for specifika fall ger en battre indikator pa felkallan an ett stort fall somenkapsulerar manga koncept.

2.1.4 Isolering av tester

En barande ide med enhetstester ar att de ska testa enheten individuellt. Testerna mastedarfor isoleras. Vad menas da med detta? De olika komponenterna i ett program arbetartillsammans och ar beroende av varandra. I ett valskrivet program forsoker man atthalla kopplingen sa lag som mojligt, men fullstandig isolering gar sallan att uppna. Foratt sakerstalla att testerna beror endast enheten under test, sa ersatter man externakomponenter med sa kallade mocks.

2.2 Jest-ramverket

Jest ar ett ramverk for testning i JavaScript som implementeras direkt via konfigure-ring av package.json. Jest anvander jamforelser emellan forvantnings- och matchnings-objekt for att utfora tester. Om det utvarderingen av forvantningsobjektet inte godkannsav matchningsobjektet sa misslyckas ett test.

t e s t ( ’ a r i thmet i c s ’ , ( ) => {expect (3+4). toBe ( 7 ) ;expect (1−5). toBeLessThan ( 0 ) ;expect ( 0 . 1 ∗ 3 ) . toBeCloseTo ( 0 . 3 ) ;expect ( d i v id e ( 4 , 0 ) ) . toThrow ( ’ Not al lowed ’ ) ;

} ) ;

Figur 2: Exempel av ett Jest-testfall, dar fyra jamforelser maste godkannas for att test-fallet skall lyckas.

Tag koden i figur 2 som exempel. Det finns fyra stycken jamforelser som ar del avsamma test arithmetics. Den forsta jamforelsen testar for ekvivalens emellan tva ut-tryck. Den andra testar inte efter exakthet, utan om det forvantade vardet ar mindre



an det bifogade. Det tredje testar ekvivalens efter decimaltal, for att motverka fel p.g.a.avrundningar. Det fjarde forvantar sig att det givna anropet inte kommer vara tillatet,utan kommer istallet kasta ett undantag. Om nagon av jamforelserna emellan expect-och toBe-paren ej lyckas, sa misslyckas hela testen. Ramverket skriver aven ut eget for-materade felmeddelanden vid slutet av testningen, dar namnet urskiljer fallen. Jest haraven manga fler varianter av matchnings-objekt for att testa efter olika varden.

2.3 Exempelapplikationen

Applikationen som kommer att anvandas for testning kommer ta form av ett stampelurfor anstallda, som administratorer sedan utvarderar manuellt. De anstallda pa foretagetanvander en IoT-enhet, som fungerar som ett digitalt stampelur, for att registrera sinaarbetade perioder pa foretaget. Administratorer pa foretaget har tillgang till en vy foratt se dessa arbetade perioder for de anstallda. Forutom att se en lista over alla anstalldaoch dess arbetade timmar, kan de aven se perioder som ar specifika for en individuellanstalld.

Notera att systemet skall inte tillata nagon behandling av datan forutom berakningarna.Det ar inte avsett att administratorerna skall kunna gora nagot annat an att navigera ige-nom systemet och se de sammanstallda perioderna. Berakningarna sjalva utfors primarti bakgrunden, vilket beror pa om anvandaren har valt att se en generell sammanstallningav allas arbetade tider, eller for en enskild anstalld.

2.3.1 Stampelur

Ett stampelur ar en enhet som anvands for att registrera timanstalldas arbetsskift paen organisation. Genom att anstallda registrerar bade ankomst till och avgang ifranarbetsplatsen via denna typ av system, kunde informationen associerad med registre-ringen sedan anvandas for att berakna den totala arbetsinsatsen, och darmed lonen. Enankomst eller avgang har en associerad anstalld och en tidpunkt, varav den arbetadeperioden beraknas som skillnaden i tid emellan dessa.

Dessa stampelur har anvants anda sedan 1900-talet. Vanligtvis registrerades tidernamekaniskt pa kort i tjock papp genom att stampla i den nuvarande tiden i ett givetfalt. Vissa andra implementationer slog aven hal i kortet for att undvika overskrivelseav redan utforda stamplingar. Det var inte forran slutet av 1900-talet som dessa enheterborjade bli strikt elektroniska, via magnetkort och PIN-koder. Stampeluret sjalv leveran kvar idag i olika elektroniska former. Kortet sjalv har dock ersatts via andra enheter,losenord och biometriska losningar som fingeravtryck eller ansikts igenkanning.

2.4 Code Coverage

Code coverage gar ut pa att varje rad av kod i ett program skall testas. Vardera boranropas atminstone en gang med passande data under for att garantera att hela kodenfungerar korrekt. Det finns olika typer av kod som kategoriseras av denna teknik:



Figur 3: Stampelur fran Simplex Time Recorder Co.

• Straight-line code har endast en vag igenom funktionen. Det finns ingen flow-control, utan samma kod exekveras i samma sekvens alltid.

• Branch coverage gar ut pa att vardera vag i ett program skall testas. Detta ardirekt lankat till flow-control, specifikt if-else-satser. For vardera av dessa satserbehovs varje utfall av koden att testas. Om det t.ex. finns en switch maste allafall testas, samt dess default-fall.

• Loop coverage ar snarlikt med ovan-kategorin, men ar istallet forknippat med whi-le- och for-loopar. Det viktiga har ar att testa for fall dar innehallet av loopeninte skall exekveras alls, samt oandliga loopar. Det behovs aven tester for typiskaloop-rutiner, dar innehallet endast exekveras en gang, samt flera ganger.

• Return-values beror det resultat som returneras av funktionsanrop. Dessa retur-nerade varden behover ocksa testas for.

Code Coverage finns tillgangligt i Jest via ramverket Istanbul, som aktiveras via kom-mandot: jest --coverage



3 Implementation

3.1 Struktur

Applikationen sjalv ar strukturerad efter amnesomradena for seminarierna, se figur4. Applikationens centrum bestar av Node.js och Express.js funktionaliteten, specifiktberakningarna for arbetsperioderna, kopplingarna till IoT-enheterna och interaktionenmed databasen. For detta seminarie ar komponenten Calculations i NodeJS applica-tion-grupperingen i fokus, eftersom de resterande komponenterna behandlar interaktionmed kopplade komponenter.

Figur 4: Designstruktur for exempelapplikationen. De graskalade komponenterna ar intei fokus i detta seminarie.

De anstalldas interaktion ar mestadels automatiserad, eftersom de interagerar endastvia IoT-enheterna for att stampla in och ut ifran systemet. Calculations daremot om-fattar den logiska funktionalitet som administratorerna kraver for sin arbetsuppgift,d.v.s. utvarderingen av arbetade perioder.

3.2 Utvecklad funktionalitet

Eftersom vi endast beror enhetstestning for servern nu, sa begransar vi oss till denfunktionalitet som kraver testning. Den berakning som ar av intresse just nu ar den totaltarbetad tid for enstaka anstallda, utifran en samling med tidsstamplar. Administratorerbehover framst denna berakningar for att kunna berakna arbetsinsatserna via webb-granssnittet.

Referera till figur 5 for denna berakning utav en samling med tidsstamplar. Funk-tionen calculateHours forvantar en array av JSON-objekt med strukturen {id:int,in:bool, time:number}. For calculateHours sa ar endast talet i faltet time av in-tresse. Faltets innehall skall vara ett tal som representerar en klockslag pa 24-timmarsklockan, och maste darmed vara inom intervallet [0..24). Den beraknade tiden for enperiod har ar differensen emellan tva grannelement i den bifogade arrayen. Alla des-



sa perioder tillsammans ackumuleras for att ge en totalt arbetad tid. Avsikten ar attfunktionen calculateHours anropas med antingen alla stamplar som ar relevanta foren anstalld, eller endast in- och ut-checkningar for en period.

module . export s = {ca l cu la t eHour s : ( timestamps ) => {

l e t totalTime = 0f o r ( l e t i = 0 ; i<timestamps . l ength ; i +=2) {

l e t s t a r t = timestamps [ i ] . timel e t stop = timestamps [ i +1] . timei f ( isNaN ( s t a r t ) | | isNaN ( stop ) ){

throw ’ Time i s NaN! ’}i f ( s t a r t < 0 | | stop < 0 | | s t a r t > 2 4 | | stop > 24) {

throw ’Wrong time ! ’}i f ( stop − s t a r t < 0) {

throw ’ I n v a l i d time span ’}l e t time = stop − s t a r ttotalTime += time

}re turn totalTime

}}

Figur 5: stats.js-funktionalitet for att berakna tidsintervall.

3.3 Testning

Testerna for berakningarna har flera olika gransfall och illustreras i figur 6.Det forsta testet avser ett forvantat fall utan problem. I detta fall har ett antal

stamplar associerade med en anstalld redan hamtats och bifogats till calculateHours.Den beraknade differensen ar redan forberaknad aven i matchningsobjektet och funktio-nen resultat i forvantningsobjektet skall vara ekvivalent med detta.

Det andra testet avser en kontroll av felaktig data. Datan har ar inte giltig, eftersomperioden har startat efter att den har slutats. Det skall inte returneras nagot beraknatresultat har, utan berakningen forvantas producera ett undantag.

Det tredje testet avser flera olika tester av felaktiga datastrukturer. Notera att each-funktionen ar tillhandahallen av Jest-ramverket just for att tillhandahalla olika para-metrar till samma test-funktionalitet. expect-uttrycket anropas darmed med varderaelement i den totala array som tillhandahalls i each, som ar subarrayerna med relevant



const s t a t s = r e q u i r e ( ’ . . / s ta t s ’ )d e s c r i b e ( ’ S ta t s module ’ , ( ) => {t e s t ( ’ Ca l cu l a t e s the t o t a l time o f g iven timestamps ’ ,

( ) => {l e t stamps = [{ id : 1 , in : true , time : 8 } ,

{ id : 1 , in : f a l s e , time :12 } ,{ id : 1 , in : true , time : 8 } ,{ id : 1 , in : f a l s e , time :12 } ,{ id : 1 , in : true , time : 9 } ,{ id : 1 , in : f a l s e , time :15 } ,{ id : 1 , in : true , time : 7 } ,{ id : 1 , in : f a l s e , time :12 } ]

l e t hours = s t a t s . ca l cu l a t eHour s ( stamps )expect ( hours ) . toBe (19)

})

t e s t ( ’ Throws e r r o r i f s t a r t i n g time happens be f o r e end time ’ ,()=> {

l e t stamp = [{ id : 1 , in : true , time :17 } ,{ id : 1 , in : f a l s e , time :12 } ]

expect (()=>{ s t a t s . ca l cu l a t eHour s ( stamp ) } ) . toThrowError ( )})

t e s t . each ( [[{ id : 1 , in : true , time : 9} ,{ id : 1 , in : f a l s e , time : 29} ] ,

[{ id : 1 , in : true , time : −9} ,{ id : 1 , in : f a l s e , time : 22} ] ,

[{ id : 1 , in : true , time : ”e ”} ,{ id : 1 , in : f a l s e , time : 20} ]

] ) ( ’ Ca l l s e r r o r on i n v a l i d time on o b j e c t s %o or %o ’ ,( a , b ) => {

expect ( ( ) =>{ s t a t s . ca l cu l a t eHour s ( [ a , b ] ) } ) . toThrowError ( )

})})

Figur 6: stats.tests.js-funktionalitet for enhetstestning.

data att testa. Fallen omfattar klockslag utanfor 24-timmars klockan, negativa klock-slag och icke-heltal i tidsfaltet. Vardera av dessa fall med felaktig data skall produceraundantag istallet for resultat for att passera.



3.4 Exekvering av Exempel

Exempelkoden finns att hamta direkt som komprimerad katalog under lanken i [1]. Ko-den kan aven hamtas via clone-funktionalitet hos Git som visad i figur 7. Efter attkoden hamtats, anvand Node.js for att installera den funktionalitet som applikationenar beroende av. Darefter kan enhetstesterna utforas direkt via antingen jest-kommandoteller npm test1.

g i t c l one https : // github . com/ antonkal / II1302−S1 . g i t s eminar i ecd seminar i enpm i n s t a l l

Figur 7: Bash-skript for att hamta och installera exempelapplikationen ifran dess Git-repository.

Resultatet av enhetstesterna bor ha en liknande resultat som i figur 8. For varderatestsvit (d.v.s. filerna som innehaller testerna) rapporterar Jest antingen ett lyckat ellermisslyckat slutresultat, beroende pa om dess testfall alla lyckas. Eftersom testsviternakan innehaller flera testfall dock sa rapporterar Jest aven antalet fall som accepteradesocksa.

Notera aven testfallen som beskrivs som todo i resultatet i figur 8. Detta beror sviter-na db.test.js och stats.test.js, eftersom deras testfall ar definierats med denna Jest-funktion, och inte tillhorde den utvecklade funktionalitet for detta seminarie. Dettapaverkar inte slutresultatet, eftersom dessa konstruktorer accepteras automatiskt undertestning, men eftersom de inte inkluderar nagon funktionalitet urskiljer Jest pa dessa iresultatet (raknas inte med som passed).

> j e s t

PASS t e s t s /db . t e s t . j sPASS t e s t s / c o n t r o l l e r . t e s t . j sPASS t e s t s / s t a t s . t e s t . j s

Test S u i t e s : 3 passed , 3 t o t a lTests : 7 todo , 5 passed , 12 t o t a lSnapshots : 0 t o t a lTime : 1 .495 sRan a l l t e s t s u i t e s .

Figur 8: Utdrag av exekvering av enhetstesterna och dess resultat.

1Notera att bada dessa kommandon exekverar alla testfall.



Lankar & Referenser

[1] II1302-S1 - Github. [Online; accessed 29. Apr. 2019]. April 2019. url: https://github.com/antonkal/II1302-S1.

[2] John F Dooley. Software Development, Design and Coding: With Patterns, Debug-ging, Unit Testing, and Refactoring. eng. 2nd ed. Berkeley, CA: Apress, 2017. isbn:9781484231524.

[3] Gerard O’regan. Introduction to Software Quality. eng. 2014. utg. UndergraduateTopics in Computer Science. Cham: Springer International Publishing, 2014. isbn:9783319061054.

[4] Getting Started - Jest. [Online; accessed 23. Apr. 2019]. url: https://jestjs.io/docs/en/getting-started.

[5] Time clock - Wikipedia. [Online; accessed 21. Apr. 2019]. Mars 2019. url: https://en.wikipedia.org/wiki/Time_clock.

[6] CBS Sunday Morning. Almanac: The punch clock. [Online; accessed 29. Apr. 2019].Okt. 2016. url: https://youtu.be/kCpWF5UXfQ4.


II1302 - Projekt & Projektmetoder Seminarie 2 - Integrationstesta ett System

Seminarie 2Integrationstesta ett System

Anders Gawell, Anton Kallin

30 augusti 2019

1 Introduktion

I detta labbkompendium kommer området integrationstestning av mjukvara att introdu-ceras. Till detta ändamål har applikationen ifrån det föregående seminariet utökats föratt illustrera de nya principerna kring dessa begrepp inom testning.

1.1 Krav & Begränsningar

Läsaren förväntas ha kunskaperna kring begreppen som togs upp i föregående kompen-dium, Seminarie 1 - Att Enhetstesta en Server. Det ställs inga förväntningar på läsarenskunskap kring områdena som tas upp i detta kompendium. Exempel i detta kompendiumkommer hålla sig till en applikation som är utvecklad med JavaScript. Exempelappli-kationen kommer inte genomgås i detalj, endast de delarna som är väsentliga för dettaseminarie. Läsaren kan sätta sig in i funktionaliteten i mer detalj via den publika GitHub-repositoriet som är bifogat efter behov, men detta är inte ett krav. Det är dessutom inteett krav att läsaren förstår applikationen i förväg. Principerna som beskrivs kring test-ning och utveckling i detta kompendium är universella och är inte begränsade till varkenspråket eller applikationen själv. Läsaren förväntas ta till sig av principerna som gås ige-nom och använda exemplet som en startpunkt för att förstå de abstrakta koncepten ochteknologierna, för att sedan kunna implementera dem i sina egna uppgifter.

1.2 Mål

• Beskriva vad integrationstestning innebär och hur det går till.

• Introducera några olika strategier för integrering av system och dess påverkan påhur de testas.

• Expandera exempelapplikationen som testmiljö ifrån förra seminariet.

• Ge exempel på både integrationstester och mock-funktionalitet genom exempelap-plikationen.



• Detaljera implementationen av integrationstestning på applikationens nuvarandefunktionalitet.

1.3 Teknologier

• Node.js

• Jest

• Express.js

• Supertest

2 Bakgrund

2.1 Integrationstestning

Att integrera arbete emellan två parter är ett jobb i sig själv. Förutom de klara skill-naderna i människors metod kring att angripa problem och strukturera sitt arbete, såförekommer de största problemen då ny funktionalitet utvecklas som utnyttjar flera olikamoduler som designats av olika personer. Detta kan underlättas genom att det inte finnsnågra oklarheter i API:et om vad funktioner och moduler är avsedda att användas för,deras argument och resultat på dessa, men detta är sällan givet. Som exempel så kan olikaberäkningar använda olika typer av enhetssystem (empiriska eller metriska), funktionerkan förlita sig på callbacks isället för synkronisitet, och mycket mer. Dessa problem ärexempel som låter självklara på papper, men är också klara exempel på brister i systemsom förlitar sig på detaljerna kring implementationer. Även om man tar dessa detaljer iåtanke ifrån första designfasen, så skyddar det inte projektet ifrån problem med externfunktionalitet som databaser eller fristående bibliotek.Integrationstester tar dessa problem i åtanke genom att fokusera på interaktionen

emellan flera olika API:er. Eftersom interaktionen emellan olika komponenter bygger påAPI:erna själva, så är interaktionen även beroende av resultatet ifrån dessa funktionsan-rop.Denna typ av testning finner sig emellan enhetstestning och systemtestning, men har

drag ifrån båda, på så sätt att det testar interaktionen emellan ett systems komponenter,men kontrollen av denna interaktion sker via analys av resultatet. Snarlikt med enhets-testning så fokuserar integrationstester på vad som sker internt, d.v.s. hur kommunikatio-nen sker, men eftersom det inte har enstaka enheter i fokus så är det inte ett enhetstest.Likt systemtestning är integrationstester mest berörda av slutresultatet från att fleraenheter används tillsammans, men det är inte ett systemtest eftersom kommunikationenemellan enheterna är vad som egentligen vill testas, och testet inte nödvändigtvis beröralla komponenter i en representativ miljö.



2.2 Big Bang

Eftersom integreringstester kan endast utföras efter att komponenter väl har blivit inte-grerade, så krävs det en viss strategi för hur integreringen av systemet skall gå till. BigBang-strategin låter all integrering ske efter att alla komponenter är färdigutvecklade.Efter att det initiala integrationssteget har överkommits, så testas därmed systemet istora och små delmängder. Trots att denna strategi inte kräver någon ytterligare mödaför att bygga och leverera inkrementella iterationer i utvecklingen, så tillåter det inte attproblem påträffas tidigt. Det finns inte rum att hitta buggar i färdig utvecklad program-vara tidigt med denna strategi, eftersom den kräver att alla komponenter är färdiggjorda.Att lokalisera buggar sent i utvecklingen av mjukvara kan också göra enkla problem stör-re än vad de behöver vara, eftersom det blir svårare att gå tillbaka i utvecklingen ochrevidera ens arbete.

2.3 Inkrementell Integrering

När designen av ett system har en klar trädstruktur, som t.ex. viaModel-View-Controller -mönstret, så kan denna arkitektur utnyttjas för att underlätta utvärderingen av systemet.Inkrementell integrering går just ut på lager i arkitekturen successivt byggs ihop för attbilda det färdiga systemet. Integreringen här sker lager för lager i arkitekturen, med enprocess som repeteras tills alla lager har tillagts.

2.3.1 Drivers & Stubs

När man inkrementellt testar delar av ens utvecklade system, finner man sig ofta i situ-ationer då man krävs använda en funktionalitet som fortfarande är under design för atttesta en del av systemet som redan är klar. Ett sätt att undvika att vänta in utvecklingenav dessa är att skapa objekt som simulerar dessa förväntade uppföranden för att kunnatesta ett färdigt system. Oavsett om man finner sig i en situation där man behöver enuppsättning av data för att utföra en test, eller simulera ett flöde som testen är avseddatt producera, så är principen densamma: man behöver fylla in gapen i sin kod för detsom inte finns än för att testa en moduls funktionalitet.Dessa typer av “placeholder”-objekt har många olika namn, som “test harnesses”, men

deras enda syfte är att tillåta testerna i fråga, via Drivers, Stubs och Mocks.

Drivers är sekvenser av funktionsanrop som anropar den testade modulen, ochställa upp input som funktionen under test kräver.

Stubs är emulatorer av komponenter som anropas av en testad modul för att taemot data under testerna. Ofta returnerar dessa fastställda returvärden för atttillföra just tillräckligt med funktionalitet för att utföra ett test.

Mocks är emulatorer som har förprogrammerad funktionalitet med förväntningarsom specificerar vilka anrop de förväntas få.



Figur 1: Exempelarkitektur för en applikation som följer Model-View-Controller -mönstret, i form av en trädstruktur.

2.3.2 Olika Inkrementella Strategier

Bottom-Up börjar med att producera grundläggande funktionalitet för att senarebygga upp till ett komplexare system. För MVC-mönstret motsvaras detta av attsystemets modell och verktyg etableras först och testas, för att sedan bygga upp tillkontrollers och ett användargränssnitt (se figur 1). Denna strategi kräver inga stubsalls, eftersom funktionaliteten stubs skall ersätta implementeras först, och tillåtertestning att starta väldigt tidigt av systemet, men kräver däremot drivers för attkunna testa de undre lagren efter ett passande fall. Eftersom den överskådligafunktionaliteten hos systemet också utvecklas relativt sent, ger denna strategi ävenrum för missförstånd av en kunds krav, eftersom det är svårt att producera enprototyp av systemet för dem.

Top-Down är motsatsen till Bottom-Up-strategin och lägger inkrementellt tillfunktionalitet till ett gränssnitt. För MVC-mönstret motsvaras detta av att ettsystems gränssnitts utvecklas först, medans de grundläggande modellerna och verk-tygen implementeras sist (se figur 1). Eftersom en prototyp kan tidigt bildas för attreflektera kraven hos de överordnande komponenterna, får man en större chans attidentifiera designfel under utvecklingen och kan anpassa sig tidigt. Däremot kräverdenna strategi väldigt många stubs för att prototypen ens skall vara möjlig vid etttidigt stadie, eftersom komponenterna på lägre lager måste emuleras.

Hybrid/Sandwich är en kombination av båda ovan nämnda strategier. Till skill-nad ifrån att låta kritiska fel upptäckas i de övre- eller under-ordnade lagren istrukturen, integrerar man istället i båda riktningarna för att mötas i mitten. Prak-tiskt sett är denna strategi inte alltför annorlunda, eftersom man endast flyttat omchansen för stora fel till de mellersta lagren (t.ex. Controller-modulerna i figur 1),men det tillåter båda perspektiven av Botten-Upp och Toppen-Ned att vara i fokusgenom att utnyttja båda drivers och stubs.



2.4 Olika Integreringstester

2.4.1 För Interna Enheter

Interna enheter arbetar inom ett utvecklat systems gränser, ofta som subsystem. Härtestas flera klassers kommunikation emellan varandra och deras sammansatta resultat.Den största delen av integrationstester faller inom denna typ, men blir svårare att utförai en större hierarki av komponenter, där externa enheter lätt inkluderas. För att isoleradessa testade subsystem ifrån det resterande systemet används mock-implementationsofta. Mock-implementationer överskrider den aktuella implementation av en modul föratt istället returnera ett annorlunda resultat. För långsamma externa komponenter somwebb-API:er eller filsystem kan deras exekvering tillhöra ett anrop som testas, men ärinte relevant för testning av interna enheter. Istället för att utföra dessa långsamma ochirrelevanta anrop så kan deras funktionalitet ersättas med något representativt istället,som ett statiskt värde.

2.4.2 För Externa Enheter

Utvecklade system kommunicerar alltid med något externt system, som inte är egenutvecklad del av systemet. Många applikationer bygger mycket av sin funktionalitet påexterna bibliotek av verktyg eller drar nytta av en databas för att spara sin data. Ävenapplikationer som inte drar nytta av ovan nämnda komponenter kräver externa tjänstersom filsystem eller en betjänade server för att ens tillhandahålla sin funktionalitet.Dessa kräver också en form av integrationstest emellan ens utvecklade program och

den externa omvärlden genom dess gränssnitt. Ett enkelt exempel på detta vore en data-bas med ett eget abstraherande API. API:et bör här utnyttjas i ett test för att verifieraatt programmet kan kommunicera med databasen. Detta kan vara en insättning av ettenstaka värde, en representativ frågeställning som görs ofta i praktiken, eller endast enbegäran på att hanteringssystemet utvärderar ett uttryck som alltid är sant. Integra-tionstesten här går inte ut på att stresstesta den externa databasen, endast att verifieraatt kommunikationen med den är möjlig.

2.5 Kontinuerlig Integration

Begreppet Continuous Integration är en princip som säger att kod skall integreras ochvalideras så tidigt som möjligt. Istället för att ändringar i mjukvaran hålls lokalt skalldessa delas med resten av projektet, som att lägga upp det i ett versionshanteringssy-stem. Detta möjliggörs i praktiken just genom automatisering via pipelines som Jenkinseller “Delivery Pipeline” i IBM :s molnmiljö. Genom att övervaka versionshanteringssy-stemet kan dessa pipelines agera direkt på uppdateringar genom att exekvera en kedjamed kommandon i sekvens, uppdelade i stadier. Stadierna kan beröra olika delar av in-tegrationsarbetet, från att kompilera koden, testa den, lägga upp den i en representativmiljö och testa den utförligare, lägga i bruk, etc.



3 Implementation

3.1 Förändringar

Applikationen har utökats för att inkludera fler komponenter, primärt en express- ochdatabas-komponent, samt en kontroller för att reglera flödet. Interaktionen emellan dessakomponenter är vad integrationstesterna kommer att undersöka. De inkluderade kompo-nenterna illustreras i figur 2

Figur 2: Designstruktur för exempelapplikationen.

3.2 Utvecklad funktionalitet

3.2.1 Express-applikationen

Eftersom administratörerna förväntas interagera med systemet via ett webbgränssnittså behöver servern kunna tolka användarens input. Express.js-ramverket är en back-endför servers som tillhandahåller just denna möjlighet, och är just anpassad för Node.js-applikationer.Genom att omdirigera HTTP:s GET-förfrågningar via app.get() så kan applikatio-

nens front-end tilldelas olika data, beroende på vilken resurs som efterfrågas. Koden ifigur 3 illustrerar dessa olika avsedda vyer via det första argumentet till app.get(). Föratt beräkna antalet arbetade timmar för vardera anställd, så krävs två argument i för-frågan för att begränsa tidsutrymmet på responsen. Att hämta användarstatestiken fördagen, samt för en individuell anställd, krävs det inga ytterligare argument ifrån förfrå-gan förutom en identifierare för en enskild anställd. Däremot kan en efterfrågad anställdinte existera i systemet, vilket istället returnerar ett felmeddelande som respons.Dessa tre olika former på förfrågningarna är våra end-points till applikationen, d.v.s.

de URL:er som applikationen reagerar hos HTTP-förfrågningar. Dessa end-points ärde förfrågningar som applikationen är avsedd att använda för att kommunicera medsystemet, och begära beräkningar ifrån servern. Som respons skickas ett JSON-objekt



const expre s s=r equ i r e ( ’ express ’ )const c o n t r o l l e r=r equ i r e ( ’ . / c o n t r o l l e r ’ )const app=expre s s ( )

app . use ( expre s s . s t a t i c ( ’ publ ic ’ ) )app . get ( ’/ hours ’ , ( req , r e s )=>{

i f ( ! ( req . query . from && req . query . to ) ) {r e s . s t a tu s (400)

. j son ({ e r r o r : ’ Request need to pass query params from and to ’ })} e l s e {

l e t timeData=c o n t r o l l e r . getHours ( req . query . from , req . query . to )r e s . j son ( timeData )

}}). get ( ’/ today ’ , ( req , r e s )=>{

l e t statsToday = c o n t r o l l e r . getTodaysStats ( )r e s . j son ( statsToday )

}). get ( ’/ employee / : id / s ta t s ’ , ( req , r e s )=>{

try {l e t p e r s ona lS t a t s=c o n t r o l l e r . g e tPe r sona lS ta t s ( req . params . id )r e s . j son ( pe r s ona lS t a t s )

} catch ( e ) { r e s . s t a tu s ( 4 0 4 ) . j son ({ e r r o r : e }) }})

module . export s=app

Figur 3: Server-komponentens primära process som lyssnar efter inkommande HTTP-kopplingar.

med relevant data, beroende på end-point, som utnyttjas för att uppdatera applikationensfront-end.

3.2.2 Databashantering

Koden i figur 4 tillhör modulen som används för att kommunicera med databasen. Ef-tersom databasen inte är en del av systemet än, så tilldelas primärt ett API som användsav de resterande modulerna för att ställa frågor till databaserna. Modulens syfte är attavgränsa databaskommunikation ifrån resten av programmet, men tills vidare existerarden mestadels som ett API för att kunna användas i de resterande modulerna. Dettaär dock inte ett problem för testerna, eftersom objekten som representerar databasernaöverskrids vid testning via ett mock-objekt, vilket tillåter oss att använda API:et somtilldelas av databasmodulen som den är utan större förändringar.



const c loudant = r equ i r e ( ’ c loudant ’ )

module . export s = {getEmployees ( cb ) {

l e t employeeDB = cloudant . use ( ’ Employee ’ )employeeDB . get ( cb )

} ,getEmployeeById ( id , cb ) {

l e t employeeDB = cloudant . use ( ’ Employee ’ )employeeDB . get ( id , cb )

} ,getEmployeeTota l s InInterva l ( from , to , cb ) {

l e t employeeDB = cloudant . use ( ’ Employee ’ )employeeDB . get ({ from , to } , cb )

} ,getStampsInInte rva l ( from , to , cb ) {

l e t stampDB = cloudant . use ( ’ Stamp ’ )stampDB . get ({ from , to } , cb )

} ,getStampsById ( id , cb ) {

l e t stampDB = cloudant . use ( ’ Stamp ’ )stampDB . get ( id , cb )

}}

Figur 4: Placeholder-kod för databas-funktionalitet.

3.3 Testning

3.3.1 Express-applikationen

Integrationstestet av Express.js-funktionaliteten testar primärt hanteringen av API:etsom ramverket tillhandahåller, och dess användning för att skapa responser ifrån sy-stemet (se figur 3). Eftersom syftet berör detta externa API, så är testet begränsattill att endast verifiera funktionaliteten kring systemets end-points. För att isolera ochhålla testet så enkelt som möjligt, överskrids funktionaliteten hos specifika funktioner icontroller.js-modulen via mock-implementationer. Istället för att systemet anropar denavsedda implementation för denna modul, så anropas istället denna mock-variant iställetunder testning.För integrationstestets skull så är det inte relevant vad som händer, efter att en end-

point har anropats med passande argument, eftersom dess syfte är endast att verifiera Ex-press.js-API:ets användning i app.js. Dessutom, eftersom controller.js-funktionalitetensom anropas via app.js berör primärt databasanrop, så kringgås dessa långsamma anrop



för att använda snabbare tillkomlig data via dessa mocks. Testerna själva berör verifi-kation av funktionaliteten hos vardera end-point, mer specifikt den HTTP-status somreturneras till klienten. Dessa mockade implementationer i figur 5 returnerar statisk dataför just detta syfte.Notera även attNode-biblioteket SuperTest används även här för att abstrahera HTTP-

interaktionen emot våra end-points (se figur 5). Denna används genom att binda importe-ringen av biblioteket till request-konstanten, som därefter kan skapa HTTP-förfrågningarvia get-anropet med en relevant URL. Notera att alla tester av våra end-points förHTTP i figur 6 utnyttjar detta bibliotek för att kunna kontrollera responsen i dekla-rationen let response=await request(...); (likadan deklaration för alla testerna avend-points).För <ROT_URL>/hours så kräver hämtning av alla anställdas totala arbetstid ett in-

tervall som specifieras av två argument för starten och slutet på perioden. Om dessaargument inte tillhandahålls, skall en HTTP-felkod returneras (d.v.s. 400 - Bad Request).För <ROT_URL>/today är anropet själv nog för att returnera ett resultat, så HTTP-

koden för giltiga frågeställningar kan returneras (d.v.s. 200 - OK ).För <ROT_URL>/employee/<IDENTIFIERARE>/stats så innehåller URL:en själv infor-

mationen nödvändig för att returnera ett resultat, men skall endast returnera ett resultatför identifierarna som inkluderas i mock-implementationen av getPersonalStats. På såsätt finns fall då HTTP skall ge en respons för fel, vilket är då identifieraren given intekorresponderar till någon anställd. Flera tester verifierar även att detta gäller, genom attkontrollera HTTP-responskoden.I alla dessa fall så testas implementation av app.js genom att skapa egna HTTP-

förfrågningar emot den. Eftersom app.js utnyttjas Express.js-ramverket så kommer densjälv reagera på förfrågningarnas adresser och exekvera den relevanta sekvensen av in-struktioner för vardera adress. Notera även Express.js själv möjliggör HTTP-funktionaliteten(skapa förfrågningar och passande respons, samt enkapsulera det i ett API) som primärtjämförs för att utvärdera resultatet av vardera testfall.

const r eque s t = r equ i r e ( ’ supe r t e s t ’ )const app=r equ i r e ( ’ . . / . . / app ’ )j e s t . mock ( ’ . . / . . / c on t r o l l e r ’ )c o n t r o l l e r . getHours . mockImplementation ( ( from , to )=>{})c o n t r o l l e r . g e tPe r sona lS ta t s . mockImplementation ( ( id)=>{

l e t id_nr=par s e In t ( id , 10)i f ( id_nr>3 | | id_nr<1) { throw ’Not in DB’ }e l s e re turn { id : id_nr , name : ’ Pebbles ’ , workedHours : 12 }

})d e s c r i b e ( ’HTTP i n t e r f a c e endpoints ’ , ()=>{ . . . })

Figur 5: Testfall för app.js, med en mockad implementationen för .



de s c r i b e ( ’HTTP i n t e r f a c e endpoints ’ , ()=>{de s c r i b e ( ’GET /hours ’ , ()= {

t e s t ( ’ r e tu rn s e r r o r code with no query params ’ , async ()=>{l e t re sponse = await r eque s t ( app ) . get ( ’ / hours ’ )expect ( re sponse . statusCode ) . toBe (400)} )

t e s t ( ’ r e tu rn s 200 on va l i d query params ’ , async ()=>{const t=’/hours ? from="2011−04−15"&to="2012−05−12"’;l e t r e sponse = await r eque s t ( app ) . get ( t )expect ( re sponse . statusCode ) . toBe (200)} )

})d e s c r i b e ( ’GET /today ’ , ()=>{

t e s t ( ’ r e tu rn s with s t a tu s code 200 ’ , async ()=>{l e t re sponse = await r eque s t ( app ) . get ( ’ / today ’ )expect ( re sponse . statusCode ) . toBe (200)} )

})d e s c r i b e ( ’GET /employee / : id / s ta t s ’ , ()=>{

t e s t . each ( [ 1 , 2 , 3 ] ) ( ’ r e tu rn s with s t a tu s code 200 f o r id %d ’ ,async ( id)=>{

const t=‘/employee/${ id }/ s ta t s ‘ ;l e t r e sponse=await r eque s t ( app ) . get ( t )expect ( re sponse . statusCode ) . toBe (200)} )

k t e s t . each ( [ 1 0 0 , 3 2 4 , 0 ] ) ( ’ r e tu rn s with s t a tu s code 404 f o r id %d ’ ,async ( id)=>{

l e t re sponse = await r eque s t ( app ) . get ( ‘ / employee/${ id }/ s ta t s ‘ )expect ( re sponse . statusCode ) . toBe (404)} )

t e s t . each ( [ 1 , 2 , 3 ] ) ( ’ r e tu rn s s t a t s f o r employee %d ’ ,async ( id)=>{

l e t re sponse=await r eque s t ( app ) . get ( ‘ / employee/${ id }/ s ta t s ‘ )l e t employee=response . bodyexpect ( employee . id ) . toBe ( id )expect ( employee . name ) . toBe ( ’ Pebbles ’ )expect ( employee . workedHours ) . toBe (12 )} )

})})

Figur 6: Testfall för app.js, med integrationstester inkluderade.

3.3.2 Databashantering

Notera den mockade-implementationen av databasen i figurerna 7 och 8. Denna imple-mentation används endast när Jest-ramverket utför sina testfall. Syftet kring detta är attkringgå utnyttjandet av ett externt bibliotek när det kan undvikas, speciellt för långsam-ma uppgifter som databasaccess eller filskrifter. Likt som när enhetstester designades så



const c loudant = {use (dbName) {

i f (dbName === ’Employee ’ ) { re turn employeeDB }e l s e i f (dbName ===’Stamp ’ ) { re turn stampDB }e l s e throw ’ I nva l i d DB’

}} const employeeDB = {

get ( arg0 , arg1 ) {l e t cbl e t datai f ( arguments . l ength === 2) {

l e t id = arg0cb = arg1data = getEmployeeById ( id )

} e l s e {cb = arg0data = employeeData

} cb ( nu l l , data )}

} const stampDB = {get ( arg0 , arg1 ) {

l e t cbl e t datai f ( arguments . l ength === 2 && typeo f arg0 === ’ object ’ ) {

cb = arg1l e t from = arg0 . froml e t to = arg0 . todata = getStampsInInte rva l ( from , to )

} e l s e i f ( arguments . l ength === 2 && typeo f arg0 === ’number ’ ) {cb = arg1l e t id = arg0data = getStampsById ( id )

} e l s e {cb = arg0data = stampData

}cb ( nu l l , data )

}}

Figur 7: Mockad databas funktionalitet. Omfattar urvalet av databaserna.



const getEmployeeById = ( id ) => {l e t found = employeeData . f i nd ( employee => employee . id === id )re turn found

}const getStampsInInterva l = ( from , to ) => {

l e t returned = stampData . f i l t e r ( stamp =>(( stamp . time >= from ) | | ( stamp . time <= to ) ) )

re turn returned}const getStampsById = ( id ) => {

l e t found = stampData . f i l t e r ( stamp => stamp . id === id )re turn found

}const employeeData = [

{ id : 0 , name : " Fred F l i n t s t on e "} ,{ id : 1 , name : " Barney Rubble "} ,{ id : 2 , name : " Mister K i l l "} ,{ id : 3 , name : " Muscles McBoulder"}

]const stampData = [

{ id : 0 , in : true , time : 9} ,{ id : 0 , in : f a l s e , time : 11} ,{ id : 0 , in : true , time : 13} ,{ id : 1 , in : true , time : 8} ,{ id : 1 , in : f a l s e , time : 11} ,{ id : 1 , in : true , time : 13} ,{ id : 2 , in : true , time : 9} ,{ id : 2 , in : f a l s e , time : 12} ,{ id : 3 , in : true , time :8}

]module . export s = cloudant

Figur 8: Mockad databas funktionalitet. Omfattar diverse statiska JSON-objekt som in-nehåll, samt urvalet av dessa.

bör dessa tester hållas så små som möjligt, eftersom stora och långsamma tester har enmindre chans att utföras till skillnad från motsatsen.Det är av just dessa anledningar som denna mockade-implementation utnyttjas istället

för ett riktigt databasanrop vid tester. 1 Genom att kringgå risken för att skapa storalångsamma tester ger vi ett alternativ för att låta tester ta mindre tid. Som sagt, imple-

1Notera att Jest kan kringgå mockande av moduler vid behov via t.ex. dess requireActual-anrop:https://jestjs.io/docs/en/bypassing-module-mocks



mentationen används endast utav Jest eftersom dessa mocks skall endast användas vidtestning.Funktionaliteten i figurerna 7 och 8 är såklart väldigt gles. API:et som ersätter den

egentliga databasimplementationen behöver endast tillhandahålla den funktionalitet somkrävs för testerna i fråga. Figur 7 berör endast skildringen emellan olika typer av dataatt hämta ifrån (datan i figur 8 mer specifikt). cloudant här behöver endast urskiljapå vilken databas man är intresserad av och returnera dess abstraktionsobjekt via encallbacks.Figur 8 är också begränsad, eftersom den endast tillåter hämtning av data ifrån den

urvalda “databasen”. Implementationen själv är inte av intresse, eftersom det endastfiltrerar element från en lista av objekt. Däremot innehåller stampData dock ett antalolika fall som är av intresse för testerna. En av de anställda har endast checkat in ochut, medans en annan har bara checkat in, och de två andra har checkat in och ut tvågånger. Dessa typiska fall är något systemet behöver hantera ofta i bruk förmodligen,men databasen behöver inte hålla i dessa fall själv för testernas skull, eftersom de ärinkluderade i mock-implementationen.Vidare kan denna statiska mock-implementation av databasens innehåll förändras för

att testa andra gränsfall och hur systemet uppför sig efter dessa typer av data. Kommersystemet reagera på en anställd som inte finns i databasen alls, eller på en NULL-pekarepå olika sätt? Samma principer som för gränsfallen hos enhetstestning kan även applicerashär. Eftersom innehållet kan lätt ändras får man flera möjligheter att testa efter olikatyper av relevant och felaktig data, vilket kan verifiera hur robust systemet i stort äregentligen.

3.4 Exekvering av Exempel

g i t c l one https : // github . com/antonkal / II1302−S2 . g i t s eminar i ecd seminar i enpm i n s t a l l

Figur 9: Bash-skript för att installera exempelprogrammet.

Exempelkoden finns att hämta direkt som komprimerad katalog under länken i [1].Koden kan även hämtas via clone-funktionalitet hos Git som visad i figur 9. Efter attkoden hämtats, använd Node.js för att installera den funktionalitet som applikationenär beroende av. Därefter kan alla testerna utföras direkt via antingen jest-kommandoteller npm test. Alternativt kan man även avgränsa testens omfattning genom att tilldelakatalogens namn för de tester som skall utföras. Eftersom testerna redan är avskilda idessa subkataloger, kan endast integrationstesterna utföra via kommandot jest integra-tion eller npm test integration. Resultatet av integrationstesterna bör ha en liknanderesultat som i figur 10.



> j e s t i n t e g r a t i o n

PASS __tests__/ i n t e g r a t i o n /app . t e s t . j sHTTP i n t e r f a c e endpoints

GET /hoursP re tu rn s e r r o r code with no query params (61ms)P re tu rn s 200 on va l i d query params (13ms)

GET /todayP re tu rn s with s t a tu s code 200 (6ms)

GET /employee / : id / s t a t sP re tu rn s with s t a tu s code 200 f o r id 1 (6ms)P re tu rn s with s t a tu s code 200 f o r id 2 (5ms)P re tu rn s with s t a tu s code 200 f o r id 3 (5ms)P re tu rn s with s t a tu s code 404 f o r id 100 (6ms)P re tu rn s with s t a tu s code 404 f o r id 324 (5ms)P re tu rn s with s t a tu s code 404 f o r id 0 (6ms)P re tu rn s s t a t s f o r employee #1 (6ms)P re tu rn s s t a t s f o r employee #2 (7ms)P re tu rn s s t a t s f o r employee #3 (6ms)

Test Su i t e s : 1 passed , 1 t o t a lTests : 12 passed , 12 t o t a lSnapshots : 0 t o t a lTime : 2 .336 s

Figur 10: Utdrag av exekvering av enhetstesterna och dess resultat.

Länkar

[1] II1302-S2 - Github. [Online; accessed 12. Apr. 2019]. April 2019. url: https://github.com/antonkal/II1302-S2.

[2] Martin Fowler. 1. [Online; accessed 13. May. 2019]. April 2007. url: https://martinfowler.com/articles/mocksArentStubs.html.

[3] Rex Black. ”Integration Testing”. eng. I: Pragmatic Software Testing. Indianapolis,Indiana: John Wiley & Sons, Inc., 2017, s. 307–320. isbn: 9781119419297.

[4] Model-driven testing using the UML testing profile. eng. Berliln ; New York: Springer,2008. isbn: 1-281-35283-7.


II1302 - Projekt & Projektmetoder Seminarie 3 - Fran Use Case till GUI-test

Fran Use Case till GUI-test

Anders Gawell

12 maj 2019

1 Introduktion

Manga Use Cases bestar av interaktion med ett grafiskt anvandargranssnitt (GUI). Oftakan dessa Use Cases enkelt overforas till ett GUI-test. Vi ska i detta exempel implemen-tera tva enkla Use Cases i Seleniumtest.

Use Cases, Testkod, samt en enklare applikation finns tillganglig pahttps://github.com/anga13/UseCaseToGuiTestExample

1.1 Applikationen under test

Applikationen ar en sida dar en anvandare (Administrator) kan se en information om deanstallda pa en arbetsplats. Lanken “Today” visar en lista over vilka anstallda som harcheckat in pa en arbetsplats. Varje namn pa listan ar en klickbar lank till en detaljvysom visar information om den anstallde.

1.2 Use Cases

Use Case 1

1. Administratoren besoker sidan <url>/.

2. Administratoren trycker pa lanken“Today” i navigationsmenyn.

3. Administratoren ser en lista med de anstallda som har checkat in under dagen

Use Case 2

1. Administratoren besoker sidan <url>/today

2. Administratoren trycker pa en lank for en av de anstallda

3. Administratoren ser detaljerad information om den anstallda

https://github.com/anga13/UseCaseToGuiTestExample 1


1.3 Skapa Seleniumtest

Skapa testfilen app.test.js. Kopiera in Use Case-texten i testfilen som kommentar.Detta gor det lattare nar du skriver testerna.

app.test.js

/**

* Use Case 1:

* 1 Administratoren besoker sidan <url>

* 2 A trycker pa lanken "Today" i navigationsmenyn

* 3 A ser en lista med de anstallda som har checkat in under dagen

*/

describe(’UC 1: Logged in today’, () => {

test(’Clicking \’Today\’ link takes you to <url>/today’, () => {})

test(’A list of employees is shown’, () => {})

})

/**

* Use Case 2:

* 1 Administratoren besoker sidan <url>/today

* 2 A trycker pa en lank for en av de anstallda

* 3 A ser detaljerad information om den anstallda

*/

describe(’UC 2’, () => {})

1.4 Initiera Selenium Webdriver

Installera webdrivermodulen:

npm install --dev selenium-webdriver

Lagg till foljande rader kod i app.test.js

const {Builder, By} = require(’selenium-webdriver’)

const rootUrl = ’http://localhost:8080’

let driver

beforeAll(async () => {

await driver = new Builder()

.forBrowser(’chrome’)

.build()

})



afterAll(async () => {

await driver.quit()

})

WebDriver-objektet som simulerar weblasaren skapas med factoryklassen Builder. Duhar nu en utgangspunkt for dina tester. Dags for det roliga!

2 Use Case 1

2.1 Test 1: Klicka pa lankar

I Selenium anvander du valdigt ofta element. Dessa fas med metoder getElement() forett element eller getElements() for flera. Som argument till dessa funktioner ges ettBy-objekt som kan vara t. ex. ett namn pa ett element, attribut, eller ett XPath-uttryck.

Oppna startsidan med driver.get(rootUrl)

Hitta lanken till /today inuti navigationsfaltet <nav> och klicka pa den

driver.findElement(By.tagName(’nav’)).findElement(By.linkText(’Today’)).click()

Detta satt att kedja instruktioner ar mycket vanligt i Selenium.Du kan anvanda Jests expect-funktion for att kolla att du har kommit till den forvantade

sidan.

expect(driver.getCurrentUrl()).resolves.toBe(‘${rootUrl}/today‘)

Strangen ‘${rootUrl}/today‘ ar JavaScripts template-syntax for strang-interpolering.

2.2 Test 2: Hitta flera objekt

Tillvagagangssattet i detta test ar inledningsvis identiskt det i foregaende test. Vi kandarfor flytta de gemensamma raderna in i en beforeAll()-funktion. Lagg beforeAll()

fore testerna, men inuti describe()-blocket. Jest kor nu denna funktion fore tester-na, men enbart for Use Case #1. Jamfor med beforeAll()-funktionen dar vi initierarwebdrivern. Den funktionen ligger utanfor describe-blocken ock omfattar darfor alla tes-terna.

Vi lagger nu till det nya testet.

test(’A list of employees is shown’, async () => {

const employees = await driver.findElement(By.className(’employee-list’))

.findElements(By.className(’employee’))

expect(employees).toHaveLength(5)

})



Tillvagagangssattet kanns igen fran foregaende test. Vad som ar nytt ar metoden findElements(),som ger en array av element. Har forutsatter vi att listan av anstallda ar av langden 5.Eftersom GUI-tester ofta gors i en skyddad testmiljo, sa vet man i regel mer om vilkadata som kommer att visas. I en produktionsmiljo skulle det vara olampligt att goraett sadant antagande, om det inte skulle vara ett krav pa applikationen att t.ex. listanmaximalt far vara av en viss langd.

3 Use Case 2

Vi skapar skelettkod pa samma satt som i Use Case #1

/**

* Use Case 2:

* 1 Administratoren besoker sidan <url>/today

* 2 Administratoren trycker pa en lank for en av de anstallda

* 3 Administratoren ser detaljerad information om den anstallda

*/

describe(’UC2’, () => {

describe(’Reading employee details’, () => {

beforeEach(async () => {

await driver.get(‘${rootUrl}/today‘)

})

test(’Link URL corresponds to employee ID’, async () => {})

test(’Employee link leads to stat page for corresponding id’, async () => {})

})

describe(’The stat page’, () => {

beforeAll(async () => {

await driver.get(‘${rootUrl}/employees/3‘)

})

test(’Shows details for Employee’, async () => {})

test(’Displays correct value for all stats’, async (statId, statValue) => {})

})

})

3.1 Tester

I detta test kollar vi att listan bestar av lankar, och att dessa leder till ratt URL

test(’Link URL corresponds to employee ID’, async () => {

const employee = await driver.findElement(By.className(’employee’))

const [employeeID, employeeUrl] = await Promise.all([



employee.getAttribute(’employee-id’),

employee.findElement(By.tagName(’a’)).getAttribute(’href’)])

expect(employeeUrl).toBe(‘${rootUrl}/employees/${employeeID}‘)

})

Ur listan plockar vi det forsta elementet med klassattributet “employee”. Eftersom vari-ablerna employeeID och employeeUrl kan hamtas oberoende av varandra sa hamtar videm parallellt med Promise.all(), istallet for att vanta in dem i tur och ordning. Vikan sedan jamfora den forvantade URL:en med den faktiska.

3.2 Test av detaljvyn

Vi lagger nu till de sista testerna. Pa sidan /employees/<id> visas detaljerad informa-tion om den anstallde. Vi granskar en utvald sida och kontrollerar att element for samtligastatistikategorier finns, och att de ratta vardena visas. Har kommer Jests test.each()

till anvandning.

test.each([[’employee-id’, ’ID’],

[’employee-name’, ’Name’],

[’worked-hours’, ’Worked hours’]])

(’Shows details for %s’, async (statId, statText) => {

const text = await driver.findElement(By.id(statId))

.findElement(By.className(’stat-category’)).getText()

expect(text).toBe(statText)

})

test.each([[’employee-id’, ’3’],

[’employee-name’, ’Mr. Slate’],

[’worked-hours’, ’38’]])

(’Displays correct value for %s’, async (statId, statValue) => {

const text = await driver.findElement(By.id(statId))

.findElement(By.className(’stat-value’))

.getText()

expect(text).toBe(statValue)

})

Notera att dessa tester har kunnat skrivas utan nagon som helst tillgang till kallkod forsjalva applikationen under test!

Vi har i stor utstrackning anvant oss av klass- och id-attribut som selektorer. Dettager utvecklaren frihet i implementationen av sjalva appen. Listan over anstallda ochdetaljvyn kan vara i form av numrerad eller onumrerad lista, tabell, eller paragrafer.Om man anser att detta gor specifikationen alltfor vag, kan man givetvis anvanda sigav elementnamn som selektor. Ett bra tips om man vill krava att ett specifikt elementanvands ar att gora ett sarskilt test for detta, sa behover man bara andra just det testetom andringar sker i implementationen.


TRITA-EECS-EX-2019:630

www.kth.se

Teaching software testing in a modern development environment1356814/FULLTEXT01.pdf ·...

Documents

Transcript of Teaching software testing in a modern development environment1356814/FULLTEXT01.pdf ·...