Post on 12-Oct-2020
Malardalen UniversitySchool of Innovation Design and Engineering
Vasteras, Sweden
Thesis for the Degree of Master of Science in Computer Science withspecialisation in Intelligent Embedded Systems 30.0 credits
EVALUATING THE REAL-TIMECAPABILITIES OF ZEPHYR ON UPSQUARED - A PILOT STUDY WITH
ZEPHYR AND ACRN
Charlie Kotrocko14001@student.mdh.se
Jimmy Makkelijmi14002@student.mdh.se
Examiner: Thomas NolteMalardalen University, Vasteras, Sweden
Supervisors: Anna FriebeMalardalen University, Vasteras, Sweden
Shaik MohammedMalardalen University, Vasteras, Sweden
September 25, 2020
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
Abstract
In this study we have investigated if using Zephyr (User VM) and ACRN (Hypervisor) on the UPSquared (N3350) board are suitable for use in real-time scenarios. This was achieved by conductinga series of tests on Zephyr’s scheduling API functions such as busy-wait and sleep under differentsystem configurations. The tests involved running a task in isolation and taking timestamps ofthe current system cycle count before and after each function. To do this, we used the CommonTrace Format functionality in Zephyr to collect execution traces, which were then analyzed usingBabeltrace to convert the binary trace into human-readable form. The difference between the beforeand after timestamp were compared to the configured clock frequency to see if the system behaved asexpected. Our findings in this report suggest that Zephyr and ACRN on the UP Squared board arenot suitable for use in real-time scenarios due to large differences in system cycle count dependingon the configuration.
i
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
Table of Contents
1. Introduction 21.1. Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1..1 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Background 42.1. Hypervisors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2. ACRN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3. Zephyr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4. Open-source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4..1 Quality assessment of open-source software . . . . . . . . . . . . . . . . . . 62.5. Software tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.5..1 Common Trace Format - CTF . . . . . . . . . . . . . . . . . . . . . . . . . 72.5..2 SEGGER SystemView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.5..3 DTrace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.6. Babeltrace 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3. Related work 83.1. Hypervisors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.2. Quality assessment for open-source projects . . . . . . . . . . . . . . . . . . . . . . 83.3. Quality models for open-source projects . . . . . . . . . . . . . . . . . . . . . . . . 93.4. Risk management for open-source projects . . . . . . . . . . . . . . . . . . . . . . . 93.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4. Method 11
5. Ethical considerations 13
6. Implementation and problems 146.1. Zephyr problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146.2. ACRN problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146.3. Test program implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
7. Results 17
8. Discussion 21
9. Conclusions 22
10.References 23
Appendix A Appendix A: Raw cycle count data 251.1. ACRN Debian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.2. ACRN Debian strip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.3. ACRN Clearlinux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271.4. ACRN Clearlinux strip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281.5. ACRN Ubuntu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281.6. ACRN Ubuntu strip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291.7. Zephyr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301.8. Zephyr strip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301.9. Zephyr tickless . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311.10. Zephyr tickless strip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Appendix B Appendix B: Implementation source code 33
ii
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
List of Figures
1 Overwiew of different hypervisor types . . . . . . . . . . . . . . . . . . . . . . . . . 42 High level overview of ACRN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Device Passthrough . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Trace results of Sleep compared to the baseline in a logarithmic scale (x-axis). . . . 185 Trace results of Sleep compared to the baseline. . . . . . . . . . . . . . . . . . . . . 186 Trace results of Busy-wait compared to the baseline. . . . . . . . . . . . . . . . . . 197 Trace results of Busy-wait compared to the baseline. . . . . . . . . . . . . . . . . . 198 Trace results of Loop compared to the baseline. . . . . . . . . . . . . . . . . . . . . 209 Trace results of Loop compared to the baseline. . . . . . . . . . . . . . . . . . . . . 20
iii
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
Acknowledgements
We want to thank our supervisors Anna Friebe and Shaik Mohammed, for all their help duringthis thesis. They listened to us whenever we had problems during every phase of the work, theyalso provided us with a lot of feedback as well as help on how to proceed forward. We would alsolike to thank our families for always providing support and giving us that nudge forward.
1
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
1. Introduction
Open-source software (OSS) is software that is freely available to anyone, and you are allowed tomake changes to the software. This is different compared to closed-source software, which is usuallybought, and you are not allowed to freely change the software. Both types of software come withpros and cons. This is also true when it comes to the development of both types of software. Open-source software development is different from traditional development, and an example where thesetwo approaches to development differ is when it comes to the documentation. The documentationfor closed-source software tends to be more extensive than OSS documentation. Many studies haveshown that OSS development does not use many of the best practices, but does so without loss inquality [1]. But OSS development does keep some of the best practices such as central ownership,system testing, and planning and strategy meetings, among others [1]. It is not easy to select anOSS for your system since there are no set standards for determining the quality of the software[2]. Open-source software, have several benefits such as lowered development and maintenancecosts, and the ability to customize the software [3].
Virtual Machine Monitors (VMM), also called Hypervisors, sit between the hardware and theoperating system and manage resource distribution. This allows the system to run multiple otheroperating systems on the same hardware without them affecting each other [4]. ACRN is an open-source lightweight type-1 hypervisor, which means that it runs directly on the hardware as opposedto running on an operating system. It was built for development for the Internet of Things (IoT)[5]. ACRN was developed as collaborative work between the Linux foundation and Intel becauseof Intel’s expertise in virtualization [6]. Zephyr OS [7] is an open-source operating system forresource-constrained systems. It has a small memory footprint and supports a lot of features,such as multiple scheduling algorithms, among others. Another project from the Linux foundationis Akraino-edge stack, which has an integration project called Time-Critical Edge Compute [8],which uses both Zephyr and ACRN. The three main reasons for opting to use Zephyr and ACRNare: both systems are considered lightweight, made for IoT development, and there is no previousresearch of using them together.
Edge-computing is a paradigm in which the resources are brought to the edges of the computationalnetworks. By shifting where the computation and storage take place, it becomes possible tolower latency and decrease the bandwidth usage [9]. Some examples of edge-computing stacks areAkraino Edge Stack, Apache Edgent, and Azure IoT Edge [9]. Because of the increasing demandsfor more responsive applications and less usage of bandwidth, the real-time capabilities of edgecomputing are suitable to facilitate these types of applications or services [10]. Some examples ofapplications that would require real-time performance are multiplayer games, cloud games, andstreaming services [10].
As mentioned earlier in this section, both ACRN and Zephyr are made for IoT development.But according to some researchers, most existing hypervisors have a hard time fulfilling real-timerequirements [5]. Therefore it becomes interesting to investigate if these two systems can be usedfor real-time systems.
1.1. Problem formulation
This study will focus on finding out if Zephyr and ACRN are suitable for use in real-time systemswhen ran on UP Squared. It is not known how suitable these systems are for use in real-timesystems, and to our knowledge, there is no previous research that has used these two systemstogether in real-time systems. Therefore, this study will focus on the aspects related to commonfunctionalities that are used when scheduling tasks in the system, such as sleep functions and busy-waits. By using a hypervisor, extra overhead is added to the system because of this extra layerthat each request has to go through. But this extra amount of overhead is not known, thereforethis needs to be studied to be able to determine if the systems can be used in real-time scenarios.
2
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
1.1..1 Research questions
This is our research question that we are going to look into:
• RQ: What is the performance of common scheduling API methods when run on Zephyrstandalone and when Zephyr is run via ACRN?
Our research question was chosen because we are trying to determine if ACRN and Zephyr can beused for real-time systems, and therefore it is important to investigate how these systems performwhen scheduling tasks, which is an important part of a real-time system. As real-time systemsneed to be deterministic, it is important to know by how much these functions can differ from thespecifications.
3
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
2. Background
In this section, we will talk about topics that are related to this work, such as hypervisors andquality assessment for open-source software. Hypervisors are relevant as we are using a hypervisorourselves, and we are evaluating whether it is ready to be used in real-time scenarios. But to beable to use the hypervisor, we need to have an operating system, which the hypervisor can run,and therefore we opted to use Zephyr, as this is also a system which has not been used in this typeof work before. By assessing the quality of these two systems, we can determine if they can beused for real-time systems. A common method for gathering data related to scheduling is to usevarious tracing tools such as Babeltrace and Common Trace Format. Both the hypervisor and theRTOS we have chosen are open-source systems.
2.1. Hypervisors
Hypervisors, or Virtual machine monitors, can be divided into two categories: type-1 (bare-metal)or type-2 (hosted hypervisor). The type-1 hypervisor interfaces directly with the hardware anddoes not need an operating system, unlike the type-2 versions [11]. Figure 1 shows the differencesbetween type-1 and 2 hypervisors. Some examples of type-1 hypervisors are XenServer, KVM,and VMware ESXi [12]. And some examples of type-2 hypervisors are VMware -workstation, -fusion, and VirtualBox [13]. Hypervisors allow multiple different virtual operating systems to runat the same time while still having access to the same hardware. The hypervisor manages andmonitors the system resources while also handling requests for these resources. An advantage ofusing hypervisors is the isolation they give, in the sense that if one of the guest operating systemsis having problems, these problems do not affect the other guest systems [11].
Figure 1: Overwiew of different hypervisor types [14].
4
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
2.2. ACRN
This section contains information about ACRN from the paper ACRN: A Big Little Hypervisorfor IoT Development [5] and the ACRN documentation [15].
When it comes to IoT usages, it is not easy for existing solutions to handle real-time constraints.Figure 2 shows a high-level overview of ACRN and the different operating systems it can run.ACRN is developed from scratch to be a lightweight type-1 hypervisor. This means that ACRNhas a small code size and impact on performance, which is critical for industrial usage. Being atype-1 hypervisor, ACRN does not have to rely on RTOS services. The developers argue that thismakes porting it to other hardware architectures easier since there is no need to port an RTOSfirst, which can be a problem for other hypervisors since it would require extensive source codemodification. ACRN also has the benefit of not binding to a specific RTOS, which means thatit works with various operating systems on demand. Another benefit is it being open source andfree. [5]
Figure 2: High level overview of ACRN [15].
ACRN uses x86 as a starting point, compared to other solutions which do not have mature supportfor x86 embedded systems [5]. Some of the features that ACRN supports are rich I/O virtualization,device pass-through and, mediated pass-through. According to ACRN documentation, it is possibleto allocate a physical device exclusively to a guest operating system, which allows almost near-native performance [15]. In order to avoid complex CPU-emulation most CPU features such asCPUID, MSR, etc. are passed through. It also uses Intel VT-x to help with CPU virtualization.[5]
Figure 3: Device Passthrough [15].
The ACRN hypervisor supports three types of VMs: Service VM, User VM, and Real-Time VM,each able to run an OS. The User VM and RTVM are isolated in order to guarantee that theperformance of the application is running in real-time. By using dedicated pass-through, whichcan be seen in Figure 3, the RTVM minimizes the use of emulated devices at boot time. The
5
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
hypervisor passes through access to hardware for the RTVM to optimize real-time performanceand to avoid additional latency from the Service OS. [5]
To achieve isolation, the CPU core and memory partitioning are spatially isolated. This meansthat the hardware can not be accessed by a VM without it being assigned to it. Therefore, otherVMs have no access to process this data. [5]
ACRN can be run using four different scenarios: SDC (Software Defined Cockpit) with 2 VMs, IU(Industrial Usage) without Safety VM, IU with Safety VM, and Logical Partition. However, forour UP Squared board, the only verified scenario is SDC with 2 VMs.
2.3. Zephyr
Zephyr is a small-footprint kernel intended for use in embedded systems that are resource con-strained. These systems include sensors that are wearable, smart-watches, and IoT applications,among others. It supports a lot of different architectures and boards. Zephyr supports several dif-ferent scheduling algorithms and kernel services. According to the documentation, the system alsofeatures memory protection in the form of stack-overflow protection, kernel and object permissiontracking, and thread-level memory protection. [7]
2.4. Open-source
Open-source software is software that is available to anyone, and you are allowed to change, in-spect and redistribute the modified software [16]. This type of software is often supported by acommunity, which helps out with tasks such as debugging, testing, and even developing new fea-tures. A few of the advantages for companies using OSS is that it can lower the development costs,maintenance costs, and the company can change the software to suit their needs. But this does notmean that adopting OSS is risk-free, as studies have shown that most open-source projects end upin failure [3]. Keeping most open-source projects alive can be quite hard since the group of coredevelopers is usually small and without financial backing [17]. But they also need the help of thecommunity to debug, test, and maintain the software. Maintaining this community can be hard ifthe contributors do not feel that they can learn something new or take on challenges. Recognitionwas also a motivating factor for contributors as studies have shown that most programmers arelooking for social status [1].
2.4..1 Quality assessment of open-source software
Models and metrics are suitable methods to assess the quality of software [2]. To assess the qualityof software means to look at the different parts that make up the software, e.g. the documentationand the source code. But as mentioned before in the introduction, the lack of any standards makesthis quite challenging [18]. Models and metrics are not equally useful for everybody, e.g. modelsare more useful to system-designers than the end-user [2]. An advantage that metrics have isthat they can be used by anybody as they are just measurements. Some studies have shown thatover time as the software is undergoing development, the quality tends to decline due to increasedcomplexity. This means that it becomes harder to maintain the software. A lot of different factorscontribute to this loss of quality, such as lines of code, high coupling, or lower cohesion in methods[19].
2.5. Software tracing
Software tracing is a method of logging and recording execution data for a program. Some examplesof tracing software are Common Trace Format, SEGGER SystemView, and DTrace. Softwaretraces are useful for debugging as they contain valuable information about the execution of thesystem. One of the challenges with software tracing is getting valuable system information withoutaffecting the system in question. When a purely software-based tracing approach is used, it affectsthe execution of the software as the original source code is extended with tracing calls. This is
6
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
an important factor to take into consideration for real-time systems, as the system behavior ischanged. This is not the case with hardware-implemented tracing methods. [20]
2.5..1 Common Trace Format - CTF
Common Trace Format is a binary tracing format resembling C. It uses a declarative languagecalled Trace Stream Description Language (TSDL) and it allows for different stream layouts. Apacket contains:
• A header
• A context (which is optional)
• Zero or more events which call all contain a header, stream specific context (optional), anevent specific context (optional), and a payload.
The different data types that are supported in CTF are integers, null-terminated strings of bytes,and floats, among others. All of the mentioned information is taken from the homepage [21].
2.5..2 SEGGER SystemView
SEGGER SystemView is a program that can record and visualize what happens in the systemduring run-time. SystemView can record various events that happen in the system, such as Inter-rupt Service Routines, timers, and user-defined events, among others. According to the developer’shomepage, the system limits the amount of overhead introduced in the system by only recording asmall amount of data for each event. The developers also argue that when the system runs 10000events per second the overhead from SystemView is less than 1%. [22]
2.5..3 DTrace
DTrace is another tracing software that allows the user to trace all software according to theirwebsite. It uses the language D for writing scripts. DTrace stands for Dynamic Tracing and isreferring to that it can run and change tracing scripts during program execution. Another featurethat DTrace supports is static tracing, where the user defines trace-points before deployment. Byproviding filters that can filter the data in the kernel before the data is sent to the user, DTracecan reduce the overhead of presenting and gathering data. [23]
2.6. Babeltrace 2
Babeltrace 2 is the reference implementation of CTF [21, 24]. It is an open-source program thatconverts or processes traces. Babeltrace 2 is a completely rewritten version of the old library thatcomes with a lot of changes, some of these changes are full plugin support, Babeltrace 2 can runon any major operating system, and expanded certain features of the old library.
7
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
3. Related work
3.1. Hypervisors
J. Hwang et al. [25] compared four different hypervisors to each other. The researchers comparedthe different hypervisors with regards to CPU, memory, disk I/O, and network I/O. The resultsfrom the study showed that there was no clear indication of which hypervisor that was the best,rather that it depended on the workload. This was also the case in another study made by S.Shirinbab et al. [12] They compared three different type-1 hypervisors with each other when itcomes to CPU-utilization and Disc-utilization during different scenarios. The results showed thatthe three different hypervisors performed differently in each case and that there was no conclusiveevidence that one hypervisor was better than the others. This was not the results in the studyby S. Pawar and S. Singh [26], in which they compared VMware ESXi 6.0 and Citrix XenServer6.5. In their study, they compared the two hypervisors based on CPU, memory, disk, and systemup-time, on the guest operating system Microsoft Windows Server 2008 R2 (64-bit). The resultsshowed that the ESXi hypervisor was significantly better than XenServer.
J. Ruh and W. Steiner [27] compared the system latencies of the hypervisors: KVM, ACRNand, Xen RTDS. They evaluated each of the hypervisors based on four requirements, and theydetermined that none of the hypervisors available fulfilled all of the requirements. Based on this,the authors state that none of the hypervisors qualified as deterministic virtualization. Experimentsusing the RTEval tool was conducted to measure the system latency of each hypervisor. Eachhypervisor was tested twice, the first test involved running one VM, and the second test involvedrunning two VMs at the same time. Their results showed that KVM and ACRN had low averagesystem latencies, but could have peaks up to 1.968ms. On the other hand, Xen had higher averagesystem latency, but a maximum latency of 267µs. From these results, they drew the conclusionthat KVM and ACRN are not suitable for acting as control VMs because of the peaks in latency,almost reaching two milliseconds.
3.2. Quality assessment for open-source projects
Jospeh Tassone et al. [2] reviewed the body of research to find out which characteristics that affectthe quality of open-source projects the most. There are four main characteristics in which open-source projects (OSPs) differ from closed-source projects (CSPs): different levels of programmers,frequent beta releases, platform independence, and support for many run- and compile-time con-figurations. OSPs also differ from CSPs in regard to the development process as they do not usemore traditional processes such as the Waterfall model and that the development process changesover time as the project grows. An advantage OSPs have over CSPs is that they are available toanyone, which means that the larger the community around the project becomes, the more peoplecan help out. Two different approaches are used to measure quality: models and metrics. Butthese approaches are useful for different people in the project. Metrics is a useful tool to determineif there are problems within the project and can also be used to analyze errors. Models are usedto bring together different attributes related to quality to get a final result of the quality of thesoftware. An advantage of models is the utilization of metrics to establish a larger picture overthe process. But a disadvantage is that they can not be used by anyone, as they need to haveall the information needed to establish a working model. The authors conclude that both metricsand models are suitable for determining the quality of the software. Therefore, it is preferable toconcentrate effort on the idea of maintainability. Care should be taken when gathering metrics, asthere are a lot of tools that are not comparable with each other, and there are no standards setyet.
Other researchers have also looked at the differences between OSPs and CSPs but from anotherpoint of view. Mark Aberdour [1] wrote an article about what lessons can be learned and howthe development process is different between OSPs and CSPs. Some of the highlighted differencesbetween OSPs and CSPs were: effort put into planning, level of documentation, the methodologyregarding testing, peer review, and people management. Peer review was something that con-tributed a great deal to the quality of the code if the comments and the code got quick responses,
8
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
which helped keep the contributors interested. When it comes to managing the project, OSPshave a different mindset. In OSPs, the development process is more people-focused, which meanspeople volunteer to work on certain parts of the system. This results in a culture that fosterscreativity and innovation compared to more traditional development processes. The community ishighlighted as a key element for success in OSPs, as it is the community that builds new features,test and debug the system. But to sustain the community, the project must attract volunteeringprogrammers and motivate people to work by giving opportunities to take on challenges, learn newtechnology or tools. The author concludes that the study raised many questions but also foundsome guidelines for open-source software, some of which were: a large sustainable community, codemodularity, good documentation, rapid release cycles, and that the culture and the environmentare equally important as the system design.
3.3. Quality models for open-source projects
Normi Sham Awang Abu Bakar and Nadiah Arsat [16] studied which factors affected the quality ofopen-source systems the most. In the study, McCall’s Quality Factor Model was used to determinethe quality of two different open-source systems used by the Malaysian government: MyMeetingand MyTaskManager. The researchers also used six object-oriented measurements by Chidamberand Kemerer to measure quality. The PHP Depend tool was used to visualize the different statisticsof the two different systems. One result from the study was that there is no correlation between thenumber of lines of code and the depth of the inheritance tree. Another result was that the weightedmethods per class increased as the lines of code increased, which suggested to the researchers thatthe lines of code are a good indicator of complexity and coupling. This meant that the factors thataffected the quality the most are: correctness, maintainability, efficiency, and usability.
Won Jun Sung et al. [18] proposed in their work a quality model for selecting open-source software(OSS) components. This model determines the quality of an OSS component by looking at thedifferent characteristics of the component. The model has four main categories: functionality,reusability, usability and portability, and ten sub-categories. The researchers conclude that themodel provides basic measurements for the quality and that it is expected to be used to gatherpractical data.
3.4. Risk management for open-source projects
Xavier Franch et al. [3] propose a framework for risk management in OSS adoption processes.This proposed framework will provide the companies with tools and methods that are centeredaround a community-based development approach. Their framework builds upon the conceptsof business and software ecosystems. These different ecosystems handle different aspects of thedevelopment, the business ecosystem handles all things related to business such as production,distribution, and costs, while the software ecosystem handles all matters about software such aslicensing problems and release schedules. The platform provided by the framework is dividedinto two tiers: the decisional and the technological. The technological tier provides the necessaryobservations needed for the decisional tier to be able to make any decisions. One important dutyof the decisional tier is risk management, by utilizing the data provided from the technological tierand the business, it can detect potential risks related to the ecosystems. One of the most commonmistakes companies make when adopting OSS is having insufficient risk management, which canlead to costly failures. A scenario-based approach is planned to validate the framework.
Nguyen Duc Linh et al. [28] also studied the risks that come with using open-source software. Theresearchers studied different methods for assessing the risk involved with using OSS components ina system. Two of the different methods were: Qualitative and Quantitative analysis. In qualitativerisk assessment, each risk is evaluated based on the rate that they would occur and the severity.Some examples of tools that can help with qualitative risk management are the risk matrix andrisk breakdown structure. In quantitative risk assessment, a risk matrix is also used, which theproject manager uses. Risk assessment needs to be re-evaluated every time a major risk occurs,after phase end-reviews, and when the project needs to be re-planned. When a component has
9
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
been selected, different methods can be used to integrate it into the project, some examples thatthe researchers list are: having an expert as the supervisor, using a version control system, andbackup systems periodically. A case study was conducted in order to learn what difficulties thereare when integrating OSS components. The results from the case study showed that the communityfor Broadleaf was not strong, and Broadleaf’s products were not diverse. They also found that thecause for selecting unsuitable components came down to three different causes. The first cause wasthat the team had not fully defined their requirements. The second cause was that the team hadonly checked the homepage for Broadleaf and not checked the reviews of the system. The thirdand last cause was that the team had not compared Broadleaf to any similar products.
3.5. Summary
All of these different topics are connected to our work in at least one way. As mentioned earlierin the introduction text in the background section, both systems we have chosen to work withare open-source. There are various risks involved when selecting OSS and also the quality can bevarying as well. These risks can also be higher depending on the quality of the proposed softwaree.g. if the documentation is not understandable, then risks for failure might increase and thereforethe company might opt to select another software instead. One of the ways to assess the qualityof a software is to use metrics and models. These can be used to evaluate different parts of thesystem as well as different aspects such as portability, quality of documentation, and how easy thesystem is to use among others.
10
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
4. Method
We evaluated if Zephyr and ACRN can be used for real-time systems by conducting a series oftests on different configurations, e.g. different service VMs, clock frequencies, and tick-rates, whichcan be seen in Table 1. We ran the tests using two different clock frequency and tick-rate baselineconfigurations, these were 19.2 MHz with a tick-rate of 10000 ticks/s, and 25 MHz with a tick-rateof 100 ticks/s. These are the default settings when configuring Zephyr stand-alone and Zephyr forACRN during the build process. We then used these clock frequencies to calculate the differencebetween expected clock cycles taken and actual clock cycles taken. We used clock cycles as thesegave us the most accurate results, but also because many system functionalities like deadlines, andbusy-waits, etc. depend on clock cycles. As we have no expected value for the loop, we chose touse the Zephyr binary that was closest to having the correct cycle count (for sleep and busy-wait)as a baseline value for the loop. Strip means that the compiled executable does not contain debuginformation.
UART was used for serial communication and Common Trace Format (CTF) for creating traces.We opted to use CTF because it was readily available in Zephyr, and we did not have to buy anyadditional resources. We had planned to use SEGGER SystemView as it could present the resultsvery nicely. However, we could not find any information regarding if it was possible to use it onour board or not.
Configuration Clock frequency (MHz) Tick-rate (Ticks/s)ACRN Debian 19.2 10000
ACRN Debian Strip 19.2 10000ACRN Clearlinux 19.2 10000
ACRN Clearlinux Strip 19.2 10000ACRN Ubuntu 19.2 10000
ACRN Ubuntu Strip 19.2 10000Zephyr non-tickless 19.2 10000
Zephyr non-tickless Strip 19.2 10000Zephyr tickless 19.2 10000
Zephyr tickless Strip (Baseline for loop) 19.2 10000ACRN Debian 25 100
ACRN Debian Strip 25 100ACRN Clearlinux 25 100
ACRN Clearlinux Strip 25 100ACRN Ubuntu 25 100
ACRN Ubuntu Strip 25 100Zephyr non-tickless 25 100
Zephyr non-tickless Strip 25 100Zephyr tickless 25 100
Zephyr tickless Strip (Baseline for loop) 25 100
Table 1: Table showing the different configurations that we ran tests on.
The board we used is an UP Squared with an Apollo Lake Intel N3350 processor, 2 GB DDR4RAM, 32 GB eMMC. The versions of Zephyr and ACRN we have used are Zephyr v2.3.0 andACRN v2.0 with SDC as the scenario configuration. We created a test program which ran a singlethread containing three test functions. The functions that we tested were sleep, busy-wait, anda loop which ran for 100000 iterations. The sleep and busy-wait were configured so that theywould run for one second. To reduce the risk for errors, we ran the different functions four timeseach and averaged the results. This can be seen in Table 2, which shows an example using thebaseline configuration running the sleep function. During each test, we ran the test program forabout a minute before stopping the execution. While the test program was executing, we collectedthe tracing data in a text file. When we ran Zephyr directly on the UP Squared, we were ableto receive the CTF trace via the UART connection. However, when we ran ACRN, we could not
11
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
use the UART interface to capture the data, so we had to in the service VM pipe the CTF tracefrom a terminal to a text file. Because Zephyr printed out some extra text when it booted up thekernel, we had to remove it before we could read it with Babeltrace. To do this we used the sedcommand.
We created a set of timestamp functions in CTF to be able to see when a specific function hadbeen executed. These functions had a name that corresponded to the executed function e.g.ctf before sleep. We had to configure Zephyr to enable CTF, and we did so by going into aconfigurations menu when building the RTOS. In this menu, we had to go down to the optionTracing and enable CTF. We changed the size of the CTF buffer to 16 KB because we saw that insome cases there were some gaps in the trace. We also changed the tracing thread waiting periodto zero as this lead to a more responsive tracing experience. CTF produces a binary output, andwe used Babeltrace to read it. The default format for each timestamp is hh:mm:ss:ns, and becauseof this, we used a flag that converted them into clock cycles instead. Then to calculate the timetaken for each function, we calculated the difference between the before and after timestamps, andthis result was divided by the expected cycle count for each tested function, which can be seen inTable 3. This results in a number that describes how many times longer it took to execute eachfunction on every configuration compared to the baseline.
Before sleep After sleep Difference269221246 288424130 19202884288424317 307627332 19203015307627512 326829871 19202359326830052 346032877 19202825
Table 2: Example showing how the data was collected. This is the sleep function running onZephyr Strip tickless.
Configuration Sleep duration (cycles) More cycles taken relative to expected valueZephyr tickless Strip 19202770,75 1,00014431
ACRN Debian 7553942456 393,4345029ACRN Clearlinux 5961251437 310,4818457
Table 3: Calculated differences from the expected values.
12
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
5. Ethical considerations
In this work, there are no ethical considerations needed to be taken into account because the dataor the results will not be related to any company or person.
13
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
6. Implementation and problems
In this section, we will talk about the installation of Zephyr and ACRN, the various problems wehad during the installation phase. We will also talk about the solutions that we found to certainproblems and also how some problems are still unresolved.
6.1. Zephyr problems
This section will describe the problems we had with Zephyr.
• The sleep function does not sleep for the specified amount of time.
• The busy-wait function does not work with every configuration.
• Installing Zephyr when following the guide from the documentation.
• UART support for earlier versions for tracing data.
When compiling the system, it is possible to specify whether the system should use a tickless kerneland idle. Tickless idle disables periodic system clock interrupts when the kernel is idle. Ticklesskernel disables periodic clock system generation and turns the kernel into an event-driven kernel.We built the binaries with both tickless enabled and disabled and measured the time it took toperform functions such as sleep and busy-wait.
One of the problems we had was timing related when using the sleep and busy-wait functions. InZephyr v2.2.99 (Tickless), the sleep was shorter than specified. This problem seems to be fixedin v2.3 (Tickless). However, when tickless is not used, v2.2.99 would sleep for a shorter time andv2.3 for a longer time than specified. The busy-wait function seems to work with tickless enabledand disabled for v2.3, but only with tickless enabled for v2.2.99. When disabled, the thread wouldstop responding, and after a while, the system enters the idle state without finishing the remaininginstructions in the thread.
We also had problems installing Zephyr SDK v2.3 on another computer, following the guide in thedocumentation. The problem was related to an environment variable that we had to set, but thiswas missing from the guide. The solution to this problem was to go back to the guide for version2.2, and in that guide, that step was explained.
Due to UART support being introduced in v2.2, we have not been able to test these functions onearlier versions of Zephyr. This is because we rely on UART to get our tracing data. This wouldnot have been a problem if we had an alternative way of getting these data.
6.2. ACRN problems
This section will describe the problem we had using ACRN.
• Efibootmgr not saving the boot order correctly.
• Huge time differences in functions compared to running only Zephyr.
• Documentation and configuration files.
One step in the installation process of the hypervisor is to create a custom entry efibootmgr inLinux. The entry for the hypervisor is required to be the first entry, or the hypervisor will not beinitialized correctly during boot. The problem we had is that changes such as removing entries orchanging the order of them would not persist when rebooting. Because of this, we had to spendtime changing the boot order and rebooting until the board managed to save it. This was notdocumented when we started our work but was later found by another person that posted a pullrequest and updated the documentation to include this. Later we would find a solution to thisproblem that worked on our board (N3350). In the BIOS for the board, there is a setting called
14
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
UEFI harddisk drive BBS priorities that allowed us to force a change to the boot order. Thisallowed the hypervisor to initialize correctly on the first try when installing the hypervisor.
When running our test program on Zephyr using ACRN as the hypervisor, we noticed huge timedifferences in the tested functions compared to running Zephyr standalone. With one setup, thesleep function took ≈310.44 times as many cycles, the busy-wait ≈1.11 times and our simple loop≈27.35 times. Then with another setup, the sleep took ≈2.97 times as many cycles, while thebusy-wait and loop function took ≈1.56 times and ≈27.77 times. We measured the current clockcycle before and after each function and calculated the difference. This difference was then dividedby the cycles expected for the sleep and busy-wait functions, while using Zephyr v2.3 tickless asour baseline for the loop. We chose this as our baseline since the cycles reported for each functionclosely matched the cycles we expected it to take.
When we noticed this cycle difference, we posted an issue on the ACRN GitHub. We receivedan answer from one of the developers asking us about which ACRN and Zephyr version we wereusing. We were running acrn-2019w35.1-140000p with Zephyr 2.2.99. This was the only versionof ACRN that we had managed to install on our board. The developer then told us that theyhad only verified Zephyr v2.1 on ACRN v2.0 and asked us if we could test it again using theseversions. However, because of an update to ACRN, our board (N3350) was no longer the defaultboard when building, because it had been changed to the newer (N4200). This led to a problembecause the configuration files needed to build the hypervisor on our board for our scenario didnot exist. After discussing this, we received a configuration file which made it possible for us tobuild v2.0 for the N3350 board. But we could still not get measurements for Zephyr v2.1 since itlacked the UART support we relied on to get our trace measurements. Because of this, we had touse newer versions of Zephyr. Since the hypervisor was up and running, we wanted to get backto solving our original problem, posted on July 2nd, 2020, about the cycle differences. But as ofwriting this, we have yet to receive a response.
When installing ACRN, we had problems following some of the documentation. The guide usedfor our board (N3350) referenced the installation guide for another board (nuc7i7dnb) and notedchanges that had to be made in the installation process. This could make it difficult to keep trackof what changes had to be made and when to make these changes. There was also no mentionof the flag that had to be used in order to build the hypervisor for the UP Squared board (BothN3350 and N4200). We had to check the makefiles in order to find this flag. As mentioned before,we were told that Zephyr v2.1 had only been verified for ACRN v2.0. According to the developeranswering us in the GitHub issue, it was also only verified to run with the logical partition scenario.However, the documentation states that the only scenario that has been verified for Up Squared(both N3350 and N4200) is SDC with 2 VMs. The N3350 board did have scenario configurationfiles for logical partition mode but not SDC, while N4200 had configuration files for both.
15
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
6.3. Test program implementation
The pseudo-code for the program used to take all our measurements can be seen in Listing 1. Theactual code can be found in Appendix B, Listing 2. This program consists of two loops that testthe API functions sleep and busy-wait. There is also a loop doing a simple calculation. At thestart and end of each iteration, we measure the current system clock cycle by taking a timestamp.These timestamps are then used in order to calculate the number of cycles each function took sowe can compare that to the cycles they are expected to take given the system configuration.
1 void func ( )2 {3 i n t i = 4 ;45 // S leep6 whi le ( i−−) {7 t imes tamp cur r ent cyc l e ( ) ;8 s l e e p (1SECOND) ;9 t imes tamp cur r ent cyc l e ( ) ;
10 }1112 i = 4 ;13 // Busy−wait14 whi l e ( i−−) {15 t imes tamp cur r ent cyc l e ( ) ;16 busy wait (1SECOND) ;17 t imes tamp cur r ent cyc l e ( ) ;18 }1920 i = 4 ;21 // Loop22 whi l e ( i−−) {23 t imes tamp cur r ent cyc l e ( ) ;24 f o r ( i n t j = 0 ; j < 100000; j++) {25 v o l a t i l e i n t a = 2 ;26 a = a ∗ a ;27 }28 t imes tamp cur r ent cyc l e ( ) ;29 }30 }
Listing 1: Pseudo-code of the implementation .
16
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
7. Results
From our results, we can see that depending on the configuration, especially with regards to clockfrequency and tick-rate, the results changed when running ACRN. In Table 4, we can see thatthe sleep function took between 310 to 465 times more cycles than the expected number of cycles,while the busy-wait took barely any extra cycles at all. Finally, the loop took 27 times more cyclesthan the baseline. These results were taken when the system was running with a clock frequencyof 19.2 MHz and a tick rate of 10000 ticks/s.
Configuration Sleep Busy-Wait LoopACRN Debian 393,4345029 1,070640299 27,33158789
ACRN Debian Strip 402,8886401 1,111302643 27,32981885ACRN Clearlinux 310,4818457 1,107642656 27,34719077
ACRN Clearlinux Strip 307,2372667 1,000026185 27,08388763ACRN Ubuntu 447,1061502 1,076953177 27,21614067
ACRN Ubuntu Strip 465,5921071 1,101838633 27,33703651
Table 4: Difference from the baseline when running the test program on ACRN with a clockfrequency of 19.2 MHz and a tick rate of 10000 ticks/s.
Interestingly enough, when the clock frequency was changed to 25 MHz and the tick rate to 100ticks/s, the results were dramatically different from previous results, which can be seen in Table 5.When ran under this setup, we saw that the sleep function took around 2.25 to 3.98 times morecycles, the busy-wait did not change much (1.18 as maximum compared to 1.11 as the maximumfor 19.2 MHz), and the loop took around 41 times more cycles than the baseline.
Configuration Sleep Busy-Wait LoopACRN Debian 3,92751806 1,08976253 40,62336083
ACRN Debian Strip 3,65007764 1,0000202 41,04018613ACRN Clearlinux 2,28071853 1,19650823 41,05499154
ACRN Clearlinux Strip 2,24608165 1,08700284 40,80430415ACRN Ubuntu 2,66058113 1,18279499 40,67364636
ACRN Ubuntu Strip 3,98172415 1,0462964 40,62801396
Table 5: Difference from the baseline when running the test program on ACRN with a clockfrequency of 25 MHz and a tick rate of 100 ticks/s.
When running Zephyr directly on the board using a clock frequency of 19.2 MHz and a tick-rateof 10000 ticks/s, the cycle count for each test function did not differ that much from the expectedvalues, with the exception of the sleep function. When running Zephyr in non-tickless mode, thesleep function slept for about 40% more cycles than expected, but this did not happen when usingthe tickless mode. This can be seen in Table 6. But when the clock frequency and tick-ratewere changed to 25 MHz and 100 ticks/s, the results were a little different. The sleep functionactually woke up around 30% cycles earlier than expected, which can be seen in Table 7. Thecycles taken for the busy-wait and loop did not differ much between different configurations andnot even between the different clock frequencies and tick-rates.
Configuration Sleep Busy-Wait LoopZephyr non-tickless 1,405909128 1,000016745 1,061562412
Zephyr non-tickless Strip 1,405874466 1,000018867 1,060005661Zephyr tickless 1,000152891 1,000017135 0,999823097
Zephyr tickless Strip 1,00014431 1,000017526 1
Table 6: Difference from the baseline when running the test program on Zephyr directly on theboard with a clock frequency of 19.2 MHz and a tick rate of 10000 ticks/s.
17
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
Configuration Sleep Busy-Wait LoopZephyr non-tickless 0,69324044 1,00000043 1,000105753
Zephyr non-tickless Strip 0,6932399 1,00000043 1,000052876Zephyr tickless 0,69326735 1,00000042 1
Zephyr tickless Strip 0,69326683 1,00000044 1
Table 7: Difference from the baseline when running the test program on Zephyr directly on theboard with a clock frequency of 25 MHz and a tick rate of 100 ticks/s.
It is easier to see the differences in cycles taken when looking at a bar chart. We have separatedeach function and system configuration into its own charts, which can be seen below. Since thecycle difference was so large in the Sleep Difference 19.2 MHz chart, we had to use a logarithmicscale for the X-axis. This is not the case for the other charts.
1 10 100 1000
ACRN Debian
ACRN Debian Strip
ACRN Clearlinux
ACRN Clearlinux Strip
ACRN Ubuntu
ACRN Ubuntu Strip
Zephyr
Zephyr Strip
Zephyr Tickless
Zephyr Tickless Strip
Sleep Difference (19200000)
Figure 4: Trace results of Sleep compared to the baseline in a logarithmic scale (x-axis).
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5
ACRN Debian
ACRN Debian Strip
ACRN Clearlinux
ACRN Clearlinux Strip
ACRN Ubuntu
ACRN Ubuntu Strip
Zephyr
Zephyr Strip
Zephyr Tickless
Zephyr Tickless Strip
Sleep Difference (25000000)
Figure 5: Trace results of Sleep compared to the baseline.
18
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
0,94 0,96 0,98 1 1,02 1,04 1,06 1,08 1,1 1,12
ACRN Debian
ACRN Debian Strip
ACRN Clearlinux
ACRN Clearlinux Strip
ACRN Ubuntu
ACRN Ubuntu Strip
Zephyr
Zephyr Strip
Zephyr Tickless
Zephyr Tickless Strip
Busy Wait Difference (19200000)
Figure 6: Trace results of Busy-wait compared to the baseline.
0,9 0,95 1 1,05 1,1 1,15 1,2 1,25
ACRN Debian
ACRN Debian Strip
ACRN Clearlinux
ACRN Clearlinux Strip
ACRN Ubuntu
ACRN Ubuntu Strip
Zephyr
Zephyr Strip
Zephyr Tickless
Zephyr Tickless Strip
Busy Wait Difference (25000000)
Figure 7: Trace results of Busy-wait compared to the baseline.
19
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
0 5 10 15 20 25 30
ACRN Debian
ACRN Debian Strip
ACRN Clearlinux
ACRN Clearlinux Strip
ACRN Ubuntu
ACRN Ubuntu Strip
Zephyr
Zephyr Strip
Zephyr Tickless
Zephyr Tickless Strip
Loop Difference (19200000)
Figure 8: Trace results of Loop compared to the baseline.
0 5 10 15 20 25 30 35 40 45
ACRN Debian
ACRN Debian Strip
ACRN Clearlinux
ACRN Clearlinux Strip
ACRN Ubuntu
ACRN Ubuntu Strip
Zephyr
Zephyr Strip
Zephyr Tickless
Zephyr Tickless Strip
Loop Difference (25000000)
Figure 9: Trace results of Loop compared to the baseline.
20
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
8. Discussion
Many of the problems encountered during the installation of these systems stem from the docu-mentation being somewhat unclear at times. This lead to it being hard to figure out what couldhave gone wrong when the system did not function correctly. Also, the biggest hurdle was findingout how to consistently be able to install or re-install the hypervisor while also managing to get itto boot correctly every time. Most of the problems related to the documentation are easily fixedby just adding the missing information to the existing pages. This would not only make it easierto use the systems but also increase the quality of the documentation.
The differences in cycles taken between running Zephyr on the board directly and when run viaACRN are unexpectedly large. This suggests to us that there is a problem either with runningZephyr via ACRN or them running on our board. We did not have access to the other supportedboards for ACRN to make similar tests. Another problem for us was that Zephyr standalone wasnot supported on the NUC-line of boards that ACRN supported. Because of this, we could notperform any comparisons between the boards even if we had them.
When measuring the sleep duration on one ACRN configuration, the cycle count was very largecompared to another configuration (the highest being ≈465 times more cycles than expected). Forthe busy-wait function, the difference to the baseline was not that large, with ≈1.19 times morecycles than expected being the highest. The basic loop behaved differently depending on whichclock frequency and tick-rate we used. when running under ACRN with a clock frequency of 19.2MHz and a tick-rate of 10000 ticks/s, the loop took 27 times more clock cycles than expected onall configurations. But when the clock frequency was changed to 25 MHz and the tick-rate to 100ticks/s, the loop took around 41 times more clock cycles than expected on all configurations. Thiscan lead to problems since the system does not seem to behave as one would expect. The differentfunctions do not seem to exhibit the same behavior when it comes to the increases and decreasesin cycles taken to perform them. That is, the cycles taken for the sleep, busy-wait, and loop donot increase for all of them on one configuration. Rather sometimes one performs better on oneconfiguration and worse on another independently from each other.
We know that software-based tracing methods affect the execution on the hardware by addingtracing functions in the source code. However, we do not know how much CTF affects our testprogram when it is run on the systems. That is why we think it would be good to investigate howother tracing methods affect the results.
21
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
9. Conclusions
We have answered our research question about the performance of common scheduling API methodswhen run on Zephyr and ACRN by performing a series of tests. According to our results, the currentversions of Zephyr and ACRN are not ready to be used for real-time systems, at least not on theUP Squared board. This is because depending on the configuration, the time taken to performeach function varied greatly. When ACRN ran Zephyr using a 19.2 MHz clock frequency and atick-rate of 10000 ticks/s, the highest results for the sleep function was 465 times more cycles thanthe baseline, while the lowest result was 307 times more cycles, which is a substantial disparity.While the busy-wait ran for almost the specified time, producing results that only varied a littlebit, with the lowest being 1.000026 times more cycles and the highest being 1.11 times more cycles.The loop took around 27 times more cycles than expected, which is a very unexpected outcomehere. However, when the clock frequency was changed to 25 MHz and a tick rate of 100 ticks/s,the results became closer to the expected amount of time, but still off by a large margin. The sleepfunction on this configuration took around two to four times more cycles than the baseline, and thebusy-wait produced similar when running these settings (1.000020 - 1.19 times more cycles than thebaseline). The time it took for the loop did not change much between the different configurations,with 40.623 times more cycles being the lowest and 41.05 times more cycles being the highest.From these results, we can clearly see that the system does not behave in a deterministic manner,which is one of the more important attributes for a real-time system.
When we ran Zephyr directly on the board, the results for the busy-wait and the loop did not changemuch between the different configurations, and clock frequency, and tick-rate. The sleep function,however, changed behavior completely between the different setups. When running Zephyr non-tickless with a clock of 19.2 MHz and 10000 ticks/s, the sleep function slept for too long, around40%, but this did not happen in tickless mode. When the clock frequency was changed to 25 MHzand tick-rate to 100 ticks/s, the sleep function slept around 30% less than expected. This is alsosomething that is considered non-deterministic behavior.
We would argue that based on the problems we encountered during the installation of each systemthat the documentation for both systems needs more work in order to make them clearer andunderstandable. For both systems, the documentation is still usable but can be hard to follow oruse. However, some sections are missing some crucial information that needs to be added.
22
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
References
[1] M. Aberdour, “Achieving quality in open-source software,” IEEE Software, vol. 24, no. 1, pp.58–64, 2007.
[2] J. Tassone, S. Xu, C. Wang, J. Chen, and W. Du, “Quality assessment of open source software:A review,” in 2018 IEEE/ACIS 17th International Conference on Computer and InformationScience (ICIS), 2018, pp. 411–416.
[3] J. Franch Gutierrez, A. Susi, M. C. Annosi, C. P. Ayala Martınez, R. Glott, D. Gross,R. Kenett, F. Mancinelli, P. Ramsany, C. Thomas et al., “Managing risk in open sourcesoftware adoption,” in ICSOFT 2013: Proceedings of the 8th International Joint Conferenceon Software Technologies, 2013, pp. 258–264.
[4] O. Osanaiye, S. Chen, Z. Yan, R. Lu, K. R. Choo, and M. Dlodlo, “From cloud to fogcomputing: A review and a conceptual live vm migration framework,” IEEE Access, vol. 5,pp. 8284–8300, 2017.
[5] H. Li, X. Xu, J. Ren, and Y. Dong, “Acrn: A big little hypervisor foriot development,” in Proceedings of the 15th ACM SIGPLAN/SIGOPS InternationalConference on Virtual Execution Environments, ser. VEE 2019. New York, NY,USA: Association for Computing Machinery, 2019, p. 31–44. [Online]. Available:https://doi-org.ep.bib.mdh.se/10.1145/3313808.3313816
[6] Intel R© testimonial: Introducing acrnTM: A big little hypervisor for iot development.[Online]. Available: https://projectacrn.org/intel-testimonial/
[7] Zephyr project: Introduction. [Online]. Available: https://docs.zephyrproject.org/latest/introduction/index.html
[8] Time-critical edge compute. [Online]. Available: https://wiki.akraino.org/display/AK/Time-Critical+Edge+Compute
[9] J. Liang, F. Liu, S. Li, and Z. Cai, “A comparative research on open source edge computingsystems,” in Artificial Intelligence and Security, X. Sun, Z. Pan, and E. Bertino, Eds. Cham:Springer International Publishing, 2019, pp. 157–170.
[10] K. Bilal and A. Erbad, “Edge computing for interactive media and video streaming,” in 2017Second International Conference on Fog and Mobile Edge Computing (FMEC), 2017, pp. 68–73.
[11] N. Alam, “Survey on hypervisors,” Indiana University, Bloomington, School of Informaticsand Computing, 2009.
[12] S. Shirinbab, L. Lundberg, and D. Ilie, “Performance comparison of kvm, vmware andxenserver using a large telecommunication application,” in Cloud Computing. IARIA XPSPress, 2014.
[13] V. Timcenko, B. Djordjevic, S. B. Rakas, and N. Davidovic, “Performance examinationof type-2 hypervisors: Case of particular database application in a virtual environment,”in Proceedings of the International Conference on Information Systems and Design ofCommunication, ser. ISDOC ’14. New York, NY, USA: Association for ComputingMachinery, 2014, p. 122–126. [Online]. Available: https://doi-org.ep.bib.mdh.se/10.1145/2618168.2618187
[14] Wikipedia: Hypervisor. [Online]. Available: https://en.wikipedia.org/wiki/Hypervisor
[15] Introduction to project acrn. [Online]. Available: https://projectacrn.github.io/latest/introduction/index.html
23
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
[16] N. S. Awang Abu Bakar and N. Arsat, “Investigating the factors that influence the quality ofopen source systems,” in The 5th International Conference on Information and Communica-tion Technology for The Muslim World (ICT4M), 2014, pp. 1–6.
[17] G. Avelino, E. Constantinou, M. T. Valente, and A. Serebrenik, “On the abandonment andsurvival of open source projects: An empirical investigation,” in 2019 ACM/IEEE Interna-tional Symposium on Empirical Software Engineering and Measurement (ESEM), 2019, pp.1–12.
[18] W. J. Sung, J. H. Kim, and S. Y. Rhew, “A quality model for open source software selection,”in Sixth International Conference on Advanced Language Processing and Web InformationTechnology (ALPIT 2007), 2007, pp. 515–519.
[19] G. Singh, “Metrics for measuring the quality of object-oriented software,” SIGSOFTSoftw. Eng. Notes, vol. 38, no. 5, p. 1–5, Aug. 2013. [Online]. Available: https://doi-org.ep.bib.mdh.se/10.1145/2507288.2507311
[20] C. Koehler, A. Mayer, and M. Wurm, “Combined hardware and software tracing of real andvirtual embedded system parts,” in Proceedings of the 19th International Conference MixedDesign of Integrated Circuits and Systems - MIXDES 2012, 2012, pp. 340–345.
[21] The - common trace format. [Online]. Available: https://diamon.org/ctf/#specification
[22] Systemview. [Online]. Available: https://www.segger.com/products/development-tools/systemview/
[23] About dtrace. [Online]. Available: http://dtrace.org/blogs/about/
[24] Babeltrace - a trace manipulation toolkit. [Online]. Available: https://babeltrace.org/#bt2-get
[25] J. Hwang, S. Zeng, F. y. Wu, and T. Wood, “A component-based performance comparisonof four hypervisors,” in 2013 IFIP/IEEE International Symposium on Integrated NetworkManagement (IM 2013), May 2013, pp. 269–276.
[26] S. Pawar and S. Singh, “Performance comparison of vmware and xen hypervisor on guest os,”IJICSE, vol. 2, pp. 56–60, 2015.
[27] J. Ruh and W. Steiner, “The need for deterministic virtualization in the industrial internetof things,” in Proceedings of the Workshop on Fog Computing and the IoT, ser. IoT-Fog’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 26–30. [Online].Available: https://doi.org/10.1145/3313150.3313222
[28] N. D. Linh, P. D. Hung, V. T. Diep, and T. D. Tung, “Risk management inprojects based on open-source software,” in Proceedings of the 2019 8th InternationalConference on Software and Computer Applications, ser. ICSCA ’19. New York, NY,USA: Association for Computing Machinery, 2019, p. 178–183. [Online]. Available:https://doi-org.ep.bib.mdh.se/10.1145/3316615.3316648
24
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
A Appendix A: Raw cycle count data
Here is the raw data containing all the cycle counts measure before and after the function aswell as the difference between them. Each section contains all the measurements taken for eachconfiguration. Because of the size of these tables we could not fit them on this page. You can findthem on the next pages.
25
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
1.1.
ACRN
Debian
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce23
3183
7746
9956
2033
2776
2436
5581
32547609462
32566809949
19200487
32629845052
32630042612
197560
9956
2037
6517
6339
9029
876
7778
6533
32566812503
32586066888
19254385
32630042858
32630233364
190506
1763
3990
769
2517
3830
035
7539
8392
6632586069522
32607954920
21885398
32630233531
32630424102
190571
2517
3830
529
3254
7608
971
7373
7784
4232607957508
32629842413
21884905
32630424257
32630618120
193863
Tab
le8:
Con
figu
rati
on
:1000
tick
s,19200000H
z
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce27
2325
7699
2843
5037
0012
0246
001
3116011401
3141011896
25000495
3224998210
3225188869
190659
2843
5041
3229
4092
6060
9742
1928
3141014485
3174989260
33974775
3225189064
3225382166
193102
2940
9265
4930
2405
8765
8313
2216
3174991937
3199992416
25000479
3225382392
3225576416
194024
3024
0592
4731
1601
0908
9195
1661
3199994999
3224995503
25000504
3225576581
3225767065
190484
Tab
le9:
Con
figu
rati
on
:100
tick
s,25000000H
z
1.2.
ACRN
Debian
strip
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce27
8738
1016
1039
3517
079
7606
1360
6333729230327
33748922642
19692315
33814588521
33814786059
197538
1039
3517
549
1814
6230
790
7752
7132
4133748925191
33770810604
21885413
33814786311
33814976818
190507
1814
6231
088
2615
7270
444
8011
0393
5633770813142
33792697796
21884654
33814976984
33815167546
190562
2615
7270
947
3372
9229
849
7571
9589
0233792700334
33814585995
21885661
33815167702
33815361545
193843
Tab
le10:
Con
figu
rati
on
:1000
tick
s,19200000H
z
26
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce13
1043
782
2192
4277
488
1989
92485192513
510192945
25000432
593339311
593533272
193961
2192
4325
731
2815
591
9357
2334
510195545
543330402
33134857
593533466
593726574
193108
3128
1607
339
3939
597
8112
3524
543333020
568333546
25000526
593726799
593917285
190486
3939
4006
348
5192
004
9125
1941
568336156
593336661
25000505
593917448
594111486
194038
Tab
le11:
Con
figu
rati
on
:100
tick
s,25000000H
z
1.3.
ACRN
Clearlinux
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce40
5264
3619
1006
7686
159
6015
0425
4027897651281
27917242022
19590741
27982728720
27982922850
194130
1006
7686
648
1601
7405
535
5949
7188
8727917244650
27939125798
21881148
27982923118
27983120959
197841
1601
7405
991
2197
5320
019
5957
9140
2827939128384
27960815847
21687463
27983121132
27983311618
190486
2197
5320
497
2789
7650
789
5922
3302
9227960818427
27982726031
21907604
27983311852
27983502336
190484
Tab
le12:
Con
figu
rati
on
:1000
tick
s,19200000H
z
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce23
5861
7710
2420
8904
7762
2727
672586691212
2611691699
25000487
2706352232
2706
550260
198028
2420
8909
5924
7617
5571
5528
4612
2611694205
2647492426
35798221
2706550511
2706
744563
194052
2476
1760
5825
3144
8463
5527
2405
2647495038
2672495590
25000552
2706744822
2706
935309
190487
2531
4488
8225
8669
0951
5524
2069
2672498010
2706349573
33851563
2706935472
2707
129337
193865
Tab
le13:
Con
figu
rati
on
:100
tick
s,25000000H
z
27
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
1.4.
ACRN
Clearlinuxstrip
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce22
1180
2961
8217
8763
5460
0607
3393
25807626921
25826827408
19200487
25884438802
25884629455
190653
8217
8768
0813
9021
8575
856
8430
8950
25826829843
25846030356
19200513
25884629658
25884823534
193876
1390
2186
220
1982
9988
948
5927
8027
2825846032841
25865233308
19200467
25884823690
25885014176
190486
1982
9989
393
2580
7626
408
5977
6370
1525865235807
25884436351
19200544
25885014410
25885204894
190484
Tab
le14:
Con
figu
rati
on
:1000
tick
s,19200000H
z
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce31
7551
1020
3230
5159
4555
0049
253400121141
3425121656
25000515
3508831530
3509
025560
194030
3230
5164
2932
8578
1655
5526
5226
3425124117
3458822845
33698728
3509025772
3509
218913
193141
3285
7821
0933
4430
7144
5852
5035
3458825536
3483826056
25000520
3509219173
3509
409661
190488
3344
3076
4034
0012
0619
5581
2979
3483828548
3508829069
25000521
3509409824
3509
603856
194032
Tab
le15:
Con
figu
rati
on
:100
tick
s,25000000H
z
1.5.
ACRN
Ubuntu
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce25
7593
4843
1126
6979
285
8691
0444
4236913689212
36932889636
19200424
36996409742
36996603931
194189
1126
6979
806
1983
3180
637
8566
2008
3136932892179
36952631521
19739342
36996604191
36996794773
190582
1983
3181
140
2839
5986
880
8562
8057
4036952634161
36974518818
21884657
36996794960
36996988943
193983
2839
5987
344
3691
3688
669
8517
7013
2536974521396
36996406977
21885581
36996989135
36997179618
190483
Tab
le16:
Con
figu
rati
on
:1000
tick
s,19200000H
z
28
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce39
6526
5973
4038
7109
6973
4449
964231326108
4264207924
32881816
4349616521
4349
810850
194329
4038
7114
9441
0556
4549
6685
3055
4264210717
4289211214
25000497
4349811121
4350
001645
190524
4105
5650
5241
7067
1225
6510
6173
4289213906
4314214448
25000542
4350001875
4350
195759
193884
4170
6716
9242
3132
5581
6065
3889
4314217102
4349613746
35396644
4350195924
4350
386407
190483
Tab
le17:
Con
figu
rati
on
:100
tick
s,25000000H
z
1.6.
ACRN
Ubuntu
strip
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce69
9063
043
9519
8696
9588
2080
6652
36456538850
36475739278
19200428
36541170036
36541367689
197653
9519
8701
9818
7956
5901
792
7578
8819
36475741713
36497392692
21650979
36541367936
36541558525
190589
1879
5659
522
2752
7326
353
8731
6668
3136497395253
36519280279
21885026
36541558682
36541749184
190502
2752
7326
842
3645
6538
368
8929
2115
2636519282795
36541167569
21884774
36541749335
36541943245
193910
Tab
le18:
Con
figu
rati
on
:1000
tick
s,19200000H
z
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce23
4048
9948
2447
4731
6010
6983
212
2738664406
2763665038
25000632
2843304546
2843497813
193267
2447
4736
2425
5527
2179
1077
9855
52763667636
2788668139
25000503
2843498055
2843692183
194128
2555
2727
2426
4556
0543
9028
7819
2788670706
2818298701
29627995
2843692384
2843882866
190482
2645
5610
8327
3866
3912
9310
2829
2818301374
2843301884
25000510
2843883025
2844073505
190480
Tab
le19:
Con
figu
rati
on
:100
tick
s,25000000H
z
29
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
1.7.
Zephyr B
efor
esl
eep
Aft
ersl
eep
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce28
6466
711
3134
6883
027
0021
19394441257
413641611
19200354
471243375
471250884
7509
3134
6901
134
0450
251
2698
1240
413641901
432842209
19200308
471251073
471258577
7504
3404
5043
436
7437
604
2698
7170
432842389
452042717
19200328
471258757
471266261
7504
3674
3778
539
4441
077
2700
3292
452042899
471243195
19200296
471266442
471273929
7487
Tab
le20:
Con
figu
rati
on
:1000
tick
s,19200000H
z
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce25
0620
983
2679
1227
617
2912
93319945073
344945088
25000015
419945151
419949881
4730
2679
1228
828
5256
507
1734
4219
344945097
369945106
25000009
419949890
419954618
4728
2852
5651
730
2600
765
1734
4248
369945114
394945123
25000009
419954625
419959353
4728
3026
0077
531
9945
059
1734
4284
394945132
419945142
25000010
419959360
419964088
4728
Tab
le21:
Con
figu
rati
on
:100
tick
s,25000000H
z
1.8.
Zephyrstrip
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce28
8952
972
3159
6887
627
0159
04396924860
416125209
19200349
473727037
473734537
7500
3159
6905
734
2942
681
2697
3624
416125389
435325845
19200456
473734717
473742182
7465
3429
4286
436
9930
499
2698
7635
435326030
454526394
19200364
473742363
473749854
7491
3699
3068
039
6924
676
2699
3996
454526575
473726855
19200280
473750034
473757538
7504
Tab
le22:
Con
figu
rati
on
:1000
tick
s,19200000H
z
30
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce33
9710
338
3570
0155
017
2912
12409034375
434034389
25000014
509034450
509039180
4730
3570
0156
237
4345
842
1734
4280
434034397
459034407
25000010
509039189
509043917
4728
3743
4585
339
1690
151
1734
4298
459034414
484034424
25000010
509043924
509048652
4728
3916
9016
140
9034
361
1734
4200
484034432
509034441
25000009
509048659
509053386
4727
Tab
le23:
Con
figu
rati
on
:100
tick
s,25000000H
z
1.9.
Zephyrtick
less
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce33
5219
875
3544
2342
719
2035
52412032340
431232666
19200326
488834383
488841450
7067
3544
2360
937
3627
685
1920
4076
431232848
450433206
19200358
488841631
488848696
7065
3736
2786
539
2830
326
1920
2461
450433387
469633740
19200353
488848877
488855939
7062
3928
3050
741
2032
160
1920
1653
469633923
488834202
19200279
488856120
488863185
7065
Tab
le24:
Con
figu
rati
on
:1000
tick
s,19200000H
z
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce22
8650
413
2459
4239
717
2919
84297977195
322977209
25000014
397977268
397981998
4730
2459
4240
926
3287
313
1734
4904
322977218
347977227
25000009
397982007
397986734
4727
2632
8732
328
0632
269
1734
4946
347977235
372977245
25000010
397986742
397991470
4728
2806
3227
929
7977
180
1734
4901
372977252
397977261
25000009
397991477
397996204
4727
Tab
le25:
Con
figu
rati
on
:100
tick
s,25000000H
z
31
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
1.10.
Zephyrtick
less
strip
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce26
9221
246
2884
2413
019
2028
84346033059
365233333
19200274
422835133
422842201
7068
2884
2431
730
7627
332
1920
3015
365233514
384433874
19200360
422842382
422849448
7066
3076
2751
232
6829
871
1920
2359
384434057
403634409
19200352
422849629
422856694
7065
3268
3005
234
6032
877
1920
2825
403634589
422834949
19200360
422856874
422863939
7065
Tab
le26:
Con
figu
rati
on
:1000
tick
s,19200000H
z
Bef
ore
slee
pA
fter
slee
pD
iffer
ence
Bef
ore
Loop
Aft
erL
oop
Diff
eren
ceB
efore
Loop
Aft
erL
oop
Diff
eren
ce22
5924
510
2432
1643
017
2919
20295251242
320251256
25000014
395251316
395256046
4730
2432
1644
326
0561
350
1734
4907
320251265
345251275
25000010
395256055
395260782
4727
2605
6136
027
7906
313
1734
4953
345251282
370251293
25000011
395260790
395265518
4728
2779
0632
329
5251
226
1734
4903
370251300
395251309
25000009
395265525
395270252
4727
Tab
le27:
Con
figu
rati
on
:100
tick
s,25000000H
z
32
Charlie Kotro, Jimmy Makkeli Evaluating the RT capabilities of ACRN and Zephyr on UP2
B Appendix B: Implementation source code
This is the full version of the implemented function that we use to test sleep, busy-wait and oursimple loop. The program consists of a single thread running three loops. Each loop will performa a test and a timestamp is collected before and after the function. The ctf-functions are customCTF messages that contains the current system clock cycle. This makes it easier to see whichtimestamps that belong to which functions, as well as if the timestamp is before or after it. TheseCTF messages are captured over UART and stored in a binary trace file that can be translatedinto a human-readable format by Babeltrace.
1 void func ( void ∗p1 , void ∗p2 , void ∗p3 )2 {3 i n t i = 4 ;45 // S leep6 whi le ( i−−) {7 c t f b e f o r e s l e e p ( k c y c l e g e t 3 2 ( ) ) ;8 k s l e e p (K MSEC(1000 ) ) ;9 c t f a f t e r s l e e p ( k c y c l e g e t 3 2 ( ) ) ;
10 }1112 i = 4 ;13 // Busy−wait (1 second )14 whi l e ( i−−) {15 c t f b e f o r e bu s ywa i t ( k c y c l e g e t 3 2 ( ) ) ;16 k busy wait (1000000) ;17 c t f a f t e r b u s ywa i t ( k c y c l e g e t 3 2 ( ) ) ;18 }1920 i = 4 ;21 // Loop22 whi l e ( i−−) {23 c t f b e f o r e l o o p ( k c y c l e g e t 3 2 ( ) ) ;2425 f o r ( i n t j = 0 ; j < 100000; j++)26 {27 v o l a t i l e i n t a = 2 ;28 a = a ∗ a ;29 }30 c t f a f t e r l o o p ( k c y c l e g e t 3 2 ( ) ) ;31 }32 }
Listing 2: The implementation of the test program.
The k sleep function takes K MSEC as a parameter, K MSEC converts the number of millisecondsto the correct format in the k timeout t data structure, which is required for the k sleep function.
In this case, the k sleep function will sleep for one second. The k busy wait function takes mi-croseconds as input and will also run for one second. However, there are certain instances wherethe k busy wait function will run for more time than specified by the input. We did not measurethe time taken for one iteration of the loop in order to set the number of iterations to a numberthat would also take one second.
33