Performance Evaluationof Real-Time Operating Systems

RTEMS, RTLinux, eCos

Real-Time Systems

Systems that interact predictably with events in the outside world

Examples: Flight control systems Collision avoidance systems Satellite guidance systems Patient Monitoring System Control Systems for synchrotrons

Real-Time Operating Systems

Free the applications programmer from writing code for task scheduling and dispatching.

Manage time-sharing of a processor for a number of different tasks and interrupt sources, while adhering to strict time constraints.

Widely used in all kinds of applications both on PCs and in embedded systems.

Generally operate unattended -- consequences of failure can be catastrophic.

What is available?

Many RTOSs have been developed: Commercial Open source

RTOS vendors and developers publish performance metrics to showcase their products. However, their evaluations are: not performed on a common platform not comprehensive

Evaluation of an RTOS

How would you choose an appropriate RTOS for your application? Need for an impartial evaluation.

Several such projects have been conducted. Only evaluated some performance characteristics Only looked at a few commercial operating systems Many are now out of date

‘Real-Time Consult’ introduced a more comprehensive project. Developed a thorough test methodology Evaluated several commercial RTOSs Have not evaluated open source RTOSs

Research Objective

Performance evaluation, on a common platform, of three open source real-time operating systems: RTEMS RTLinux eCos

Real-Time Operating Systems

RTOSRTOS

TaskScheduling

TaskDispatching

TaskSynchronization

Overview

Task Scheduling

Time Slicing

Task 1

ArrivalOrder

First

Last

Task 2

Task 3

Task 1

Task 2

Task 3

Time

Task Scheduling

Preemptive Priority

Priority

Low

High

Task 1

Task 2

Task 3

Task 1

Task 2

Time

Task Dispatching

Task-control blocks is the most popular method of identifying and managing tasks.Task-Control Block (TCB) contains:

a context (e.g. program counter and register contents)

an identification string a status, such as ready, executing or blocked a priority (if applicable)

A Typical Task Control Block

Program Counter

Task Status

Task ID Number

Pointer to next TCB

Other Context

Priority

:

Register Contents

Task Synchronization

Events

Messages

Semaphores

Mutexes

Desired Performance Characteristics

Tasks should run with: minimal event latency (the time between the

triggering of a task and its start) minimal jitter (the variation in running times of

a task that is supposed to run at a fixed period).

Performance Evaluation

Throughput – speed at which the system executes instructions

Responsiveness – how fast it starts to handle in interrupt request

Determinism – how it reacts under load how long does it take to finish what it is doing

and start handling the interrupt request

Performance Metrics

Low Level Tests Context switching Interrupt latency Exclusion objects

Semaphores Mutexes

Synchronization events

Context Switch Time

Task ATask A

Save contextof Task A

Save contextof Task A Start

Task BStart

Task B

Time

Switch to Task B

Context Switch Time

PC

Register 0

::

Other Context

Load contextof Task B

Load contextof Task B

PC

Register 0

::

Other Context

Other Considerations

Not an isolated event

Affected by number of tasks pending

Depends on priorities of pending tasks

Amount of context to be saved

Interrupt Latency

Interrupt and Interrupt Dispatch

Latencies

Time

TaskTask

Interrupt Occurs

ISRISR

First Instruction inInterrupt Service Routine

(ISR) is executed

TaskTask

Taskresumes

ISR ends

InterruptLatency

Interrupt Dispatch Latency

Multiple Interrupt Latencies

Interrupt-to-Task Run

Interrupt Dispatch Time

Interrupt Service Routine

Other Interrupt

Pre-emption Disabled

Scheduling

Context Switch

Return from System Call

InterruptInterrupt

Task RunTask Run

Interrupt Task Response Time

Real response to an interrupt often occurs in a task synchronized by, but outside of, the actual ISR.

Task response also depends on whether the kernel is preemptable. If not, then kernel call must be completed before ISR.

Performance Metrics

High Level Tests

Network throughput

Stress tests

Real-Time Consult Project

Evaluation and comparison of RTOS performance

Commercial systems only, to date

Evaluations available for: Intime 1.20 RTX 4.2 Hyperkernel 4.3 VxWorks/x86 5.3.1 pSOSystem/x86 2.2.6 QNX 6.1 CE 3.0

Real-Time Consult Project

Test Method: timing is measured using an external PCI bus analyzer.

Two types of tests: Performance Tests Stress Tests

Qualitative Evaluation (API richness, etc.)

Real-Time Consult Project

Performance Tests (context switch & latencies): Thread switch latency

time required to preempt current thread Interrupt latency

preempt current thread and start interrupt handler Interrupt dispatch latency

switch from interrupt context to context of interrupted thread or next thread in queue

Thread creation and deletion

Task Switch Latency Test

Create tasksTSK1, …, TSKN

Task startTSK0

Delete tasksTSK1, …, TSKN

Write trace on PCIBus

TRC_TSK0_TS0

YieldProcessor

Yes

No

End of Test?

Task endTSK0

Each Task

Write trace on PCIBus

TRC_TSKN_TSN

YieldProcessor

Task startTSKN

Real-Time Consult Project

Performance Tests (objects, file, network): Synchronization objects

time to create and delete a synchronization object Exclusion objects

time to create and delete an exclusion object File system operations

creating and deleting files, reading from files and writing to files in synchronous mode

Network stack performance of TCP/IP stack bandwidth for various packet sizes, and CPU usage

Real-Time Consult Project

Stress Tests: Two simultaneous interrupts Interrupt nesting maximum sustainable interrupt frequency maximum number of objects memory leaks

Focus of my Research

Evaluation of three Open Source Real-Time Operating Systems: RTEMS

Real Time Executive for Multiprocessor Systems RTLinux

Real Time Linux eCos

Embedded Configurable Operating System

RTEMS

Developed by the U.S. military as alternative to using commercial RTOS

Small, easy to port

High level of user configurability

Kernel is preemptible

Target HardwareTarget Hardware

Application Dependent SoftwareApplication Dependent Software

Standard Application ComponentsStandard Application Components

Device DriversDevice Drivers R T E M SR T E M S

RTLinux

Abstraction layer between the hardware and the standard Linux kernel Appears as actual hardware to standard Linux

kernel Lowest priority is assigned to standard Linux

kernel, which then runs as a independent task.

RTLinux executive is nonpreemptibleLeaves Linux kernel essentially untouched so it doesn’t hinder future Linux development

ECOS

Targeted at high-volume applications: consumer electronics, telecommunications, automotive other deeply embedded applications.

Configurable: lets developer configure a system that best matches the

needs of the application. Typical configuration options:

type of scheduler number of priority levels.

current release of the system has over 200 options

PortableFully preemptable

Typical Application

Control system for the Canadian Light Source (CLS) uses an open source real-time operating system and control software:

RTOS RTEMS Control Software EPICS

(Experimental Physics and Industrial Control System)