0

ARTIST2 Summer School 2008 in EuropeARTIST2 Summer School 2008 in EuropeAutrans Autrans (near (near GrenobleGrenoble), France), France

September 8September 8--12, 200812, 2008

Invited Speaker: David F. BaconInvited Speaker: David F. Bacon

IBM ResearchIBM Research

Real-Time Programmingin Java

Real-Time in the Age of Complex Systems

1

Classical Real-Time Control

Anti-lock Braking System

SENSE

COMPUTE

ACTUATE

2

Complex Real-Time Systems

3

What’s Changing?

• Processing Power (instructions per 10ms)– 1970: 1K Now: 40M 2020: 1G

• Ubiquitous Sensing and Actuation– e.g. video stream per cell phone

• More and more computing is real-time

4

Implications for Real-Time Systems

• Much Larger (in code and scope)

• Highly Dynamic

• Non-deterministic– non-determinism of underlying system desirable

• Different Kind of Software Engineering

5

Memory ManagementDYNAMIC

(malloc/free)

• C• C++• Ada• Pascal

(new/garbage collect)

• Java• Lisp• C#• Smalltalk

STATIC(arrays)

• Fortran• Esterel• Verilog

Determinism Verifiability

Flexibility Software Engineering

6

RE

SE

AR

CH

CO

MM

ITTE

D T

OP

RO

DU

CTI

ON

DDG-1000 Destroyer

Telco SIP Switch

Air Java(w/ Berkeley CE)

JAviator(w/ Salzburg)

Java-basedSynthesizer

TAR

GE

TED

Playstation/Xbox etc

Automotive Electronics

33%

7%

22%

Trade Execution

7

Three Approaches in Java

Metronome:Real-Time Garbage Collection

Flexotasks:Time-Portable Java

RTSJ Standard:Scoped/Immortal Memory

8

Why is Real-Time Difficult in Java?• No real-time scheduling APIs

• Garbage Collection

• Dynamic Class Loading

• Just-in-Time (JIT) Compilation

• Dynamic, average-case-based optimization

9

Real-Time Specification for Java

Two primary additions to the Java language:

– Real-Time Scheduling APIs

– Semi-manual memory management

(real-time garbage collection “known” impossible)

10

Basic Java Memory Architecture

Stack

rr

pp TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

HeapStack

ss

qq

class Foo {Foo a;Foo b;

}

11

RTSJ Memory Architecture

Stack

rr

ppXXa

b

ZZa

b

WWa

b

YYa

b

Heap

Stack

ss

qq

Immortal

Inner Scope

Scope Scope

AAa

b

BBa

b

DDa

b

CCa

b

TTa

b

UUa

b

Control LoopIteration

12

Inner Scope

Outer Scope

aa bb

Heap

Inner Scope

Outer Scope

Heap

aa

bb

Inner Scope

Outer Scope

Heap

aa

bb

aa bb

aa

bb aa

bbaa bb

f(a,b) { a.x = b; }

13

RTSJ Memory Management Issues• Complex Programming Model

• Non-compositional

• Run-time Failures

• Checking Overhead

• Storage Leaks in Immortal Memory

• Often Poorly Suited (e.g. Producer/Consumer)

14

Metronome:Real-Time Garbage Collection

15

Stack

GC: A Simple Problem (?)

• Transitive Graph Closure• Application is Stopped During Collection• Memory is only freed at the end

rr

pp TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

16

Basic Approaches: Mark/Sweep

Stack

rr

pp TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

WWa

b

ZZa

b

YYa

b

XXa

b

freefree

• O(live) mark phase but O(heapsize) sweep• Usually requires no copying• Mark stack is O(maxdepth)

17

Basics II: Semi-space Copying

• O(live)• If single-threaded, no mark stack needed• Wastes 50% of memory

Stack

rr

pp

TTa

b XXa

b

UUa

b ZZa

b

WWa

b

YYa

b

WWa

b

ZZa

b

YYa

b

XXa

b

18

Demo: Synthesizer in Java

• Human performance:– Latency < 8ms end-to-end– Jitter < 10us

• MIDI/soundcard latency: 2.5ms

19

Kinds of “Concurrent” Collection

• “Stop the World”

• Parallel

• Concurrent

• Incremental

APP GC APPAPP APPAPP APP

APP GC APPAPP APPAPP APP

GCGC

APPAPP

APP APPGC

APP GCAPPAPP

APPAPPAPP

GC APPAPPAPP

GC APPAPPAPP

20

Our Subject: Metronome-2 System

• Parallel, Incremental, and Concurrent• No increment exceeds 450us• Real-time Scheduling• Smooth adaptation from under- to over-load• Implementation in production JVM

GC

APP GC

APP GC APP GC APP

APP

APP GC APPAPP APP APP

21

What Does “Real-time” Mean?

• Minimal, predictable interruption of application

• Collection finishes before heap is exhausted

• “Real space” - bounded, predictable memory

• Honor thread priorities

22

The Cycle of Life

AllocateAllocate

FreeFree

MutateMutate

• Not really a “garbage collector”…• … but a memory management subsystem

23

Metronome Memory Organization

• Page-based• Segregated free lists• Ratio bounds internal & page-internal fragmentation

24

Large Objects: Arraylets

• (Almost) eliminates external fragmentation• (Almost) eliminates need for compaction• Very large arrays still need contiguous pages• Extra indirection for array access

25

Handling the Concurrency

?

rr

pp TTa

b

XXa

b UUa

b

ZZa

b WWa

b

YYa

b

YYa

b

XXa

b

GCThreads

Application(“mutator”)Threads

• Mark• Collect• Format

• Load pointer• Store pointer• Allocate

26

Yuasa Snapshot Algorithm (1990)• Logically

– Take a “copy-on-write” heap snapshot– Collect the garbage in that snapshot

• Physically– Stop all threads– Copy their stacks (“roots”)– Force them to save over-written pointers– Trace roots and over-written pointers

* Dijktstra’75, Steele’76

27

1: Take Logical Snapshot

Stack

rr

pp TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

28

2(a): Copy Over-written Pointers

Stack

pp

rr

TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

29

2(b): Trace

Stack

pp TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

WWa

b

ZZa

b

YYa

b

XXa

b

* Color is per-object mark bit

30

2(c): Allocate “Black”

Stack

pp TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

ss VVa

b

WWa

b

ZZa

b

YYa

b

XXa

b

31

3(a): Sweep Garbage

Stack

pp TTa

b

XXa

b UUa

b

ZZa

b

WWa

b

YYa

b

WWa

b

ZZa

b

YYa

b

XXa

b

ss VVa

b

freefree

32

3(b): Allocate “White”

Stack

pp TTa

b

XXa

b UUa

b

ZZa

b

YYa

b

WWa

b

ZZa

b

YYa

b

XXa

b

ss VVa

b

freefree

V2V2a

b

tt

33

4: Clear Marks

Stack

pp TTa

b

XXa

b UUa

b

ZZa

b

YYa

b

ZZa

b

YYa

b

XXa

b

ss VVa

b

freefree

V2V2a

b

tt

VVa

b

WWa

b

WWa

b

34

Yuasa Algorithm Phases• Snapshot stack and global roots

• Trace

• Flip

• Sweep

• Flip

• Clear Marks

* Synchronous

35

Metronome-2 Concurrency

GCWorkerThreads

GCMasterThread

ApplicationThreads

(may do GC work)

36

Metronome-2 Phases• Initiation

– Setup• turn double barrier on

• Root Scan– Active Finalizer scan– Class scan– Thread scan**

• switch to single barrier, color to black– Debugger, JNI, Class Loader scan

• Trace – Trace*– Trace Terminate***

• Re-materialization 1– Weak/Soft/Phantom Reference List Transfer– Weak Reference clearing** (snapshot)

• Re-Trace 1– Trace Master– (Trace*)– (Trace Terminate***)

• Re-materialization 2– Finalizable Processing

• Clearing– Monitor Table clearing– JNI Weak Global clearing– Debugger Reference clearing– JVMTI Table clearing– Phantom Reference clearing

• Re-Trace 2– Trace Master– (Trace*)– (Trace Terminate***)– Class Unloading

• Completion– Finalizer Wakeup– Class Unloading Flush– Clearable Compaction**– Book-keeping

* Parallel** Callback*** Single actor symmetric

• Flip– Move Available Lists to Full List* (contention)

• turn write barrier off– Flush Per-thread Allocation Pages**

• switch allocation color to white• switch to temp full list

• Sweeping– Sweep*– Switch to regular Full List**– Move Temp Full List to regular Full List* (contention)

37

http://www.research.ibm.com/metronome

https://sourceforge.net/projects/tuningforkvp