Practical Concerns for Scalable Synchronization

Jonathan Walpole (PSU)Paul McKenney (IBM)

Tom Hart (University of Toronto)

2www.cs.pdx.edu/~walpoleJonathan WalpoleCS533 Winter 2006

The problem

“i++” is dangerous if “i” is global

CPU 0load r1,i

inc r1store r1,i

The problem

load r1,iload r1,i

inc r1store r1,i

load r1,i

The problem

inc r1load r1,i

inc r1store r1,i

inc r1

The problem

store r1,iload r1,i

inc r1store r1,i

store r1,i

The solution – critical sections

Classic multiprocessor solution: spinlocks– CPU 1 waits for CPU 0 to release the lock

Counts are accurate, but locks have overhead!

spin_lock(&mylock);

spin_unlock(&mylock);

Critical-section efficiency

Lock Acquisition (Ta )

Critical Section (Tc )

Lock Release (Tr )

Critical-section efficiency =Tc

Tc+Ta+Tr

Ignoring lock contention and cache conflicts in the critical section

Critical section efficiency

Performance of normal instructions

Have synchronization instructions got faster?– Relative to normal instructions?– In absolute terms?

What are the implications of this for the performance of operating systems?

Can we fix this problem by adding more CPUs?

Questions

What’s going on?

Taller memory hierarchies– Memory speeds have not kept up with CPU

speeds– 1984: no caches needed, since instructions

were slower than memory accesses– 2005: 3-4 level cache hierarchies, since

instructions are orders of magnitude faster than memory accesses

Why does this matter?

Synchronization implies sharing data across CPUs– normal instructions tend to hit in top-level cache– synchronization operations tend to miss

Synchronization requires a consistent view of data– between cache and memory– across multiple CPUs– requires CPU-CPU communication

Synchronization instructions see memory latency!

… but that’s not all!

Longer pipelines– 1984: Many clock cycles per instruction– 2005: Many instructions per clock cycle

● 20-stage pipelines

Out of order execution– Keeps the pipelines full– Must not reorder the critical section before its

Synchronization instructions stall the pipeline!

Reordering means weak memory consistency

Memory barriers- Additional synchronization instructions are needed to manage reordering

What is the cost of all this?

Instruction Cost 1.45 GHz

3.06GHzIBM POWER4 Intel Xeon

Normal Instruction 1.0 1.0

Atomic increment

Normal Instruction

Atomic Increment

Memory barriers

Normal Instruction

Atomic Increment

SMP Write Memory Barrier

Read Memory Barrier

Write Memory Barrier

183.1328.6328.9400.9

402.30.0

Lock acquisition/release with LL/SC

Normal Instruction

Atomic Increment

Read Memory Barrier

Local Lock Round Trip

183.1328.6328.9400.91057.5

402.30.0

402.30

1138.8

Compare & swap unknown values (NBS)

Normal Instruction

Atomic Increment

Read Memory Barrier

CAS Cache Transfer & Invalidate

183.1328.6328.9400.91057.5247.1

402.30.0

402.30

1138.8847.1

Compare & swap known values (spinlocks)

Normal Instruction

Atomic Increment

Read Memory Barrier

CAS Cache Transfer & Invalidate

CAS Blind Cache Transfer

183.1328.6328.9400.91057.5247.1257.1

402.30.0

402.30

1138.8847.1993.9

The net result?

1984: Lock contention was the main issue

2005: Critical section efficiency is a key issue

Even if the lock is always free when you try to acquire it, performance can still suck!

How has this affected OS design?

Multiprocessor OS designers search for “scalable” synchronization strategies– reader-writer locking instead of global locking– data locking and partitioning– Per-CPU reader-writer locking– Non-blocking synchronization

The “common case” is read-mostly access to linked lists and hash-tables– asymmetric strategies favouring readers are good

Review - Global locking

A symmetric approach (also called “code locking”)– A critical section of code is guarded by a lock– Only one thread at a time can hold the lock

Examples include– Monitors– Java “synchronized” on global object– Linux spin_lock() on global spinlock_t

What is the problem with global locking?

Review - Global locking

A symmetric approach (also called “code locking”)– A critical section of code is guarded by a lock– Only one thread at a time can hold the lock

Examples include– Monitors– Java “synchronized” on global object– Linux spin_lock() on global spinlock_t

Global locking doesn’t scale due to lock contention!

Review - Reader-writer locking

Many readers can concurrently hold the lock

Writers exclude readers and other writers

The result?– No lock contention in read-mostly scenarios– So it should scale well, right?

Review - Reader-writer locking

Many readers can concurrently hold the lock

Writers exclude readers and other writers

The result?– No lock contention in read-mostly scenarios– So it should scale well, right?– … wrong!

Scalability of reader/writer locking

Reader/writer locking does not scale due to criticalsection efficiency!

Review - Data locking

A lock per data item instead of one per collection– Per-hash-bucket locking for hash tables– CPUs acquire locks for different hash chains in

parallel– CPUs incur memory-latency and pipeline-flush

overheads in parallel

Data locking improves scalability by executing critical section “overhead” in parallel

Review - Per-CPU reader-writer locking

One lock per CPU (called brlock in Linux)– Readers acquire their own CPU’s lock– Writers acquire all CPU’s locks

In read-only workloads CPUs never exchange locks– no memory latency is incurred

Per-CPU R/W locking improves scalability by removing memory latency from read-lock acquisition for read-mostly scenarios

Scalability comparison

Expected scalability on read-mostly workloads– Global locking – poor due to lock contention– R/W locking – poor due to critical section

efficiency– Data locking – better?– R/W data locking – better still?– Per-CPU R/W locking – the best we can do?

Actual scalability

Scalability of locking strategies using read-only workloads in a hash-table benchmark

Measurements taken on a 4-CPU 700 MHz P-III system

Similar results are obtained on more recent CPUs

Scalability on 1.45 GHz POWER4 CPUs

Performance at different update fractions on 8 1.45 GHz POWER4 CPUs

What are the lessons so far?

Avoid lock contention !

Avoid synchronization instructions !– … especially in the read-path !

What are the lessons so far?

How about non-blocking synchronization?

Basic idea – copy & flip pointer (no locks!)– Read a pointer to a data item – Create a private copy of the item to update in place– Swap the old item for the new one using an atomic

compare & swap (CAS) instruction on its pointer– CAS fails if current pointer not equal to initial value– Retry on failure

NBS should enable fast reads … in theory!

Problems with NBS in practice

Reusing memory causes problems– Readers holding references can be hijacked during

data structure traversals when memory is reclaimed

– Readers see inconsistent data structures when memory is reused

How and when should memory be reclaimed?

Immediate reclamation?

In practice, readers must either– Use LL/SC to test if pointers have changed, or– Verify that version numbers associated with data

structures have not changed (2 memory barriers)

Synchronization instructions slow NBS readers!

Reader-friendly solutions

Never reclaim memory ?

Type-stable memory ?– Needs free pool per data structure type– Readers can still be hijacked to the free pool– Exposes OS to denial of service attacks

Ideally, defer reclaiming memory until its safe!– Defer reclamation of a data item until references

to it are no longer held by any thread

Wait for a while then delete?– … but how long should you wait?

Maintain reference counts or per-CPU “hazard pointers” on data that is in use?

How should we defer reclamation?

Wait for a while then delete?– … but how long should you wait?

Maintain reference counts or per-CPU “hazard pointers” on data that is in use?– Requires synchronization in read path!

Challenge – deferring destruction without using synchronization instructions in the read path

Coding convention:– Don’t allow a quiescent state to occur in a read-

side critical section

Reclamation strategy:– Only reclaim data after all CPUs in the system

have passed through a quiescent state

Example quiescent states:– Context switch in non-preemptive kernel– Yield in preemptive kernel– Return from system call …

Quiescent-state-based reclamation

Coding conventions for readers

Delineate read-side critical section– rcu_read_lock() and rcu_read_unlock() primitives– may compile to nothing on most architectures

Don’t hold references outside critical sections– Re-traverse data structure to pick up reference

Don’t yield the CPU during critical sections– Don’t voluntarily yield– Don’t block, don’t leave the kernel …

Overview of the basic idea

Writers create (and publish) new versions– Using locking or NBS to synchronize with each other

– Register call-backs to destroy old versions when safe● call_rcu() primitive registers a call back with a reclaimer

– Call-backs are deferred and memory reclaimed in batches

Readers do not use synchronization– While they hold a reference to a version it will not be

destroyed

– Completion of read-side critical sections is “inferred” by the reclaimer from observation of quiescent states

Overview of RCU API

Writer Reader

Reclaimer

rcu_dereference ()

rcu_assign_pointer ()

rcu_read_lock ()

synchronize_rcu ()

call_rcu ()

Memory Consistency of Mutable PointersCollection of versions of Immutable Objects

Context switch as a quiescent state

CPU 1 RC

May hold referenceCan't hold reference to old version, but RCU can't tell

Can't hold reference to old version

Grace periods

CPU 1 RC

Grace Period

Example quiescent states– Context switch (non-preemptive kernels)– Voluntary context switch (preemptive kernels)– Kernel entry/exit– Blocking call

Grace periods– A period during which every CPU has gone

through a quiescent state

Quiescent states and grace periods

Efficient implementation

Choosing good quiescent states– They should occur anyway– They should be easy to count– Not too frequent or infrequent

Recording and dispatching call-backs– Minimize inter-CPU communication– Maintain per-CPU queues of call-backs– Two queues – waiting for grace period start and

'Next' RCUCallbacks

RCU's data structures

'Current' RCU

Callback

Grace-Period

Number

Global Grace-Period

Number

Global CPUBitmask

call_rcu()

Counter

CounterSnapshot

End of PreviousGrace Period (If

End of CurrentGrace Period

RCU implementations

DYNIX/ptx RCU (data center)

Linux– Multiple implementations (in 2.5 and 2.6 kernels)– Preemptible and nonpreemptible

Tornado/K42 “generations”– Preemptive kernel– Helped generalize usage

Experimental results

How do different combinations of RCU, SMR, NBS and Locking compare?

Hash table mini-benchmark running on 1.45 GHz POWER4 system with 8 CPUs

Various workloads– Read/update fraction– Hash table size– Memory constraints– Number of CPUs

Scalability with working set in cache

Scalability with large working set

Performance at different update fractions (8 CPUs)

Performance at different update fractions (2 CPUs)

Performance in read-mostly scenarios

Impact of memory constraints

Performance and complexity

When should RCU be used?– Instead of simple spinlock?– Instead of per-CPU reader-writer lock?

Under what environmental conditions?– Memory-latency ratio– Number of CPUs

Under what workloads?– Fraction of accesses that are updates– Number of updates per grace period

Analytic results

Compute breakeven update-fraction contours for RCU vs. locking performance, against:– Number of CPUs (n)– Updates per grace period ()– Memory-latency ratio (r)

Look at computed memory-latency ratio at extreme values of for n=4 CPUs

Breakevens for RCU worst case(f vs. r for Small )

Breakeven for RCU best case(f vs. r, Large )

Real-world performance and complexity

SysV IPC– >10x on microbenchmark (8 CPUs)– 5% for database benchmark (2 CPUs)– 151 net lines added to the kernel

Directory-Entry Cache– +20% in multiuser benchmark (16 CPUs)– +12% on SPECweb99 (8 CPUs)– -10% time required to build kernel (16 CPUs)– 126 net lines added to the kernel

Real-world performance and complexity

Task List– +10% in multiuser benchmark (16 CPUs)– 6 net lines added to the kernel

● 13 added● 7 deleted

Summary and Conclusions (1)

RCU can provide order-of-magnitude speedups for read-mostly data structures– RCU optimal when less than 10% of accesses are

updates over wide range of CPUs– RCU projected to remain useful in future CPU

architectures

In Linux 2.6 kernel, RCU provided excellent performance with little added complexity– Currently over 1000 uses of RCU API in Linux

kernel

Summary and Conclusions (2)

RCU introduces a new model and API for synchronization– There is additional complexity– Visual inspection of kernel code has uncovered

some subtle bugs in use of RCU API primitives– Tools to ensure correct use of API primitives are

needed

A thought

“Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it !”

– Brian Kernighan

UseUse

the right the right tooltool

for the for the job!!!job!!!

Practical Concerns for Scalable Synchronization

Documents

Transcript of Practical Concerns for Scalable Synchronization

Synchronization: Why care? Synchronization in Digital Logic Circuits

Algorithms for scalable synchronization on shared-memory ...johnmc/scalable_synch/tocs91.pdf · Algorithms for Scalable Synchronization on Shared — Memory Multi~rocessors o 23 be

Hierarchical Phasers for Scalable Synchronization and Reductions in Dynamic Parallelism IPDPS 2010 April 22 nd, 2010 Jun Shirako and Vivek Sarkar Rice.

NB-FEB: A Universal Scalable Easy-to-Use Synchronization Primitive for Manycore Architectures

PTN Synchronization Subject V1.0. Contents Introduction to Synchronization Principle Basic Concepts of Synchronization Difference between Time Synchronization.

Scalable Reader-Writer Synchronization for Shared- Memory Multiprocessors Mellor-Crummey and Scott Presented by Robert T. Bauer.

Concurrency: Mutual Exclusion and Synchronization - Chapter 5 (Part 1) Concurrency –Basic Requirement and Difficulties Operating System Concerns Degree.

John M. Mellor-Crummey Algorithms for Scalable Synchronization on Shared- Memory Multiprocessors Joseph Garvey & Joshua San Miguel Michael L. Scott.

Separating Synchronization Concerns with Frameworks and ... · Thread synchronization in object-oriented systems is difﬁcult to implement, in part, because traditional synchronization

Foundations: Concurrency Concerns Synchronization Basics

Synthetic Data Generation for Realistic Analytics Examples ...€¦ · Synthetic Data • No licensing, privacy, or intellectual property concerns • Scalable: Laptops to Clusters!

Spatial and temporal synchronization of water and energy ... · Spatial and temporal synchronization of water and energy systems: ... India and the Mid-3 dle East, concerns about

Veriﬁable and Resource -Aware Component-based Device Model ...perso.ens-lyon.fr/olivier.laurent/llipido/181218_yb.pdf · §Modular §Scalable = SOA answers someof the concerns of

Advanced Locking Techniques Coarse-grained Synchronization Fine-grained Synchronization Optimistic Synchronization Lazy Synchronization.

Synchronization strategy for mobile network · PDF file–IP-RAN NodeBs that require more bandwidth and Ethernet interfaces ... • WCDMA TDD systems ... PTP Domains Dimensioning concerns

Kinetix Servo Drive Family - Electrical Supply & Industrial … Servo Drive family... · 2016-09-01 · synchronization of discrete and motion control. Kinetix Servo Drives • Scalable:

SALSA: Scalable and Low-synchronization NUMA-aware Algorithm for Producer-Consumer Pools

Insertion Tree Phasers: Efficient and Scalable Barrier Synchronization for Fine-grained Parallelism

Synchronization made easy! February 2009 Preferred Synchronization methods.

Algorithms for scalable synchronization on shared-memory multiprocessorscourses.csail.mit.edu/6.852/08/papers/p21-mellor-crummey.pdf · 22 . J, M Mellor-Crummey and M. L Scott Categories