Post on 29-Jan-2016
MotivationMotivation
Parallel programming is difficult
Culprit: Non-determinism• Interleaving of parallel threads• But required to harness parallelism
Sequential programs produce deterministic results
To make parallel programming easy• We want determinism• Same input => semantically same output.
Determinism EffortsDeterminism Efforts Language design [DPJ, X10,Yada]
• Deterministic by design: Types and annotations• Constrains programming• May need non-determinism for better performance
Deterministic runtime [DMP, Kendo]
Race detection • Absence of data races does not imply determinism• Data races could be benign and could help in improving
performance
Determinism Checker [SingleTrack, Torrellas et al.]• Dynamic determinism checker.
Determinism with respect to abstraction [Rinard et al.]
Our GoalOur Goal
Provide a framework that can express relevant determinism directly and easily• Separate from functional correctness
specification• Allow data races and healthy non-
determinism• Language independent• Useful for QA
Determinism specification: A sweet spot?
• Lightweight, but precise.
Determinism specification: A sweet spot?
• Lightweight, but precise.
Our GoalOur Goal How to specify correctness of parallelism?
Implicit:
No sources ofnon-determinism(no data races)
Implicit:
No sources ofnon-determinism(no data races)
Explicit:
Full functional correctness.
Explicit:
Full functional correctness.
OutlineOutline
Motivation
Deterministic Specification [FSE’09]
Checking: Active Testing
Experimental Evaluation
Future Work + Conclusions
Deterministic SpecificationDeterministic Specification
Goal: Specify deterministic behavior.• Same initial parameters => same image.• Non-determinism is internal.
// Parallel fractal render mandelbrot(params, img); // Parallel fractal render mandelbrot(params, img);
Deterministic SpecificationDeterministic Specification
Program: , Initial State: , Schedules:
Specifies: Two runs from same initial program state have same result state for any pair of schedules
deterministic { // Parallel fractal render mandelbrot(params, img);}
deterministic { // Parallel fractal render mandelbrot(params, img);}
111'
0100 ))'()(('.,, sssssss
P
s0
, '
double A[][], b[], x[];...deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x);}
double A[][], b[], x[];...deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x);}
Deterministic SpecificationDeterministic Specification
Too restrictive – different schedules may give slightly different floating-point results.
set t = new RedBlackTreeSet();deterministic { t.add(3) || t.add(5);}
set t = new RedBlackTreeSet();deterministic { t.add(3) || t.add(5);}
Deterministic SpecificationDeterministic Specification
Too restrictive – internal structure of set may differ depending on order of adds.
deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data);}
deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data);}
Deterministic SpecificationDeterministic Specification
Too restrictive – search can correctly return any tree with optimal cost.
Semantic DeterminismSemantic Determinism
Too strict to require every interleaving to give exact same program state:
deterministic { P }
deterministic { P }
111'
0100 ))'()(('.,, sssssss
Semantic DeterminismSemantic Determinism
Too strict to require every interleaving to give exact same program state:
deterministic { P }
deterministic { P } Predicate!
Should beuser-defined.
111'
0100 ))'()(('.,, sssssss
Semantic DeterminismSemantic Determinism
Too strict to require every interleaving to give exact same program state:
Specifies: Final states are equivalent.
deterministic { P } assert Post(s1,s1’)
deterministic { P } assert Post(s1,s1’)
)',())'()(('.,, 111'
0100 ssPostsssss
double A[][], b[], x[];...deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x);} assert (|x – x’| < ε)
double A[][], b[], x[];...deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x);} assert (|x – x’| < ε)
Semantic DeterminismSemantic Determinism
“Bridge” predicate
Resulting sets are semantically equal.
set t = new RedBlackTreeSet();deterministic { t.add(3) || t.add(5);} assert (t.equals(t’))
set t = new RedBlackTreeSet();deterministic { t.add(3) || t.add(5);} assert (t.equals(t’))
Semantic DeterminismSemantic Determinism
deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data);} assert (t.cost == t’.cost())
deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data);} assert (t.cost == t’.cost())
Semantic DeterminismSemantic Determinism
Too strict – initial states must be identical• Not compositional.
Preconditions for DeterminismPreconditions for Determinism set t = … deterministic { t.add(3) || t.add(5); } assert (t.equals(t’)) … deterministic { t.add(4) || t.add(6); } assert (t.equals(t’))
set t = … deterministic { t.add(3) || t.add(5); } assert (t.equals(t’)) … deterministic { t.add(4) || t.add(6); } assert (t.equals(t’))
Preconditions for DeterminismPreconditions for Determinism Too strict to require identical initial states:
deterministic { P} assert Post(s1,s1’)
deterministic { P} assert Post(s1,s1’)
)',())'()(('.,, 111'
0100 ssPostsssss
Preconditions for DeterminismPreconditions for Determinism Too strict to require identical initial states:
deterministic assume (s0 = s0’) { P} assert Post(s1,s1’)
deterministic assume (s0 = s0’) { P} assert Post(s1,s1’)
Post(s1, s1)
))'()''()(('.,,', 001'
01000 ssssssss
Preconditions for DeterminismPreconditions for Determinism Too strict to require identical initial states:
deterministic assume (s0 = s0’) { P} assert Post(s1,s1’)
deterministic assume (s0 = s0’) { P} assert Post(s1,s1’)
Predicate! Should beuser-defined.
Post(s1, s1)
))'()''()(('.,,', 001'
01000 ssssssss
Preconditions for DeterminismPreconditions for Determinism Too strict to require identical initial states:
Specifies:
deterministic assume Pre(s0,s0’) { P} assert Post(s1,s1’)
deterministic assume Pre(s0,s0’) { P} assert Post(s1,s1’)
Post(s1, s1)
))',(Pr)''()(('.,,', 001'
01000 ssessssss
deterministic assume Pre(s0,s0’) { P} assert Post(s1,s1’)
deterministic assume Pre(s0,s0’) { P} assert Post(s1,s1’)
Bridge predicates/assertionsBridge predicates/assertions
“Bridge”predicate
“Bridge”assertion
set t = ...deterministic assume (t.equals(t’) { t.add(4) || t.add(6);} assert (t.equals(t’))
set t = ...deterministic assume (t.equals(t’) { t.add(4) || t.add(6);} assert (t.equals(t’))
Specifies: Semantically equal sets yield semantically equal sets.
Preconditions for DeterminismPreconditions for Determinism
AdvantageAdvantage
Separately check parallelism and functional correctness.• Show parallelism is outwardly deterministic.• Reason about correctness sequentially.• Decompose correctness proof!
Example:• Write Cilk program and prove (or test)
sequential correctness.• Add parallelism, answers should not change
Determinism vs. AtomicityDeterminism vs. Atomicity Internal vs. external parallelism/non-determinism
• Complementary notions
AtomicDeterministic
“Closed program” “Open program”
Other TestingOther TestingUse sequential program as spec
deterministic Pre(s0,s0’) {
if (*) {
SequentialPgm;
} else {
ParallelPgm;
}
} assert Post(s,s’);
Regression testing
deterministic Pre(s0,s0’) {
if (*) {
PgmVersion1;
} else {
PgmVersion2;
}
} assert Post(s,s’);
OutlineOutline
Motivation
Deterministic Specification [FSE’09]
Checking: Active Testing [PLDI 08,09, FSE 08, CAV 09]
Experimental Evaluation
Future Work + Conclusions
Checking: Active TestingChecking: Active Testing
Predicting and Exploring
“interesting” Schedules
Active Testing:Active Testing:Predict and Test Potential BugsPredict and Test Potential Bugs
Predict potential bugs:• Data races: Eraser or lockset based• Atomicity violations: cycle in transactions
and happens-before relation• Deadlocks: cycle in resource acquisition
graph• Memory model bugs: cycle in happens-
before relation
Test schedules to create those bugs
[ASE 07, PLDI 08, FSE 08, PLDI 09, FSE 09, CAV 09]
Active Testing Cartoon: Phase IActive Testing Cartoon: Phase I
31
Potential Collision
1
2
1
2
3
Active Testing Cartoon: Phase IIActive Testing Cartoon: Phase II
32
1
2
1
2
3
OutlineOutline
Motivation
Deterministic Specification
Checking: Active Testing
Experimental Evaluation
Future Work + Conclusions
Ease of Asserting DeterminismEase of Asserting Determinism
Implemented a deterministic assertion library for Java.
Manually added deterministic assertions to 13 Java benchmarks with 200 – 4k LoC
Typically 5-10 minutes per benchmark• Functional correctness very difficult.
Ease of Use: ExampleEase of Use: Example
Deterministic.open();Predicate eq = new Equals();Deterministic.assume(width, eq);… (9 parameters total) …Deterministic.assume(gamma, eq);
// Compute fractal in threadsint matrix[][] = …;
Deterministic.assert(matrix, eq);Deterministic.close();
Effectiveness in Finding BugsEffectiveness in Finding Bugs
13 Java benchmarks of 200 – 4k LoC
Ran benchmarks on ~100 schedules• Schedules with data races and other
“interesting” interleavings (active testing)
For every pair of executions ofdeterministic Pre { P } Post:
check that:
Pre(s0, s0) Post(s1, s1))''()( 1
'010 ssss
Experiments: Java Grande ForumExperiments: Java Grande Forum
Benchmark LoCData Races
Found | Errors
High-Level Races
Found | Errors
sor 300 2 0 0 0
moldyn 1.3k 2 0 0 0
lufact 1.5k 1 0 0 0
raytracer 1.9k 3 1 0 0
montecarlo 3.6k 1 0 2 0
Experiments: Parallel Java LibExperiments: Parallel Java Lib
Benchmark LoCData Races
Found | Errors
High-Level Races
Found | Errors
pi 150 9 0 1+ 1
keysearch3 200 3 0 0+ 0
mandelbrot 250 9 0 0+ 0
phylogeny 4.4k 4 0 0+ 0
tsp* 700 6 0 2 0
Experimental EvaluationExperimental Evaluation
Across 13 benchmarks:
Found 40 data races.• 1 violates deterministic assertions.
Experimental EvaluationExperimental Evaluation
Across 13 benchmarks:
Found 40 data races.• 1 violates deterministic assertions.
Found many interesting interleavings(non-atomic methods, lock races, etc.)• 1 violates deterministic assertions.
Determinism ViolationDeterminism Violation
Pair of calls to nextDouble() mustbe atomic.
deterministic { // N trials in parallel. foreach (n = 0; n < N; n++) { x = Random.nextDouble(); y = Random.nextDouble(); … }} assert (|pi - pi’| < 1e-10)
deterministic { // N trials in parallel. foreach (n = 0; n < N; n++) { x = Random.nextDouble(); y = Random.nextDouble(); … }} assert (|pi - pi’| < 1e-10)
OutlineOutline
Motivation
Deterministic Specification
Checking: Active Testing
Experimental Evaluation
Future Work + Conclusions
How to verify: Essential RuleHow to verify: Essential Rule
deterministic assume(Φ1)
A11;A12;A13|| A21;A22;A23|| A31;A32;A33
} assert (Φ2)
How to verify: Essential RuleHow to verify: Essential Rule
deterministic assume(Φ1)
A11;A12;A13|| A21;A22;A23|| A31;A32;A33
} assert (Φ2)
Find an invariant Φ and prove that for all i, j, l, m
deterministic assume(Φ)
Aij|| Alm
} assert (Φ)
and Φ1 → Φ and Φ → Φ2
Prove for the followingProve for the followingdeterministic assume(true) {
parallel while (!wq.is_empty()) {
[work = wq.get();]
if (bound(work) >= best)
continue;
if (size(work) <= threshold) {
s = find_best_solution(work);
[best = min(best,cost(s));]
} else {
[wq.add_all(split(work));]
}
}
} assert(best=best’)
Decompose the proofDecompose the proof
Let us prove that P ≈ S
That is prove that
deterministic assume(Pre(S0,S’0)) {
if (*) P else S
} assert (Post(S,S’))
Decompose the proofDecompose the proof
Let us prove that P ≈ S
Create a non deterministic sequential program NS [Galois]
P ≡ NS ≈ S
Decompose the proofDecompose the proof
Let us prove that P ≈ S
Create a non deterministic sequential program NS
P ≡ NS ≈ S
deterministic assume(S0=S’0) {
if (*) P else NS
} assert (S=S’)
deterministic assume(Pre(S0,S’0)) {
if (*) NS else S
} assert (Post(S,S’))
How to verify?How to verify?while (!wq.is_empty()) {
[work = wq.nondet_get();]
if (bound(work) >= best)
continue;
if (size(work) <= threshold) {
s = find_best_solution(work);
[best = min(best,cost(s));]
} else {
[wq.add_all(split(work));]
}
}
deterministic assume(true) {
parallel while (!wq.is_empty()) {
[work = wq.get();]
if (bound(work) >= best)
continue;
if (size(work) <= threshold) {
s = find_best_solution(work);
[best = min(best,cost(s));]
} else {
[wq.add_all(split(work));]
}
}
} assert(best=best’)
Not deterministic sequential program
How to verify?How to verify?while (!wq.is_empty()) {
[work = wq.nondet_get();]
if (bound(work) >= best)
continue;
if (size(work) <= threshold) {
s = find_best_solution(work);
[best = min(best,cost(s));]
} else {
[wq.add_all(split(work));]
}
}
Barrier()
deterministic assume(true) {
parallel while (!wq.is_empty()) {
[work = wq.get();]
if (bound(work) >= best)
continue;
if (size(work) <= threshold) {
s = find_best_solution(work);
[best = min(best,cost(s));]
} else {
[wq.add_all(split(work));]
}
}
} assert(best=best’)
Not deterministic sequential program
How to verify?How to verify?while (!wq.is_empty()) {
[work = wq.nondet_get();]
if (bound(work) >= best && non_det)
continue;
if (size(work) <= threshold) {
s = find_best_solution(work);
[best = min(best,cost(s));]
} else {
[wq.add_all(split(work));]
}
}
Barrier()
deterministic assume(true) {
parallel while (!wq.is_empty()) {
[work = wq.get();]
if (bound(work) >= best)
continue;
if (size(work) <= threshold) {
s = find_best_solution(work);
[best = min(best,cost(s));]
} else {
[wq.add_all(split(work));]
}
}
} assert(best=best’)
Not deterministic sequential program
Verifying DeterminismVerifying Determinism
Verify determinismof each piece.
No need to considercross product of allinterleavings.
P
P
P
P
P
P
Verifying DeterminismVerifying Determinism
Compositional reasoning for determinism?
P
Q
Q
Q
Q
Q
Q
P P
SummarySummary
“Bridge” predicates and assertions• Simple to assert natural determinism• Semantic determinism
Active Testing• Direct search based on imprecise bug reports
Verify/prove determinism• Prove exact equivalence with non deterministic
sequential programs• Prove semantic equivalence between non
deterministic sequential programs and sequential programs
Deterministic Assertion LibraryDeterministic Assertion Library Implemented assertion library for Java:
Records set to check: eq.apply(set0,set0’) => eq.apply(set,set’)
Predicate eq = new Equals();Deterministic.open();Deterministic.assume(set, eq); ...Deterministic.assert(set, eq);Deterministic.close();
Predicate eq = new Equals();Deterministic.open();Deterministic.assume(set, eq); ...Deterministic.assert(set, eq);Deterministic.close();
Checking DeterminismChecking Determinism
Run P on some number of schedules.
For every pair and ofexecutions of P:
deterministic assume Pre(s0,s0’) { P} assert Post(s1,s1’)
deterministic assume Pre(s0,s0’) { P} assert Post(s1,s1’)
s0 s1
s0 s1
Pre(s0,s0) Post(s1,s1)
OutlineOutline
Motivation (3 min)
Deterministic Specification (6 min)
Experimental Evaluation (5 min)
Related Work (3 min)
Future Work + Conclusions (3 min)