CAPP: Change-Aware Preemption Prioritization
description
Transcript of CAPP: Change-Aware Preemption Prioritization
CAPP: Change-Aware Preemption Prioritization
Vilas Jagannath, Qingzhou Luo, Darko Marinov
Sep 6th 2011
Motivation• Parallel code required for performance
– Dominant paradigm is shared-memory, multithreaded code
• Difficult to develop (correct) multithreaded code– Different behavior under different schedules– Afflicted by bugs like data races, deadlocks, atomicity violations…
• Difficult to test multithreaded code– Exploration required, time consuming– Current exploration techniques focus on one code version– Code evolves, need efficient regression testing
• CAPP: Change-Aware Preemption Prioritization2
@Testpublic void testFilterWriteThreadSafety() throws InterruptedException { … FilterWriteThread fwThread1 = new FilterWriteThread(); FilterWriteThread fwThread2 = new FilterWriteThread(); fwThread1.start(); fwThread2.start(); fwThread1.join(); fwThread2.join(); assertEquals(0, fwThread1.result); assertEquals(0, fwThread2.result);}class FilterWriteThread extends Thread { int result = 0; ProtocolCodecFilter pc; … public void run() { try { pc.filterWrite(nextFilter, session, writeRequest); … } catch (Exception e) { e.printStackTrace(); result = 1;} } }
Apache Mina Test for filterWrite()
3
Exploration
4
S0
S1
T1
S2
T1
S3
T1
S..
T2
S..
T1
S..
T2
Schedule
Preemption
S..
T2
Transition
(S0, T1) (S0, T2)
Multithreaded Testing
5
Code
Test
Test execution explores many schedules
Work Pool
(S1, T2)
Exploration Order
6
(S0, T1) (S0, T2)
S1
T1
(S1, T1)
S2
T1
S0
Eg.DFS: StackBFS: Queue…
Current Work Pool
Exploration Prioritization
7
S1
T1
S2
T1
S0
Next Work Pool
(S0, T1) (S0, T2)
(S1, T2)(S1, T1) S..
T2
S..
T2
(S.., T2)
(S.., T1)….. …..
Eg.CHESSGambitActive TestingConTestAVIOCTrigger…
Regression Multithreaded Testing
8
Code
Test
Code`
Change-Unaware Prioritization
9
Code
Test
Code`
Change-Unaware Prioritization
10
Code
Test
Code`
CAPP:Change-Aware Preemption Prioritization
11
Code
Test
Code`
Use evolution information to prioritize exploration of schedules that perform preemptions at change impacted code.
CAPP is a general framework. We instantiate with 14 heuristics.
Apache Mina r912148 to r912149 public void filterWrite(NextFilter nextFilter, IoSession session, WriteRequest writeRequest) throws Exception { … flushWithoutFuture(); …}public void flushWithoutFuture() { Queue<Object> bufferQueue = getMessageQueue(); for (;;) { Object encodedMessage = bufferQueue.poll(); encodedMessage == null) { break; }if (// Flush only when the buffer has remaining. if (!(encodedMessage instanceof IoBuffer) || ((IoBuffer) encodedMessage).hasRemaining()) { …} } }
12
public void filterWrite(NextFilter nextFilter, IoSession session, WriteRequest writeRequest) throws Exception { … Queue<Object> bufferQueue = getMessageQueue(); while (!bufferQueue.isEmpty()) { Object encodedMessage = bufferQueue.poll(); // Flush only when the buffer has remaining. if (!(encodedMessage instanceof IoBuffer) || ((IoBuffer) encodedMessage).hasRemaining()) { … } } …
• DIRMINA-803• 58 schedules to detect failure using CHESS• 5 schedules using one of the CAPP heuristics
CAPP Parts
Impacted Code Points (ICP) Collection(Static)
Change-Aware Exploration Prioritization(Dynamic)
13
Impacted Code Points (ICP) Collection
14
Code Code`
• Inspect structure diff to collect impacted:– Classes, Methods, Lines, Fields
• Focus on changes to thread communication:– Expand synchronized blocks, methods– Fields accessed by changed lines– Fields accessed in methods called by changed lines
Current Work Pool
Change-Aware Exploration Prioritization
15
S1
T2
S2
T1
S0
Next Work Pool
(S0, T1) (S0, T2)
(S1, T2)(S1, T1)
Use ICPs to partition
Use ICPs to partition
Family of Heuristics• Defined by ICP Match Mode and Prioritization Mode
• ICP Match Mode: Is transition impacted by changes?– Class (on Stack), Method (on Stack), Line (on Stack), Field
• Prioritization Mode: Which work pool to add transitions to?– All, Some
• 14 heuristics– CA (Class All)– …– MOS (Method on Stack Some)…
– …
16
ICP Match Mode• ICP Match Mode: Is transition impacted by changes?
– Class (on Stack), Method (on Stack), Line (on Stack), Field
17
S..
ICPs: {………, class A,………….....}
(S.., T1) (S.., T2)
T1: Executing class A, method m, line 55 T2: Executing class B, method k, line 32 Class
(S.., T1) (S.., T2)(S.., T1) (S.., T2)
Prioritization Mode
18
Current Work Pool
S..
Next Work Pool
(S.., T1)
• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted
(S.., T2) (S.., T3)
Prioritization Mode
19
Current Work Pool
S..
Next Work Pool
• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted
All (S.., T1) (S.., T2) (S.., T3)
Prioritization Mode
20
Current Work Pool
S..
Next Work Pool
• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted
All (S.., T1) (S.., T2) (S.., T3)
Prioritization Mode
21
Current Work Pool
S..
Next Work Pool
• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted
Some (S.., T1) (S.., T2) (S.., T3)
Prioritization Mode
22
Current Work Pool
S..
Next Work Pool
• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted
Some (S.., T1) (S.., T2) (S.., T3)
T1
Implementations• Two implementations• Java PathFinder (JPF)
– Stateful exploration using checkpoints– Depth first, unbounded– CAPP implemented using a custom SimplePrioritySearch
• ReEx– Stateless exploration using re-execution– CHESS (iterative context bound 2)– CAPP implemented using a custom SchedulingStrategy
23
Evaluation Subjects
• 15 multithreaded programs:– Ranging from 52-54,000 loc, with 2-607 classes, running 2-9 threads– Evolution from non-buggy version to buggy version
• 7 deadlock faults, 8 data race and atomicity violation faults24
Subject SLOC Threads ICPsAirline 136 6 6Allocation 209 3 5BoundedBuffer 110 9 2BubbleSort 89 4 4Deadlock 52 3 3ReadersWriters 154 5 2ReplWorkers 432 3 2RAXextended 166 6 2Groovy 54,872 3 60Lang 48,369 3 3Mina 34,804 3 36Pool1 10,042 3 148Pool2 4,473 3 201Pool3 10,802 2 29Pool4 10,783 3 47
Experiments• Independent variables:
– 14 heuristics
– Stateful vs. Stateless exploration
– Default exploration order vs. 50 Random exploration orders
• 19,890 explorations, around a month of computing time
• Measured savings in exploration cost compared to change-unaware exploration:– Transitions for JPF implementation
– Schedules for ReEx implementation
25
Default Non-Randomized Exploration
26
CA CS COA COS MA MS MOAMOS LA LS LOA LOS FA FS0
1
2
3
4
5
6StatefullStateless
Heuristics
Ave
rage
Exp
lora
tion
Red
uctio
n
5.3x
2.7x0.8x
CAPP• Current multithreaded testing techniques focus on one code
version
• Code evolves, need techniques that leverage evolution info
• CAPP uses evolution info to prioritize exploration:– Up to 5.3x average reduction in exploration cost
• Ongoing work:– Regression Schedule Selection [ICST 10, STVR in
revision]– Schedule Expression [ESEC/FSE 11]– Multithreaded Tests Generation
27
Possible Project Ideas
28
• Better impact analysis
• Combine selection and prioritization (RMC)
• Prioritize across tests
• Multiple faults
• Improve ReEx, revisit stateless randomized search (PRSS)• Combine IMUnit with CAPP
– How to find/generate/simplify buggy IMUnit schedules?– How to decide whether some IMUnit schedules is valid or not
after code evolves (a.k.a., identify bad concurrency bug fixes)?
UPCRC
29
• Universal Parallel Computing Research Center– Funded by Intel and Microsoft– Goal: improve parallelism for client computation– Two centers: Berkeley and Illinois
• UPCRC Illinois– About 20 faculty + 40 grad students– Research on entire computation stack: architecture, runtime
systems, compilers, programming languages, design patterns, applications… and testing
Results Summary• Does CAPP help?
– Up to 5.3x reduction on average– Only one heuristic (out of 28) increased cost (0.8x)
• How do heuristics compare?– Field ICP Match Mode heuristics perform best– ALL Prioritization Mode heuristics perform best
• Stateful vs. Stateless?– Greater benefit for stateless exploration
• Default vs. Random?– Conclusions from random results similar to default results– Except ICP Match Mode
• Detailed results in the paper30
Multicore World
31
Performance!
Shared Memory
Multithreaded
Difficult to Develop Multithreaded Code
• Scheduling non-determinism• Data races• Deadlocks• Atomicity violations• …
32
Correct
Shared Memory
Multithreaded
Difficult to Test Multithreaded Code• Exploration required• Time consuming• Current techniques focus
on one code version• Code evolves• Need efficient regression
testing
33
Code
Test