SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a...
Transcript of SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a...
![Page 1: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/1.jpg)
SYNPLEXA task-parallel scheme for the revised simplex method
(Part 2)
Julian Hall
School of Mathematics
University of Edinburgh
June 23rd 2005
SYNPLEX, a task-parallel scheme for the revised simplex method
![Page 2: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/2.jpg)
SYNPLEX
• Synchronous variant of PARSMI
![Page 3: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/3.jpg)
SYNPLEX
• Synchronous variant of PARSMI
• INVERT not overlapped with basis changes ⇒ numerical stability
![Page 4: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/4.jpg)
SYNPLEX
• Synchronous variant of PARSMI
• INVERT not overlapped with basis changes ⇒ numerical stability
• CHUZC uses up-to-date reduced costs ⇒ better candidate persistence
![Page 5: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/5.jpg)
SYNPLEX
• Synchronous variant of PARSMI
• INVERT not overlapped with basis changes ⇒ numerical stability
• CHUZC uses up-to-date reduced costs ⇒ better candidate persistence
• Target platform: shared memory Sun Fire E15k (OpenMP)
![Page 6: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/6.jpg)
SYNPLEX
• Synchronous variant of PARSMI
• INVERT not overlapped with basis changes ⇒ numerical stability
• CHUZC uses up-to-date reduced costs ⇒ better candidate persistence
• Target platform: shared memory Sun Fire E15k (OpenMP)
P P P P F InvP P P
F Inv
PB InvB InvF InvF Inv3
F Inv
F Inv F InvPPPPPPPPB InvB InvF InvF Inv4
F Inv
F Inv
PPPPPPPB InvB InvF InvF Inv1
INVERTF Inv0
P
PPPPPPPPB InvB InvF InvF Inv2
F InvF Inv
SYNPLEX, a task-parallel scheme for the revised simplex method 1
![Page 7: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/7.jpg)
Data parallelism
When using 1 + p processors
• Rows distributed over p processors
![Page 8: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/8.jpg)
Data parallelism
When using 1 + p processors
• Rows distributed over p processors for data parallel
◦ FTRAN for UPDATE etas
◦ CHUZR
◦ UPDATE tableau in minor iterations
◦ UPDATE RHS
![Page 9: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/9.jpg)
Data parallelism
When using 1 + p processors
• Rows distributed over p processors for data parallel
◦ FTRAN for UPDATE etas
◦ CHUZR
◦ UPDATE tableau in minor iterations
◦ UPDATE RHS
• Columns distributed over p processors
![Page 10: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/10.jpg)
Data parallelism
When using 1 + p processors
• Rows distributed over p processors for data parallel
◦ FTRAN for UPDATE etas
◦ CHUZR
◦ UPDATE tableau in minor iterations
◦ UPDATE RHS
• Columns distributed over p processors for data parallel
◦ PRICE
◦ CHUZC
SYNPLEX, a task-parallel scheme for the revised simplex method 2
![Page 11: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/11.jpg)
Data location challenge
• B0 factored serially on one processor
![Page 12: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/12.jpg)
Data location challenge
• B0 factored serially on one processor
• Factors used serially on all processors to solve linear systems
![Page 13: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/13.jpg)
Data location challenge
• B0 factored serially on one processor
• Factors used serially on all processors to solve linear systems
• Each solution used for data parallel operations over all processors
SYNPLEX, a task-parallel scheme for the revised simplex method 3
![Page 14: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/14.jpg)
Practical implementation
• Use a task manager processor
![Page 15: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/15.jpg)
Practical implementation
• Use a task manager processor
◦ Task parallel operations allocated according to processor activity
◦ Enables different computational components to overlap
![Page 16: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/16.jpg)
Practical implementation
• Use a task manager processor
◦ Task parallel operations allocated according to processor activity
◦ Enables different computational components to overlap
• Prevent different processors from writing to consecutive components
![Page 17: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/17.jpg)
Practical implementation
• Use a task manager processor
◦ Task parallel operations allocated according to processor activity
◦ Enables different computational components to overlap
• Prevent different processors from writing to consecutive components
◦ Insert “padding” between row partitions: implemented
![Page 18: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/18.jpg)
Practical implementation
• Use a task manager processor
◦ Task parallel operations allocated according to processor activity
◦ Enables different computational components to overlap
• Prevent different processors from writing to consecutive components
◦ Insert “padding” between row partitions: implemented
◦ Insert “padding” between column partitions: not yet implemented
SYNPLEX, a task-parallel scheme for the revised simplex method 4
![Page 19: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/19.jpg)
Results
1 Processor Speed-up
Model Rows Columns CPU (s) 4 processors 8 processors
cre-a 3517 4067 5.76 1.16 1.83
25fv47 822 1571 8.78 1.54 1.99
greenbea 2393 5405 29.22 - 2.30
ken-11 14695 21349 41.26 1.40 2.52
stocfor3 16676 15695 98.44 1.50 2.76
pds-06 9882 28655 138.84 1.58 3.05
SYNPLEX, a task-parallel scheme for the revised simplex method 5
![Page 20: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/20.jpg)
Poor performance: cre-a
Least speed-up (1.83) on 8 processors
0
1
2
3
4
5
6
7
3.84800 3.85000 3.85200 3.85400 3.85600 3.85800
![Page 21: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/21.jpg)
Poor performance: cre-a
Least speed-up (1.83) on 8 processors
0
1
2
3
4
5
6
7
3.84800 3.85000 3.85200 3.85400 3.85600 3.85800
Operation Slow-down in total time Overall speed-up
Inv-FTRAN 3.29 -
![Page 22: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/22.jpg)
Poor performance: cre-a
Least speed-up (1.83) on 8 processors
0
1
2
3
4
5
6
7
3.84800 3.85000 3.85200 3.85400 3.85600 3.85800
Operation Slow-down in total time Overall speed-up
Inv-FTRAN 3.29 -
Inv-BTRAN 2.11 -
![Page 23: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/23.jpg)
Poor performance: cre-a
Least speed-up (1.83) on 8 processors
0
1
2
3
4
5
6
7
3.84800 3.85000 3.85200 3.85400 3.85600 3.85800
Operation Slow-down in total time Overall speed-up
Inv-FTRAN 3.29 -
Inv-BTRAN 2.11 -
PRICE 2.94 2.04
SYNPLEX, a task-parallel scheme for the revised simplex method 6
![Page 24: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/24.jpg)
Fair performance: pds-06
Best speed-up (3.05) on 8 processors
0
1
2
3
4
5
6
7
46.64000 46.64800 46.65600 46.66400 46.67200 46.68000
![Page 25: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/25.jpg)
Fair performance: pds-06
Best speed-up (3.05) on 8 processors
0
1
2
3
4
5
6
7
46.64000 46.64800 46.65600 46.66400 46.67200 46.68000
Operation Slow-down in total time Overall speed-up
Inv-FTRAN 2.02 -
![Page 26: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/26.jpg)
Fair performance: pds-06
Best speed-up (3.05) on 8 processors
0
1
2
3
4
5
6
7
46.64000 46.64800 46.65600 46.66400 46.67200 46.68000
Operation Slow-down in total time Overall speed-up
Inv-FTRAN 2.02 -
Inv-BTRAN 1.79 -
![Page 27: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/27.jpg)
Fair performance: pds-06
Best speed-up (3.05) on 8 processors
0
1
2
3
4
5
6
7
46.64000 46.64800 46.65600 46.66400 46.67200 46.68000
Operation Slow-down in total time Overall speed-up
Inv-FTRAN 2.02 -
Inv-BTRAN 1.79 -
PRICE 1.69 3.55
SYNPLEX, a task-parallel scheme for the revised simplex method 7
![Page 28: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/28.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
![Page 29: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/29.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
![Page 30: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/30.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
• Serial INVERT limits scalability
![Page 31: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/31.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
• Serial INVERT limits scalability
SYNPLEX refinements:
• Parallel INVERT
◦ Should allow larger problems to be solved than serial revised simplex solvers
◦ Impressive results from parallel direct methods for linear systems give hope
![Page 32: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/32.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
• Serial INVERT limits scalability
SYNPLEX refinements:
• Parallel INVERT
◦ Should allow larger problems to be solved than serial revised simplex solvers
◦ Impressive results from parallel direct methods for linear systems give hope
Future prospects:
• Pure data parallel revised simplex without multiple pricing
SYNPLEX, a task-parallel scheme for the revised simplex method 8
![Page 33: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/33.jpg)
Bibliography
Paper: http://www.maths.ed.ac.uk/hall/ParSimplexThis talk: http://www.maths.ed.ac.uk/hall/CSC05
SYNPLEX, a task-parallel scheme for the revised simplex method 9
![Page 34: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/34.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
![Page 35: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/35.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
![Page 36: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/36.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
• Serial INVERT limits scalability
![Page 37: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/37.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
• Serial INVERT limits scalability
SYNPLEX refinements:
• Parallel INVERT
◦ Should allow larger problems to be solved than serial revised simplex solvers
◦ Impressive results from parallel direct methods for linear systems give hope
![Page 38: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/38.jpg)
Conclusions
SYNPLEX limitations:
• Algorithmic limitations of revised simplex with multiple pricing
◦ Wasted FTRANs
◦ Increased number of iterations
• Inevitable(?) data management limitations
◦ Some operations are significantly slower in parallel than in serial
• Serial INVERT limits scalability
SYNPLEX refinements:
• Parallel INVERT
◦ Should allow larger problems to be solved than serial revised simplex solvers
◦ Impressive results from parallel direct methods for linear systems give hope
Future prospects:
• Pure data parallel revised simplex without multiple pricing
SYNPLEX, a task-parallel scheme for the revised simplex method 10
![Page 39: SYNPLEX A task-parallel scheme for the revised simplex ... · June 23rd 2005 SYNPLEX, a task-parallel scheme for the revised simplex method. SYNPLEX • Synchronous variant of PARSMI.](https://reader034.fdocuments.in/reader034/viewer/2022042612/5f75c4df6f21391df878f045/html5/thumbnails/39.jpg)
Bibliography
Paper: http://www.maths.ed.ac.uk/hall/ParSimplexThis talk: http://www.maths.ed.ac.uk/hall/CSC05
SYNPLEX, a task-parallel scheme for the revised simplex method 11