From the heroic to the logistical Non-traditional applications for parallel computers

Ian Foster

Computation InstituteArgonne National Lab & University of Chicago From the heroic to the logistical

Non-traditional applications for parallel computers

*

Abstract High-performance computers such as the petascale systems being installed at DOE and NSF centers in the US are conventionally focused on heroic computations in which many processors are applied to a single task. Yet a growing number of science applications are equally concerned with logistical issues: that is, with the high-performance and reliable execution of many tasks that operate on large shared data and/or are linked by communication-intensive producer-consumer relations. Such applications may require the extreme computational capacity and specialized communication fabrics of petascale computers, but are not easily expressed using conventional parallel programming models such as MPI. To enable the use of high-performance computers for these applications, we need new methods for the efficient dispatch, coupling, and management of large numbers of communication-intensive tasks. I discuss how work on scripting languages, high-throughput computing, and parallel I/O can be combined to build new tools that enable the efficient and reliable execution of applications involving from hundreds to millions of uniprocessor and multiprocessor tasks, with aggregate communication requirements of tens of gigabytes per second. I illustrate my presentation by referring to our experiences adapting the Swift parallel programming system (www.ci.uchicago.edu/swift) for efficient execution in both large-scale grid and petascale cluster environments.

*

Many-core growth ratesPat Helland, Microsoft, The Irresistible Forces Meet the Movable Objects, 200720042006200820102012201420162018

What will we do with 1+ Exaflops and 1M+ cores?

*

*Bigger problemsComputationalscientistas hero

*

More complex problemsComputationalscientistas logisticsofficer

*

*More complex problemsEnsemble runs to quantify climate model uncertaintyIdentify potential drug targets by screening a database of ligand structures against target proteinsStudy economic model sensitivity to parametersAnalyze turbulence dataset from many perspectivesPerform numerical optimization to determine optimal resource assignment in energy problemsMine collection of data from advanced light sources Construct databases of computed properties of chemical compoundsAnalyze data from the Large Hadron ColliderAnalyze log data from 100,000-node parallel computations

*

*Programming model issuesMassive task parallelismMassive data parallelismIntegrating black box applicationsComplex task dependencies (task graphs)Failure, and other execution management issuesData management: input, intermediate, outputDynamic computations (task graphs)Dynamic data access to large, diverse datasetsLong-running computationsDocumenting provenance of data products

*

Extreme scriptingSimplescriptsComplexscriptsBigcomputersSmallcomputersMany activitiesNumerous filesComplex dataData dependenciesMany programsMany processorsStorage hierarchyFailureHeterogeneityPreservingfile system semantics,ability to call arbitrary executables

*

Functional magnetic resonance imaging (fMRI) data analysis

*

The messy data problem (1)Scientific data is often logically structuredE.g., hierarchicallyCommon to map functions over dataset membersNested map operations can scale to millions of objects

*

The messy data problem (2)But physically messyHeterogeneous storage format and access protocolLogically identical dataset can be stored in textual File (e.g. CSV), spreadsheet, database, Data available from filesystem, DBMS, HTTP, WebDAV, ..Metadata encoded in directory and file namesHinders program development, composition, execution

./knottastictotal 58drwxr-xr-x 4 yongzh users 2048 Nov 12 14:15 AAdrwxr-xr-x 4 yongzh users 2048 Nov 11 21:13 CHdrwxr-xr-x 4 yongzh users 2048 Nov 11 16:32 EC

./knottastic/AA:total 4drwxr-xr-x 5 yongzh users 2048 Nov 5 12:41 04nov06aadrwxr-xr-x 4 yongzh users 2048 Dec 6 12:24 11nov06aa

. /knottastic//AA/04nov06aa:total 54drwxr-xr-x 2 yongzh users 2048 Nov 5 12:52 ANATOMYdrwxr-xr-x 2 yongzh users 49152 Dec 5 11:40 FUNCTIONAL

. /knottastic/AA/04nov06aa/ANATOMY:total 58500-rw-r--r-- 1 yongzh users 348 Nov 5 12:29 coplanar.hdr-rw-r--r-- 1 yongzh users 16777216 Nov 5 12:29 coplanar.img

. /knottastic/AA/04nov06aa/FUNCTIONAL:total 196739-rw-r--r-- 1 yongzh users 348 Nov 5 12:32 bold1_0001.hdr-rw-r--r-- 1 yongzh users 409600 Nov 5 12:32 bold1_0001.img-rw-r--r-- 1 yongzh users 348 Nov 5 12:32 bold1_0002.hdr-rw-r--r-- 1 yongzh users 409600 Nov 5 12:32 bold1_0002.img-rw-r--r-- 1 yongzh users 496 Nov 15 20:44 bold1_0002.mat-rw-r--r-- 1 yongzh users 348 Nov 5 12:32 bold1_0003.hdr-rw-r--r-- 1 yongzh users 409600 Nov 5 12:32 bold1_0003.img

*

XML Dataset Typing & Mapping (XDTM)Describe logical structure by XML SchemaPrimitive scalar types: int, float, string, date, Complex types (structs and arrays)Use mapping descriptors for mappingsHow dataset elements are mapped to physical representationsExternal parameters (e. g. location)Use XPath for dataset selectionXDTM: XML Dataset Typing and Mapping for Specifying Datasets [EGC05]

*

XDTM: ImplementationVirtual integrationEach data source treated as virtual XML sourceData structure defined as XML schemaMapper responsible for accessing source and translating to/from XML representationBi-directionalCommon mapping interfaceData providers implement the interfaceResponsible for data access detailsStandard mapper implementations providedString, file system, CSV,

*

SwiftScriptTyped parallel programming notationXDTM as data model and type systemTyped dataset and procedure definitions Scripting languageImplicit data parallelismProgram composition from procedures Control constructs (foreach, if, while, )[SIGMOD05, Springer06] Clean application logic Type checkingDataset selection, iteration Discovery by types Type conversion

A Notation & System for Expressing and Executing Cleanly Typed Workflows on Messy Scientific Data [SIGMOD05]

*

fMRI type definitionsin SwiftScripttype Study { Group g[ ]; }

type Group { Subject s[ ]; }

type Subject { Volume anat; Run run[ ]; }

type Run { Volume v[ ]; }

type Volume { Image img; Header hdr; }type Image {};

type Header {};

type Warp {};

type Air {};

type AirVec { Air a[ ]; }

type NormAnat {Volume anat; Warp aWarp; Volume nHires;}Simplified version of fMRI AIRSN Program (Spatial Normalization)

*

AIRSN program definition(Run snr) functional ( Run r, NormAnat a, Air shrink ) {Run yroRun = reorientRun( r , "y" );Run roRun = reorientRun( yroRun , "x" );Volume std = roRun[0];Run rndr = random_select( roRun, 0.1 );AirVector rndAirVec = align_linearRun( rndr, std, 12, 1000, 1000, "81 3 3" );Run reslicedRndr = resliceRun( rndr, rndAirVec, "o", "k" );Volume meanRand = softmean( reslicedRndr, "y", "null" );Air mnQAAir = alignlinear( a.nHires, meanRand, 6, 1000, 4, "81 3 3" );Warp boldNormWarp = combinewarp( shrink, a.aWarp, mnQAAir );Run nr = reslice_warp_run( boldNormWarp, roRun );Volume meanAll = strictmean( nr, "y", "null" )Volume boldMask = binarize( meanAll, "y" );snr = gsmoothRun( nr, boldMask, "6 6 6" );}(Run or) reorientRun (Run ir, string direction) { foreach Volume iv, i in ir.v { or.v[i] = reorient(iv, direction); }}

*

MonitoringSwiftScriptSpecificationExecutionSchedulingSwift + Falkon: scripting + multi-Level schedulingYong Zhao, Mihael Hatigan, Ioan Raicu, Mike Wilde, Ben Clifford(Run snr) functional ( Run r, NormAnat a) {Run Ry = reorientRun( r , "y" );Run Rx = reorientRun( Ry, "x" );}(Run or) reorientRun (Run ir, string direction) { foreach Volume iv, i in ir.v { or.v[i] = reorient(iv, direction); } }

*

B. Berriman, J. Good (Caltech)J. Jacob, D. Katz (JPL)

*

MontageYong Zhao and Ioan Raicu, U.ChicagoMPI: ~950 lines of C for one stagePegasus: ~1200 lines of C + tools to generate DAG for specific dataset SwiftScript: ~92 lines for any dataset

Chart1

544.5273.5295.666666666740.305086527640.305086527631.819805153431.819805153429.365512652329.3655126523

785343.3333333333297.333333333335.355339059335.35533905936.42910050736.429100507347.721413781847.7214137818

261.511591.66666666673.53553390593.53553390594.24264068714.24264068715.13160143945.1316014394

21914515115.556349186115.55634918619.89949493669.899494936625.63201123625.632011236

39271188.333333333311.31370849911.3137084991.41421356241.41421356247.50555349957.5055534995

29031190.51255103.2375900532103.237590053245.961940777145.9619407771102.1028892833102.1028892833

GRAM/Clustering

MPI

Falkon

Components

Time (s)

Sheet1

mProjectmDiff/FitmBackgroundmAdd(sub)mAddtotalmImgtblmImgtblmOverlapsmConcatFitmBgModelmImgtblmSubset

GRAM/Clustering545785262219392290383352381759180

MPI274343115145711191232127481335

Falkon29629792151188125528461100335

GRAM/CLU573810259208384297681332851788778

516760264230400283085361901729582

stdev403541611103

296346118152721223

MPI251348112138701158232127481335

336

stdev326410146

2642449612418811402837481326

Falkon30133693154196129030436102335

32231286175181133526574117334

stdev29485268102

Sheet1

00040.305086527640.305086527631.819805153431.819805153429.365512652329.3655126523

00035.355339059335.35533905936.42910050736.429100507347.721413781847.7214137818

0003.53553390593.53553390594.24264068714.24264068715.13160143945.1316014394

00015.556349186115.55634918619.89949493669.899494936625.63201123625.632011236

00011.31370849911.3137084991.41421356241.41421356247.50555349957.5055534995

000103.2375900532103.237590053245.961940777145.9619407771102.1028892833102.1028892833

GRAM/Clustering

MPI

Falkon

Components

Time (s)

Sheet2

Sheet3

*

Many many tasks:Identifying potential drug targets 2M+ ligands Protein x target(s) (Mike Kubal, Benoit Roux, and others)

*

startreportDOCK6Receptor(1 per protein:defines pocketto bind to)ZINC3-DstructuresligandscomplexesNAB script parameters(defines flexibleresidues, #MDsteps)Amber Score:1. AmberizeLigand3. AmberizeComplex5. RunNABScriptendBuildNABScriptNABScriptNABScriptTemplateAmber prep:2. AmberizeReceptor4. perl: gen nabscriptFREDReceptor(1 per protein:defines pocketto bind to)Manually prepDOCK6 rec fileManually prepFRED rec file1 protein(1MB)PDBproteindescriptionsFor 1 target:4 million tasks 500,000 cpu-hrs(50 cpu-years)

*

*

IBM BG/P570 Tflop/s, 164,000 cores, 80 TB mem

*

DOCK on BG/P: ~1M tasks on 119,000 CPUs 118784 cores 934803 tasks Elapsed time: 7257 sec Compute time: 21.43 CPU years Average task: 667 sec

Relative efficiency 99.7% (from 16 to 32 racks)Utilization: 99.6% sustained, 78.3% overallIoan Raicu et al.Time (sec)

*

Managing 160,000 coresSlower shared storageHigh-speed local diskFalkon

*

***Storage resource scalabilityIBM BlueGene/P160K CPU coresGPFS 8GB/s I/O rates (16 servers)On160K CPU BG/P, we achieve 0.3Mb/s per CPU coreOn 5.7K CPU SiCortex achieved 0.06Mb/s per CPU core GPFS vs. localRead Throughput1 node: 0.48Gb/s vs. 1.03Gb/s 2.15x160 nodes: 3.4Gb/s vs. 165Gb/s 48x11Mb/s per CPU vs. 515Mb/s per CPURead+Write Throughput:1 node: 0.2Gb/s vs. 0.39Gb/s 1.95x160 nodes: 1.1Gb/s vs. 62Gb/s 55xMetadata (mkdir / rm -rf)1 node: 151/sec vs. 199/sec 1.3x160 nodes: 21/sec vs. 31840/sec 1516x

c_data_gpfs_cache_r_1l_task

11.0946184626.942.792.4396298804196.4295.3378.3

12.034316605927.5241.583.3197.94290.8884312194374.8

11.752283891127.1142.3478.6159.484285.7740877695365.8

10.427380018923.6535.162102.1282051282223.6540351644353.4

7.274793975616.4616.955578.088154.4378252017203.5577572619

3.33850627935.5036601495.3121.490482486533.463333333337.890841813133.69548891

0.5420.98766194180.642.60368991683.80522388063.86601830326.468305304

0.05723852730.0726806910.060.26937614170.360.35998942530.6391818472

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size

Read Throughput (tasks/sec)

c_data_gpfs_cache_r+w_1l_task

10.9526.4932.692.3687533826188.1268.3374.8

11.6226.929.975.2187.53268.5901639344375.6848668052

11.4126.628526849632.370.7035835117147.912248.272517881364.5

10.0422.817670324825.758.997.5025641026206.6286636735347.8979893811

7.1414.94148889620.9151.936195572270.156134.3303161485180.2590442162

2.93077196084.94958576228.821.558802475930.326666666736.642095469837.4716696046

0.27426964990.59102746431.071.81943526443.73208955223.74006545113.1568036185

0.02127167380.05313743330.070.16249016170.24630067930.33301852680.3333351255

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_data_gpfs_cache_r_8l_task

25.6443.1550.52118.4200.1684210526398.3381.1

25.643.3646.98119.2195.43400.4380.4

25.7343.6452.75118.4133.4666666667336.8368.6

25.8943.0851.88108.494.2604651163284.8202489396358.2

21.833443.1580.5101.4206.3891968155243.6229205176

10.466.56174706529.3322.421230102339.44886.028732252470.5727884458

1.2520.6124519130.741.7172218573.48545910036.7848718567.215007215

0.130.06638150280.070.14744913010.26719102910.37169539560.6560859785

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_cache_sched_r_8l_task

25.1948.06107.9159.7721997933305405.3459.4761343878

25.148.27106.8158.2872821424307.7405.3446.8932409579

25.2748.96110.2159.4518841482298.8408453.7247299917

25.2348109.5156.6827327672280401.1753183154465.7191586128

19.6341.3687.3156.5509860878254.7897486937400.3910068426449.6158068057

10.3720.6626.304297567482.4161.0569361434247.8218780252293.5779816514

1.2722.494.959125954110.215.39247812894067.1

0.12930.20984156960.3083683460.64738013350.73481290971.23970944312.5897824987

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_data_gpfs_cache_r+w_8l_task

25.2342.1242.61113.7240747425202.7052631579365.8373.4

25.212980745642.7444.05114199.42365.3880463872367.9

25.304569155142.556728451545.55110.7145.6703703704286.6340097971358.9

25.582731656141.897669851344.19103.599.2048780488237.064475055337.2736418511

21.55607948832.59900674935.1276.897082566786.6157894737185.4064819844196.4

4.193.35625462964.817.088311861735.4858.88103041867.2998643148

0.450.34437392550.481.20637493761.61093247593.70678733037.9065376446

0.0460.03306850970.040.07483769570.14285714290.28618621930.6110903034

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_cache_sched_r+w_8l_task

24.3246.0970559107104.4195.3406299926290.2494331066397.8050793959457.1

24.002625287147.4326609074104.266367987197.3642999976299.3058092802394.9440.9279293826

24.925758239648.474520107106.7000104199192.4766805291290.5067555587400.156311059448.5326325011

23.831134073446.8392644772104.1603092259178.3816740702270.5952302306368.5773418519413.9464375947

19.20120007539.184173267583.0326373404143.6991299467226.2733399624296.9406988546352.9209029812

4.238.4614.851341551833.767.8135.593220339212.7571161438

0.4748760870.91292936211.38933360833.65.438889657613.828.4

0.04865440170.08436994110.10197968050.23324295170.36424689110.6736274841.6596510717

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_r_task

12.441679626732.9263.8842098696157.3931755303262.8420484.8

13.105187043332.680677230462.6644636191161.0062893082271.8410.318056599486.0278849006

13.087767283131.650362402865.0292917586150.339511837256.6325481.3

11.560041092326.656427544157.9341169149116.9083228679244.8249477.9184411645

8.306698898418.455272098140.469109699373.405017921180.9159.1346.5899475377

4.39654802289.081717721816.641476931126.637185406818.420413559942.940.695479384

0.92802041641.73041955912.76000366563.62270131823.81176435553.60631951153.9340130775

0.10519740950.19711716150.3165809260.41025115060.41846749360.42404887160.4218797379

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_r+w_task

10.793056.6121.2239.3292.2504749249437.8

11.3730.061502781657.7706943485119.1945058782232.028550388284.2432.2

11.043527026428.9857.745446343115.601715963235.7545758029258435.7

9.86968925923.444833665344.936917169599.0688112226226.8246760439211.6417.7

6.896737519915.107924283324.520192042942.039576319969.4661149176128.6152.0415738679

2.40646738113.35475923715.24864427117.691556563710.816977584313.112.4695567462

0.26516297160.44051195750.81645670551.11627582480.7900941941.24708475671.2570710245

0.03201408620.05169277690.09139018750.11826446890.09860261610.12623585390.1271157826

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_wrapper_r_task

10.141425345618.744965951527.01846965725.503088264619.900497512418.827336409920.8672970329

10.879265649619.556913674626.98145025324.350803766819.961013645218.96226065721.1920529801

10.725657784519.012253991826.286066331224.648565376518.860973992518.753548339821.5415684955

9.551526005516.479979426.803476075829.629629629621.761305678318.707638343423.2748431676

7.19101123613.065224048224.679456280728.780213603123.607524898619.37412494822.39720035

4.18567387717.187780772713.236130517.780247256618.581694127915.599530795417.4090445427

0.92839062041.61239528882.67838459933.69739160583.69659076143.58227334423.7820588583

0.10449184310.19565642730.31402720260.40236388780.40772641560.40955019870.4206181773

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_wrapper_r+w_task

8.953553441514.652853299823.882824890427.532802753324.490576867918.123252274319.8881292728

9.517790088114.569461043824.148665220327.604054345523.81173844298.536463369921.2219182625

9.561157796515.080113100823.701509119527.85333478424.970737417116.026293137220.7961007311

8.577076423113.89793702523.780771017229.868160074723.64677627939.17151813721.4783119389

6.474292506511.030203800219.342652058925.623060754722.01393069055.852532191820.6551557205

2.42403583033.26114649685.19238180237.51438299879.56687469649.14767600799.984399376

0.27572420690.41791551580.73698119831.12352153781.2074332611.21531486621.0946064975

0.03052852510.05726454430.09116419530.11485383060.12646921660.12797338150.1277865964

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_data_gpfs_cache_r_1l_io

0.00008875690.00021520.00034160.0007395170.00157120.00236240.0030264

0.09627453280.220160.3320.66641.583522.32710744982.9984

0.94018271132.16883.38726.28812.7587222.861927021629.264

8.341904015218.9228.0849.681.7025641026178.9232281315282.72

58.1983518045131.68135.6440624.7041235.50260161381628.462058095

267.0805023409440.2928119189424.81719.23859892132677.06666666673031.26734505092695.6391128023

433.6790.12955346885122082.95193344323044.17910447763092.81464254435174.6442432083

457.9082180216581.44552809344802155.009133534828802879.9154024855113.4547778843

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size

Read Throughput (Mb/s)

c_data_gpfs_cache_r+w_1l_io

0.00008760.000211920.00026080.000738950.00150480.00214640.0029984

0.092960.21520.23920.60161.500242.14872131153.0054789344

0.91282.1302821482.5845.656286680911.8329619.861801430529.16

8.03218.254136259820.5647.1278.0020512821165.3029309388278.3183915049

57.12119.5319111682167.28415.4895645779561.2481074.6425291881442.0723537293

234.4617568604395.96686097667041724.70419807152426.13333333332931.36763758682997.7335683707

219.4157199364472.8219714468561455.54821148092985.6716417912992.05236091632525.442894836

170.1733907314425.0994666335601299.92129382791970.40543400872664.14821405812666.6810036613

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size

Read+Write Throughput (Mb/s)

c_data_gpfs_cache_r_8l_io

0.000205120.00034520.000404160.00094720.00160134740.00318640.0030488

0.20480.346880.375840.95361.563443.20323.0432

2.05843.49124.229.47210.677333333326.94429.488

20.71234.46441.50486.7275.408372093227.8561991517286.56

174.64272345.2644811.21651.11357452381948.9833641405

836.8524.9397652125746.41793.69840818023155.846882.29858018995645.8230756606

1001.6489.96153036425921373.77748561162788.36728025275427.89748482685772.0057720058

1040531.0520223545601179.59304040112137.52823248962973.56316498635248.687828043

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_data_gpfs_cache_sched_r_8l_io

0.000201520.000384480.00086320.00127817760.002440.00324240.0036758091

0.20080.386160.85441.26629825712.46163.24243.5751459277

2.02163.91688.81612.756150731923.90432.6436.2979783993

20.18438.487.6125.3461862138224320.9402546523372.5753268903

157.04330.88698.41252.4078887022038.31798954963203.1280547413596.9264544457

829.61652.82104.3438053893659212884.554891475319825.750242013623486.2385321101

1017.619923967.3007632405816012313.98250308153200053680

1034.41678.73255691952466.94676791435179.04106817725878.50327795459917.675544794220718.2599898837

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_data_gpfs_cache_r+w_8l_io

0.000201840.000336960.000340880.00090979260.00162164210.00292640.0029872

0.2017038460.341920.35240.9121.595362.92310437112.9432

2.02436553243.40453827613.6448.85611.653629629622.930720783828.712

20.466185324933.51813588135.35282.879.363902439189.651580044269.8189134809

172.4486359043260.7920539921280.96615.1766605339692.92631578951483.25185587541571.2

335.2268.50037036793841367.06494893532838.44710.48243344265383.9891451832

360275.4991404104384965.09995004851288.74598070742965.42986425346325.2301156479

368264.5480774795320598.70156597881142.85714285712289.48975475874888.7224270141

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_cache_sched_r+w_8l_io

0.000194560.00036877640.00083520.0015627250.00232199550.00318244060.0036568

0.19202100230.37946128730.83413094391.57891442.39444647423.15923.5274234351

1.99406065923.87796160868.536000833615.398134442323.240540444732.012504884735.8826106001

19.064907258737.471411581783.3282473807142.7053392562216.4761841844294.8618734815331.1571500758

153.6096006313.4733861401664.26109872291149.59303957341810.18671969952375.52559083662823.3672238497

338.4676.81188.10732414792696542410847.457627118617020.5692915022

379.9008696168730.34348967251111.466886644528804351.11172607581104022720

389.2352136232674.9595287939815.83744438921865.94361351642913.9751287675389.019872010813277.2085739716

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_r_io

0.00009953340.000263360.00051107370.00125914540.00210240.003360.0038784

0.10484149630.26144541780.5013157091.28805031452.17443.28254445283.8882230792

1.04702138272.53202899225.202343340712.02716094720.5282638.504

9.248032873921.325142035346.347293531993.5266582943195.84199.2382.3347529316

66.4535911871147.6421767849323.7528775948587.2401433692647.21272.82772.7195803012

351.7238418273726.53741774131331.31815448622130.97483254211473.633084789734323255.6383507203

742.41633315931384.33564731892208.00293250392898.16105454573049.41148442912885.05560916513147.210462016

841.57927611031576.9372921032532.64740799373282.00920501023347.7399486073392.39097305963375.0379032577

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_r+w_io

0.000086320.000240.00045280.00096960.00191440.00233800380.0035024

0.090960.24049202230.46216555480.9535560471.85622840312.27363.4576

0.88348216212.31844.61963570749.24813727718.860366064220.6434.856

7.895751407218.755866932335.949533735679.2550489781181.4597408351169.28334.16

55.1739001589120.8633942667196.1615363433336.3166105592555.72891934061028.81216.3325909428

192.5173904869268.3807389643419.8915416867615.324525099865.35820674791048997.564539698

212.1303773186352.4095659925653.1653643757893.0206598237632.0753552338997.66780536221005.6568196103

256.1126895834413.5422152293731.1215003984946.115751349788.82092840531009.88683152941016.926260929

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_wrapper_r_io

0.00008113140.00014995970.00021614780.00020402470.0001592040.00015061870.0001669384

0.08703412520.15645530940.2158516020.19480643010.15968810920.15169808530.1695364238

0.85805262281.52098031932.10288530651.97188523011.50887791941.50028386721.7233254796

7.641220804413.1839835221.442780860623.703703703717.409044542714.966110674718.619874534

57.5280898876104.5217923854197.4356502458230.2417088252188.8601991885154.9929995838179.1776027996

334.8539101715575.02246181491058.89043999791422.41978052511486.53553023161247.9624636291392.7235634138

742.71249628281289.9162310262142.70767943082957.91328461252957.27260912882865.81867539853025.6470866328

835.9347448441565.25141850912512.21762085143218.91110272853261.81132460123276.40158956673364.9454182193

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


c_gpfs_wrapper_r+w_io

0.00007162840.00011722280.00019106260.00022026240.00019592460.0001449860.000159105

0.07614232070.11655568840.19318932180.22083243480.19049390750.0682917070.1697753461

0.76489262371.20640904811.89612072962.22826678271.99765899341.2821034511.6636880585

6.861661138511.1183496219.024616813723.894528059718.91742102357.337214509617.1826495511

51.794340052188.241630402154.7412164715204.9844860374176.111445524146.8202575343165.2412457641

193.9228664224260.8917197452415.390544186601.1506398967765.3499757091731.8140806318798.751950078

220.5793654897334.3324126132589.5849586744898.8172302561965.9466088105972.2518929953875.6851980126

244.2282007251458.1163543959729.3135620985918.83064443771011.7537328181023.78705229311022.2927710723

&A

Page &P

1 Executor

2 Executor

4 Executor

8 Executor

16 Executor

32 Executor

64 Executor

Data Size


data_gpfs_cache_r_1loc

Throughput (tasks/sec)Throughput (Mb/sec)

1 Executor2 Executor4 Executor8 Executor16 Executor32 Executor64 Executor1 Executor2 Executor4 Executor8 Executor16 Executor32 Executor64 Executor

1B11.126.942.792.4196.4295.3378.31B0.00.00.00.00.00.00.0

1KB12.027.541.583.3197.9290.9374.81KB0.10.20.30.71.62.33.0

10KB11.827.142.378.6159.5285.8365.810KB0.92.23.46.312.822.929.3

100KB10.423.735.162.0102.1223.7353.4100KB8.318.928.149.681.7178.9282.7

1MB7.316.517.055.078.1154.4203.61MB58.2131.7135.6440.0624.71235.51628.5

10MB3.35.55.321.533.537.933.710MB267.1440.3424.81719.22677.13031.32695.6

100MB0.51.00.62.63.83.96.5100MB433.6790.1512.02083.03044.23092.85174.6

1GB0.10.10.10.30.40.40.61GB457.9581.4480.02155.02880.02879.95113.5

data_gpfs_cache_r_8loc



1B25.643.250.5118.4200.2398.3381.11B0.00.00.00.00.00.00.0

1KB25.643.447.0119.2195.4400.4380.41KB0.20.30.41.01.63.23.0

10KB25.743.652.8118.4133.5336.8368.610KB2.13.54.29.510.726.929.5

100KB25.943.151.9108.494.3284.8358.2100KB20.734.541.586.775.4227.9286.6

1MB21.834.043.280.5101.4206.4243.61MB174.6272.0345.2644.0811.21651.11949.0

10MB10.56.69.322.439.486.070.610MB836.8524.9746.41793.73155.86882.35645.8

100MB1.30.60.71.73.56.87.2100MB1001.6490.0592.01373.82788.45427.95772.0

1GB0.10.10.10.10.30.40.71GB1040.0531.1560.01179.62137.52973.65248.7

data_gpfs_cache_sched_r_8loc



1B25.248.1107.9159.8305.0405.3459.51B0.00.00.00.00.00.00.0

1KB25.148.3106.8158.3307.7405.3446.91KB0.20.40.91.32.53.23.6

10KB25.349.0110.2159.5298.8408.0453.710KB2.03.98.812.823.932.636.3

100KB25.248.0109.5156.7280.0401.2465.7100KB20.238.487.6125.3224.0320.9372.6

1MB19.641.487.3156.6254.8400.4449.61MB157.0330.9698.41252.42038.33203.13596.9

10MB10.420.726.382.4161.1247.8293.610MB829.61652.82104.36592.012884.619825.823486.2

100MB1.32.55.010.215.440.067.1100MB1017.61992.03967.38160.012314.032000.053680.0

1GB0.10.20.30.60.71.22.61GB1034.41678.72466.95179.05878.59917.720718.3

data_gpfs_cache_r+w_1loc



1B11.026.532.692.4188.1268.3374.81B0.00.00.00.00.00.00.0

1KB11.626.929.975.2187.5268.6375.71KB0.10.20.20.61.52.13.0

10KB11.426.632.370.7147.9248.3364.510KB0.92.12.65.711.819.929.2

100KB10.022.825.758.997.5206.6347.9100KB8.018.320.647.178.0165.3278.3

1MB7.114.920.951.970.2134.3180.31MB57.1119.5167.3415.5561.21074.61442.1

10MB2.94.98.821.630.336.637.510MB234.5396.0704.01724.72426.12931.42997.7

100MB0.30.61.11.83.73.73.2100MB219.4472.8856.01455.52985.72992.12525.4

1GB0.00.10.10.20.20.30.31GB170.2425.1560.01299.91970.42664.12666.7

data_gpfs_cache_r+w_8loc



1B25.242.142.6113.7202.7365.8373.41B0.00.00.00.00.00.00.0

1KB25.242.744.1114.0199.4365.4367.91KB0.20.30.40.91.62.92.9

10KB25.342.645.6110.7145.7286.6358.910KB2.03.43.68.911.722.928.7

100KB25.641.944.2103.599.2237.1337.3100KB20.533.535.482.879.4189.7269.8

1MB21.632.635.176.986.6185.4196.41MB172.4260.8281.0615.2692.91483.31571.2

10MB4.23.44.817.135.558.967.310MB335.2268.5384.01367.12838.44710.55384.0

100MB0.50.30.51.21.63.77.9100MB360.0275.5384.0965.11288.72965.46325.2

1GB0.00.00.00.10.10.30.61GB368.0264.5320.0598.71142.92289.54888.7

data_gpfs_cache_sched_r+w_8loc



1B24.346.1104.4195.3290.2397.8457.11B0.00.00.00.00.00.00.0

1KB24.047.4104.3197.4299.3394.9440.91KB0.20.40.81.62.43.23.5

10KB24.948.5106.7192.5290.5400.2448.510KB2.03.98.515.423.232.035.9

100KB23.846.8104.2178.4270.6368.6413.9100KB19.137.583.3142.7216.5294.9331.2

1MB19.239.283.0143.7226.3296.9352.91MB153.6313.5664.31149.61810.22375.52823.4

10MB4.28.514.933.767.8135.6212.810MB338.4676.81188.12696.05424.010847.517020.6

100MB0.50.91.43.65.413.828.4100MB379.9730.31111.52880.04351.111040.022720.0

1GB0.00.10.10.20.40.71.71GB389.2675.0815.81865.92914.05389.013277.2

data_gpfs_r



1B12.432.963.9157.4262.8420.0484.81B0.00.00.00.00.00.00.0

1KB13.132.762.7161.0271.8410.3486.01KB0.10.30.51.32.23.33.9

10KB13.131.765.0150.3256.6325.0481.310KB1.02.55.212.020.526.038.5

100KB11.626.757.9116.9244.8249.0477.9100KB9.221.346.393.5195.8199.2382.3

1MB8.318.540.573.480.9159.1346.61MB66.5147.6323.8587.2647.21272.82772.7

10MB4.49.116.626.618.442.940.710MB351.7726.51331.32131.01473.63432.03255.6

100MB0.91.72.83.63.83.63.9100MB742.41384.32208.02898.23049.42885.13147.2

1GB0.10.20.30.40.40.40.41GB841.61576.92532.63282.03347.73392.43375.0

data_gpfs_r+w



1B10.830.056.6121.2239.3292.3437.81B0.00.00.00.00.00.00.0

1KB11.430.157.8119.2232.0284.2432.21KB0.10.20.51.01.92.33.5

10KB11.029.057.7115.6235.8258.0435.710KB0.92.34.69.218.920.634.9

100KB9.923.444.999.1226.8211.6417.7100KB7.918.835.979.3181.5169.3334.2

1MB6.915.124.542.069.5128.6152.01MB55.2120.9196.2336.3555.71028.81216.3

10MB2.43.45.27.710.813.112.510MB192.5268.4419.9615.3865.41048.0997.6

100MB0.30.40.81.10.81.21.3100MB212.1352.4653.2893.0632.1997.71005.7

1GB0.00.10.10.10.10.10.11GB256.1413.5731.1946.1788.81009.91016.9

data_gpfs_wrapper_r



1B10.118.727.025.519.918.820.91B0.00.00.00.00.00.00.0

1KB10.919.627.024.420.019.021.21KB0.10.20.20.20.20.20.2

10KB10.719.026.324.618.918.821.510KB0.91.52.12.01.51.51.7

100KB9.616.526.829.621.818.723.3100KB7.613.221.423.717.415.018.6

1MB7.213.124.728.823.619.422.41MB57.5104.5197.4230.2188.9155.0179.2

10MB4.27.213.217.818.615.617.410MB334.9575.01058.91422.41486.51248.01392.7

100MB0.91.62.73.73.73.63.8100MB742.71289.92142.72957.92957.32865.83025.6

1GB0.10.20.30.40.40.40.41GB835.91565.32512.23218.93261.83276.43364.9

data_gpfs_wrapper_r+w



1B9.014.723.927.524.518.119.91B0.00.00.00.00.00.00.0

1KB9.514.624.127.623.88.521.21KB0.10.10.20.20.20.10.2

10KB9.615.123.727.925.016.020.810KB0.81.21.92.22.01.31.7

100KB8.613.923.829.923.69.221.5100KB6.911.119.023.918.97.317.2

1MB6.511.019.325.622.05.920.71MB51.888.2154.7205.0176.146.8165.2

10MB2.43.35.27.59.69.110.010MB193.9260.9415.4601.2765.3731.8798.8

100MB0.30.40.71.11.21.21.1100MB220.6334.3589.6898.8965.9972.3875.7

1GB0.00.10.10.10.10.10.11GB244.2458.1729.3918.81011.81023.81022.3

c_1exec_task

12.441679626710.141425345611.0946184625.6425.1910.798.953553441510.9525.2324.32

13.105187043310.879265649612.034316605925.625.111.379.517790088111.6225.212980745624.0026252871

13.087767283110.725657784511.752283891125.7325.2711.04352702649.561157796511.4125.304569155124.9257582396

11.56004109239.551526005510.427380018925.8925.239.8696892598.577076423110.0425.582731656123.8311340734

8.30669889847.1910112367.274793975621.8319.636.89673751996.47429250657.1421.55607948819.201200075

4.39654802284.18567387713.338506279310.4610.372.40646738112.42403583032.93077196084.194.23

0.92802041640.92839062040.5421.2521.2720.26516297160.27572420690.27426964990.450.474876087

0.10519740950.10449184310.05723852730.130.12930.03201408620.03052852510.02127167380.0460.0486544017

&A

Page &P

Read GPFS

Read Wrapper GPFS

Read Caching 0% Locality


Read Caching Data-Aware Scheduler 100% Locality

Read+Write GPFS

Read+Write Wrapper GPFS

Read+Write Caching 0% Locality


Read+Write Caching Data-Aware Scheduler 100% Locality

Data Size

Throughput (tasks/sec)

c_1exec_task_large

12.441679626710.141425345611.0946184625.6425.1910.798.953553441510.9525.2324.32

13.105187043310.879265649612.034316605925.625.111.379.517790088111.6225.212980745624.0026252871

13.087767283110.725657784511.752283891125.7325.2711.04352702649.561157796511.4125.304569155124.9257582396

11.56004109239.551526005510.427380018925.8925.239.8696892598.577076423110.0425.582731656123.8311340734

8.30669889847.1910112367.274793975621.8319.636.89673751996.47429250657.1421.55607948819.201200075

4.39654802284.18567387713.338506279310.4610.372.40646738112.42403583032.93077196084.194.23

0.92802041640.92839062040.5421.2521.2720.26516297160.27572420690.27426964990.450.474876087

0.10519740950.10449184310.05723852730.130.12930.03201408620.03052852510.02127167380.0460.0486544017

&A

Page &P

Read GPFS

Read Wrapper GPFS




Read+Write GPFS





Data Size


c_1exec_io r

0.00009953340.00008113140.00008875690.000205120.000201520.00043636850.0399169727

0.10484149630.08703412520.09627453280.20480.20080.423057217111.0968085579

1.04702138270.85805262280.94018271132.05842.02164.053469363416.0980823964

9.24803287397.64122080448.341904015220.71220.18432.760059491878.9131077245

66.453591187157.528089887658.1983518045174.64157.04128.3489927345290.7931519543

351.7238418273334.8539101715267.0805023409836.8829.6287.2532144499837.1255163872

742.4163331593742.7124962828433.61001.61017.6424.65624994581028.5321736713

841.5792761103835.934744844457.908218021610401034.4481.93665043241035.6737886202

&A

Page &P

Read GPFS

Read Wrapper GPFS




Read GPFS (model)

Read Local (model)

Data Size

Throughput (Mb/s)

c_1exec_io rw

0.000086320.00007162840.00008760.000201840.000194560.00040371540.0215703192

0.090960.07614232070.092960.2017038460.19202100230.41001845328.6974672975

0.88348216210.76489262370.91282.02436553241.99406065923.881313987914.7540499867

7.89575140726.86166113858.03220.466185324919.064907258737.572889422266.9655722362

55.173900158951.794340052157.12172.4486359043153.6096006114.8653097062194.8268884551

192.5173904869193.9228664224234.4617568604335.2338.4171.1099731467341.0114120476

212.1303773186220.5793654897219.4157199364360379.9008696168190.9257890786370.2555482451

256.1126895834244.2282007251170.1733907314368389.2352136232203.5349501541389.349180605

&A

Page &P

Read+Write GPFS





Read+Write GPFS (model)

Read+Write Local (model)

Data Size

Throughput (Mb/s)

data_1exec


Read Caching 0% LocalityRead Caching 100% LocalityRead Caching Data-Aware Scheduler 0% LocalityRead Caching Data-Aware Scheduler 100% LocalityRead+Write Caching 0% LocalityRead+Write Caching 100% LocalityRead+Write Caching Data-Aware Scheduler 0% LocalityRead+Write Caching Data-Aware Scheduler 100% LocalityRead GPFSRead+Write GPFSRead Wrapper GPFSRead+Write Wrapper GPFSRead Caching 0% LocalityRead Caching 100% LocalityRead Caching Data-Aware Scheduler 0% LocalityRead Caching Data-Aware Scheduler 100% LocalityRead+Write Caching 0% LocalityRead+Write Caching 100% LocalityRead+Write Caching Data-Aware Scheduler 0% LocalityRead+Write Caching Data-Aware Scheduler 100% LocalityRead GPFSRead+Write GPFSRead Wrapper GPFSRead+Write Wrapper GPFSRead GPFS (model)Read+Write GPFS (Model)Read Local (model)Read+Write Local (model)

1B11.125.611.025.211.025.210.624.312.410.810.19.01B0.00.00.00.00.00.00.00.00.00.00.00.0

1KB12.025.611.825.111.625.212.224.013.111.410.99.51KB0.10.20.10.20.10.20.10.20.10.10.10.1

10KB11.825.711.625.311.425.312.024.913.111.010.79.610KB0.92.10.92.00.92.01.02.01.00.90.90.8

100KB10.425.910.225.210.025.610.423.811.69.99.68.6100KB8.320.78.220.28.020.58.319.19.27.97.66.9

1MB7.321.86.919.67.121.66.919.28.36.97.26.51MB58.2174.655.5157.057.1172.454.9153.666.555.257.551.8

10MB3.310.52.810.42.94.22.34.24.42.44.22.410MB267.1836.8222.3829.6234.5335.2181.2338.4351.7192.5334.9193.9

100MB0.51.30.61.30.30.50.40.50.90.30.90.3100MB433.61001.6450.71017.6219.4360.0335.2379.9742.4212.1742.7220.6

1GB0.10.10.10.10.00.00.00.00.10.00.10.01GB457.91040.0439.11034.4170.2368.0305.8389.2841.6256.1835.9244.2

StdDevStdDev

1B001.04368303831.3615777975051.48880398922.265195985700001B000.00000834950.000010892600.000040.00001191040.00001812160000

1KB001.15656586211.4940254364001.15643709781.954836176400001KB000.00925252690.0119522035000.00925149680.01563868940000

10KB001.20351367031.5627790772001.04103331231.449927634000010KB000.09628109360.1250223262000.0832826650.11599421070000

100KB000.82186890491.2746539682001.84861706652.11535355880000100KB000.65749512391.0197231745001.47889365321.6922828470000

1MB000.63885001731.7698043737000.66736988282.456627691200001MB005.110800138614.1584349898005.338959062719.65302152930000

10MB000.73063139213.906209563000.62404944022.4313512524000010MB0058.4505113672312.49676504010049.923955219194.50810019420000

100MB000.49334007411.5774022807000.49150029490.49773375570000100MB00394.67205928561261.921824541200393.20023588398.18700455540000

1GB000.2273662250.3514342782000.18911836050.214815120800001GB001818.92980025682811.4742255695001512.94688431351718.52096610420000

Local Cache Hits

1B000100010000

1KB000100010000

10KB000100010000

100KB000100010000

1MB000100010000

10MB000100010000

100MB000100010000

1GB000100010000

Global Cache Hits

1B000100010000

1KB000100010000

10KB000100010000

100KB000100010000

1MB000100010000

10MB000100010000

100MB000100010000

1GB000100010000

Time_ms/callRead Caching 0% LocalityRead Caching 100% LocalityRead Caching Data-Aware Scheduler 100% LocalityRead+Write Caching 0% LocalityRead+Write Caching 100% LocalityRead+Write Caching Data-Aware Scheduler 100% LocalityRead GPFSRead+Write GPFSRead Wrapper GPFSRead+Write Wrapper GPFSbytes per fileRead Caching 0% LocalityRead Caching 100% LocalityRead Caching Data-Aware Scheduler 100% LocalityRead+Write Caching 0% LocalityRead+Write Caching 100% LocalityRead+Write Caching Data-Aware Scheduler 100% LocalityRead GPFSRead+Write GPFSRead Wrapper GPFSRead+Write Wrapper GPFS

1B90.133789062539.001560062439.698292973491.324200913239.635354736441.118421052680.37592.678405931498.60546875111.68751B1111111111

1KB83.09570312539.062539.840637450286.058519793539.66210937541.66210937576.305664062587.950747581491.91796875105.066406251KB1000100010001000100010001000100010001000

10KB85.0898437538.865137971239.572615749987.642418930839.518554687540.11914062576.407226562590.5507812593.234375104.5898437510KB10000100001000010000100001000010000100001000010000

100KB95.901367187538.624951718839.635354736499.601593625539.088867187541.961914062586.5048828125101.3203125104.6953125116.58984375100KB100000100000100000100000100000100000100000100000100000100000

1MB137.460937545.808520384850.9424350484140.05602240946.39062552.080078125120.384765625144.99609375139.0625154.457031251MB1000000100000010000001000000100000010000001000000100000010000001000000

10MB299.5351562595.602294455196.4320154291341.20703125238.6634844869236.4066193853227.451171875415.546875238.91015625412.5351562510MB10000000100000001000000010000000100000001000000010000000100000001000000010000000

100MB1845.0184501845798.7220447284786.16352201263646.0468752222.22222222222105.81251077.56253771.2656251077.13281253626.8125100MB100000000100000000100000000100000000100000000100000000100000000100000000100000000100000000

1GB17470.757692.30769230777733.952049497347010.87521739.130434782620553.1259505.937531236.259570.12532756.251GB1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000

Time_ns/byteRead Caching 0% LocalityRead Caching 100% LocalityRead Caching Data-Aware Scheduler 100% LocalityRead+Write Caching 0% LocalityRead+Write Caching 100% LocalityRead+Write Caching Data-Aware Scheduler 100% LocalityRead GPFSRead+Write GPFSRead Wrapper GPFSRead+Write Wrapper GPFS

1B90133789.062539001560.062402539698292.973402191324200.91324239635354.736424941118421.05263168037500092678405.93141898605468.75111687500

1KB83095.70312539062.539840.637450199286058.519793459539662.10937541662.10937576305.664062587950.747581354491917.96875105066.40625

10KB8508.9843753886.5137971243957.26157499018764.24189307623951.855468754011.91406257640.722656259055.0781259323.437510458.984375

100KB959.013671875386.2495171881396.3535473642996.015936255390.888671875419.619140625865.0488281251013.2031251046.9531251165.8984375

1MB137.460937545.808520384850.9424350484140.05602240946.39062552.080078125120.384765625144.99609375139.0625154.45703125

10MB29.9535156259.56022944559.643201542934.12070312523.866348448723.640661938522.745117187541.554687523.89101562541.253515625

100MB18.45018450187.98722044737.861635220136.4604687522.222222222221.05812510.77562537.7126562510.77132812536.268125

1GB17.470757.69230769237.733952049547.01087521.739130434820.5531259.505937531.236259.57012532.75625

data_1exec_auto


Read Caching 0% LocalityRead Caching 100% LocalityRead Caching Data-Aware Scheduler 0% LocalityRead Caching Data-Aware Scheduler 100% LocalityRead+Write Caching 0% LocalityRead+Write Caching 100% LocalityRead+Write Caching Data-Aware Scheduler 0% LocalityRead+Write Caching Data-Aware Scheduler 100% LocalityRead GPFSRead+Write GPFSRead Wrapper GPFSRead+Write Wrapper GPFSRead Caching 0% LocalityRead Caching 100% LocalityRead Caching Data-Aware Scheduler 0% LocalityRead Caching Data-Aware Scheduler 100% LocalityRead+Write Caching 0% LocalityRead+Write Caching 100% LocalityRead+Write Caching Data-Aware Scheduler 0% LocalityRead+Write Caching Data-Aware Scheduler 100% LocalityRead GPFSRead+Write GPFSRead Wrapper GPFSRead+Write Wrapper GPFS

1B11.118.911.018.812.520.210.623.012.412.410.19.01B0.00.00.00.00.00.00.00.00.00.00.00.0

1KB12.018.911.818.65.625.212.224.013.110.610.99.51KB0.10.20.10.10.00.20.10.20.10.10.10.1

10KB11.819.311.618.914.125.312.024.913.111.010.79.610KB0.91.50.91.51.12.01.02.01.00.90.90.8

100KB10.419.210.218.812.425.610.423.811.69.99.68.6100KB8.315.48.215.19.920.58.319.19.27.97.66.9

1MB7.319.06.917.98.121.66.919.28.36.97.26.51MB58.2151.755.5143.264.5172.454.9153.666.555.257.551.8

10MB3.312.42.811.52.93.42.36.14.42.44.22.410MB267.1994.4222.3923.7234.5271.5181.2484.4351.7192.5334.9193.9

100MB0.61.70.61.60.30.20.40.50.90.30.90.3100MB482.61391.3450.71312.6219.4180.7335.2379.9742.4212.1742.7220.6

1GB0.10.20.10.10.00.00.00.00.10.00.10.01GB457.91295.5439.11081.0170.2172.3305.8389.2841.6256.1835.9244.2

1exec

# of TasksData Size# of ExecutorsExp DescriptionLocalitySuccessful TasksFailed TasksFailed NotificationsTime (ms)Throughput (tasks/sec)Data Size (MB)Time (sec)Throughput (Mb/s)TP STDEVTP AVERLocal Cache HitGlobal Cache HitTimeTime per Call

10241B1data_gpfs_cache_read11024009229711.090.00000192.2970.0000887569090.1337890625

10241KB1data_gpfs_cache_read11024008509012.030.00185.090.0962745328083.095703125



5121MB1data_gpfs_cache_read151200703807.27170.3858.19835180450137.4609375



81GB1data_gpfs_cache_read18001397660.061000139.766457.9082180216017470.75

10241B1data_gpfs_cache_read41024005425318.870.00000154.2530.0001509963052.9814453125






64100MB1data_gpfs_cache_read46400368001.7410036.81391.30434782610575

81GB1data_gpfs_cache_read4800494000.16100049.41295.546558704406175

10241B1data_gpfs_cache_read11024009309611.000.00000193.0960.00008799521.043683038310.99939847040093096.000000000490.9140625

10241KB1data_gpfs_cache_read11024008644211.850.00186.4420.09476874671.156565862111.84609333430086442.000000000184.416015625


1024100KB1data_gpfs_cache_read110240010010010.230.1100.18.18381618380.821868904910.22977022980010010097.75390625

5121MB1data_gpfs_cache_read151200737446.94173.74455.54350184420.63885001736.94293773050073744.0000000001144.03125


64100MB1data_gpfs_cache_read164001135900.56100113.59450.74390351260.49334007410.5634298794001135901774.84375

81GB1data_gpfs_cache_read18001457460.051000145.746439.12011307340.2273662250.05489001410014574618218.25

10241B1data_gpfs_cache_read41024005446918.800.00000154.4690.00015039751.361577797518.79968422411154469.000000000153.1923828125







81GB1data_gpfs_cache_read4800592070.14100059.2071080.95326566120.35143427820.13511915821159207.00000000027400.875

10241B1data_gpfs_cache_read_write11024008166712.540.00000181.6670.0001003098079.7529296875

10241KB1data_gpfs_cache_read_write11024001819615.630.001181.9610.04502063630177.6962890625



5121MB1data_gpfs_cache_read_write151200635068.06163.50664.49784272350124.03515625



81GB1data_gpfs_cache_read_write18003760870.021000376.087170.1733907314047010.875

10241B1data_gpfs_cache_read_write41024005064920.220.00000150.6490.0001617406049.4619140625







81GB1data_gpfs_cache_read_write48003713670.021000371.367172.336260357046420.875

10241B1data_gpfs_cache_read_write11024009686410.570.00000196.8640.00008457221.488803989210.571522960096864.000000000394.59375

10241KB1data_gpfs_cache_read_write11024008413012.170.00184.130.0973731131.156437097812.17163912990084130.000000000682.158203125



5121MB1data_gpfs_cache_read_write151200745696.87174.56954.92899194040.66736988286.86612399250074569.0000000001145.642578125

25610MB1data_gpfs_cache_read_write1256001130052.2710113.005181.23091898590.62404944022.265366440700113006441.42578125


81GB1data_gpfs_cache_read_write18002092830.041000209.283305.80601386640.18911836050.03822575170020928326160.375

10241B1data_gpfs_cache_read_write41024004443323.050.00000144.4330.00018436752.265195985723.04593432811144433.000000000243.3916015625


102410KB1data_gpfs_cache_read_write41024004108224.930.0141.0821.99406065921.44992763424.9257582396114108240.119140625






10241B1data_gpfs_read11024008230412.440.00000182.3040.0000995334080.375

10241KB1data_gpfs_read11024007813713.110.00178.1370.1048414963076.3056640625

102410KB1data_gpfs_read11024007824113.090.0178.2411.0470213827076.4072265625

1024100KB1data_gpfs_read11024008858111.560.188.5819.2480328739086.5048828125

10241MB1data_gpfs_read11024001232748.311123.27466.45359118710120.384765625

51210MB1data_gpfs_read1512001164554.4010116.455351.72384182730227.451171875

128100MB1data_gpfs_read1128001379280.93100137.928742.416333159301077.5625

161GB1data_gpfs_read116001520950.111000152.095841.579276110309505.9375

10241B1data_gpfs_read_write11024008265512.390.00000182.6550.0000991108080.7177734375

10241KB1data_gpfs_read_write11024009692110.570.00196.9210.0845224461094.6494140625



5121MB1data_gpfs_read_write151200742386.90174.23855.17390015890144.99609375



81GB1data_gpfs_read_write18002498900.031000249.89256.1126895834031236.25

2561B1data_gpfs_wrapper_read1256002524310.140.00000125.2430.0000811314098.60546875

2561KB1data_gpfs_wrapper_read1256002353110.880.00123.5310.0870341252091.91796875



2561MB1data_gpfs_wrapper_read125600356007.19135.657.52808988760139.0625



81GB1data_gpfs_wrapper_read1800765610.10100076.561835.93474484409570.125

2561B1data_gpfs_wrapper_read_write125600285928.950.00000128.5920.00007162840111.6875

2561KB1data_gpfs_wrapper_read_write125600268979.520.00126.8970.07614232070105.06640625



2561MB1data_gpfs_wrapper_read_write125600395416.47139.54151.79434005210154.45703125



81GB1data_gpfs_wrapper_read_write18002620500.031000262.05244.2282007251032756.25

gpfs cache data-aware scheduler

# of TasksData Size# of ExecutorsExp DescriptionLocalitySuccessful TasksFailed TasksFailed NotificationsTime (ms)Throughput (tasks/sec)Data Size (MB)Time (sec)Throughput (Mb/s)Overall Throughput (tasks/sec)c)Overall Throughput Standard DeviationOverall Throughput Standard Deviation Number of SamplesCache SizeCache hits (local)Cache hits (global)Cache hits/miss ratio (local)Cache hits/miss ratio (global)

10241B1data_gpfs_cache_read11024009309611.000.00000193.0960.000087995210.99939847041.043683038392108970000

10241KB1data_gpfs_cache_read11024008644211.850.00186.4420.094768746711.84609333431.156565862186119210000



5121MB1data_gpfs_cache_read151200737446.94173.74455.54350184426.94293773050.638850017373144810000



81GB1data_gpfs_cache_read18001457460.051000145.746439.12011307340.05489001410.227366225127148090000









10241B1data_gpfs_cache_read_write11024009686410.570.00000196.8640.000084572210.571522961.48880398929620480000

10241KB1data_gpfs_cache_read_write11024008413012.170.00184.130.09737311312.17163912991.15643709788340960000



5121MB1data_gpfs_cache_read_write151200745696.87174.56954.92899194046.86612399250.66736988287492160000



81GB1data_gpfs_cache_read_write18002092830.041000209.283305.80601386640.03822575170.189118360518798720000

















20481B2data_gpfs_cache_read42048005360838.200.00000153.6080.00030562638.20325324582.716390203853217932033150.991
















20481B2data_gpfs_cache_read_write42048004442846.100.00000144.4280.000368776446.09705591074.10367299194440962037110.991







































321GB4data_gpfs_cache_read_write432003137880.101000313.788815.83744438920.10197968050.30632956922934412630.810.9
















321GB8data_gpfs_cache_read43200494300.65100049.435179.04106817720.65397506180.749504309484599992410.750.78

















































































































c_2exec_task

32.9218.744965951526.943.1548.063014.652853299826.4942.1246.0970559107

32.680677230419.556913674627.5243.3648.2730.061502781614.569461043826.942.7447.4326609074

31.650362402819.012253991827.1143.6448.9628.9815.080113100826.628526849642.556728451548.474520107

26.656427544116.479979423.6543.084823.444833665313.89793702522.817670324841.897669851346.8392644772

18.455272098113.065224048216.463441.3615.107924283311.030203800214.94148889632.59900674939.1841732675

9.08171772187.18778077275.5036601496.561747065220.663.35475923713.26114649684.94958576223.35625462968.46

1.73041955911.61239528880.98766194180.6124519132.490.44051195750.41791551580.59102746430.34437392550.9129293621

0.19711716150.19565642730.0726806910.06638150280.20984156960.05169277690.05726454430.05313743330.03306850970.0843699411

&A

Page &P

Read GPFS

Read Wrapper GPFS




Read+Write GPFS





Data Size


c_2exec_io

0.000263360.00014995970.00021520.00034520.000384480.000240.00011722280.000211920.000336960.00036877640.00079938680.00075357530.07983394540.0431406385

0.26144541780.15645530940.220160.346880.386160.24049202230.11655568840.21520.341920.37946128730.79756679520.736165745622.193617115717.394934595

2.53202899221.52098031932.16883.49123.91682.31841.20640904812.1302821483.40453827613.87796160867.64852537736.97065899332.196164792829.5080999734

21.325142035313.1839835218.9234.46438.418.755866932311.1183496218.254136259833.51813588137.471411581761.516712993657.6774802413157.8262154491133.9311444724

147.6421767849104.5217923854131.68272330.88120.863394266788.241630402119.5319111682260.7920539921313.4733861401251.3523273605186.4188478932581.5863039087389.6537769101

726.5374177413575.0224618149440.2928119189524.93976521251652.8268.3807389643260.8917197452395.9668609766268.5003703679676.8576.861558394242.5279033361674.2510327745682.0228240952

1384.33564731891289.916231026790.1295534688489.96153036421992352.4095659925334.3324126132472.821971446275.4991404104730.3434896725884.7541248517371.61928741892057.0643473425740.5110964902

1576.9372921031565.2514185091581.4455280934531.0520223541678.7325569195413.5422152293458.1163543959425.099466633264.5480774795674.9595287939

From the heroic to the logistical Non-traditional applications for parallel computers

Documents

Transcript of From the heroic to the logistical Non-traditional applications for parallel computers