Profile Data Mining with PerfExplorer
-
Upload
nguyenkhue -
Category
Documents
-
view
224 -
download
1
Transcript of Profile Data Mining with PerfExplorer
![Page 1: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/1.jpg)
Profile Data Mining with PerfExplorer
Sameer Shende Performance Research Lab, University of Oregon
http://TAU.uoregon.edu
![Page 2: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/2.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France
TAU Analysis
![Page 3: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/3.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 3
TAUdb: Performance Data Mgmt. Framework
![Page 4: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/4.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 4
Using TAUdb
• Configure TAUdb (Done by each user) % taudb_configure --create-default
• Choose derby, PostgreSQL, MySQL, Oracle or DB2 • Hostname • Username • Password • Say yes to downloading required drivers (we are not allowed to distribute these) • Stores parameters in your ~/.ParaProf/taudb.cfg file
• Configure PerfExplorer (Done by each user) % perfexplorer_configure
• Execute PerfExplorer % perfexplorer
![Page 5: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/5.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 5
Using PerfExplorer
% wget http://tau.uoregon.edu/data.tgz (Contains CUBE profiles from Score-P) % taudb_configure --create-default (Chooses derby, blank user/passwd, yes to save passwd, defaults) % perfexplorer_configure (Yes to load schema, defaults) % paraprof (load each trial: DB -> Add Trial -> Type (Paraprof Packed Profile) -> OK) OR use
taudb_loadtrial –a “app” –x “experiment” –n “name” file.ppk Then, % tar zxf $TAU/data.tgz; cd data/tau; % taudb_loadtrial –a BT_MZ –x “Class_B” bt-mz_B.*.ppk % perfexplorer (Select experiment, Menu: Charts -> Speedup)
![Page 6: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/6.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 6
TAUdb and the TAU Portal
• Development of the TAU portal – Common repository for collaborative data sharing – Profile uploading, downloading, user management – Paraprof, PerfExplorer can be launched from the portal using Java
Web Start (no TAU installation required)
• Portal URL http://tau.nic.uoregon.edu
![Page 7: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/7.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 7
Performance Data Mining (PerfExplorer)
• Performance knowledge discovery framework – Data mining analysis applied to parallel performance data
• comparative, clustering, correlation, dimension reduction, … – Use the existing TAU infrastructure
• TAU performance profiles, taudb – Client-server based system architecture
• Technology integration – Java API and toolkit for portability – taudb – R-project/Omegahat, Octave/Matlab statistical analysis – WEKA data mining package – JFreeChart for visualization, vector output (EPS, SVG)
![Page 8: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/8.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 8
PerfExplorer - Cluster Analysis
• Performance data represented as vectors - each dimension is the cumulative time for an event
• k-means: k random centers are selected and instances are grouped with the "closest" (Euclidean) center
• New centers are calculated and the process repeated until stabilization or max iterations
• Dimension reduction necessary for meaningful results • Virtual topology, summaries constructed
![Page 9: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/9.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 9
PerfExplorer - Cluster Analysis (sPPM)
![Page 10: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/10.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 10
PerfExplorer - Correlation Analysis (Flash)
• Describes strength and direction of a linear relationship between two variables (events) in the data
![Page 11: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/11.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 11
PerfExplorer - Correlation Analysis (Flash)
• -0.995 indicates strong, negative relationship
• As CALC_CUT_ BLOCK_CONTRIBUTIONS() increases in execution time, MPI_Barrier() decreases
![Page 12: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/12.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 12
PerfExplorer - Comparative Analysis
• Relative speedup, efficiency – total runtime, by event, one event, by phase
• Breakdown of total runtime • Group fraction of total runtime • Correlating events to total runtime • Timesteps per second
![Page 13: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/13.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 13
PerfExplorer - Interface
Select experiments and trials of interest
Data organized in application, experiment, trial structure (will allow arbitrary in future)
Experiment metadata
![Page 14: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/14.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 14
PerfExplorer - Interface
Select analysis
![Page 15: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/15.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 15
PerfExplorer - Relative Efficiency Plots
![Page 16: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/16.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 16
PerfExplorer - Relative Efficiency by Routine
![Page 17: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/17.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 17
PerfExplorer - Relative Speedup
![Page 18: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/18.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 18
PerfExplorer - Timesteps Per Second
![Page 19: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/19.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 19
Usage Scenarios: Evaluate Scalability
• Goal: How does my application scale? What bottlenecks occur at what core counts? • Load profiles in taudb database and examine with PerfExplorer
![Page 20: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/20.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 20
Usage Scenarios: Evaluate Scalability
![Page 21: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/21.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 21
PerfExplorer
![Page 22: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/22.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 22
PerfExplorer
![Page 23: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/23.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France 23
Performance Regression Testing
![Page 24: Profile Data Mining with PerfExplorer](https://reader030.fdocuments.in/reader030/viewer/2022020203/58a2d1121a28ab90398bc2c2/html5/thumbnails/24.jpg)
VI-HPS TW15: VI-HPS Tuning Workshop, Saclay, France
• U.S. Department of Energy (DOE) – Office of Science
– ASC/NNSA, Tri-labs (LLNL,LANL, SNL)
• U.S. Department of Defense (DoD) – HPC Modernization Office (HPCMO)
• NSF Software Development for Cyberinfrastructure (SDCI)
• Juelich Supercomputing Center, NIC • Argonne National Laboratory • Technical University Dresden • ParaTools, Inc. • NVIDIA
Support Acknowledgments