Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20,...
-
Upload
arthur-glenn -
Category
Documents
-
view
214 -
download
0
Transcript of Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20,...
![Page 1: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/1.jpg)
Enhance CMAQ Performance to Meet Future Challenges: I/O AspectDavid WongAMAD, EPAOctober 20, 2009
![Page 2: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/2.jpg)
Motivation
Seeing degradation of performance as the number of processors increase
Need to use large number of processors is inevitable
Over due
![Page 3: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/3.jpg)
![Page 4: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/4.jpg)
Background info
CMAQ performs I/O through IOAPI_3 IOAPI_3 operates in serial mode PARIO (PARallel IO) was created over 10 years
ago, on top of IOAPI_3 to provide parallel I/O functionalities
Current I/O design: any processor can perform read but only I/O processor can perform write
![Page 5: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/5.jpg)
![Page 6: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/6.jpg)
![Page 7: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/7.jpg)
![Page 8: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/8.jpg)
![Page 9: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/9.jpg)
![Page 10: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/10.jpg)
![Page 11: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/11.jpg)
![Page 12: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/12.jpg)
![Page 13: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/13.jpg)
![Page 14: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/14.jpg)
![Page 15: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/15.jpg)
![Page 16: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/16.jpg)
Observation
• Read operation overhead• Write operation synchronization cost
![Page 17: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/17.jpg)
![Page 18: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/18.jpg)
![Page 19: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/19.jpg)
![Page 20: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/20.jpg)
![Page 21: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/21.jpg)
![Page 22: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/22.jpg)
![Page 23: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/23.jpg)
![Page 24: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/24.jpg)
![Page 25: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/25.jpg)
![Page 26: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/26.jpg)
![Page 27: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/27.jpg)
![Page 28: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/28.jpg)
Summary
CMAQ degradation comes from read operation as well as synchronization cost for write operation. This implies not “suitable” to run with large number of processors
AQF’s I/O design does show better performance DFIO approach takes care the degradation
problem
![Page 29: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/29.jpg)
Next step
Pick a better configuration to compare aqf and cmaq performance
Determine the actual I/O performance using the real model code
Pnetcdf Multicore architecture
![Page 30: Enhance CMAQ Performance to Meet Future Challenges: I/O Aspect David Wong AMAD, EPA October 20, 2009.](https://reader035.fdocuments.in/reader035/viewer/2022062906/5a4d1b387f8b9ab05999db34/html5/thumbnails/30.jpg)
THANK YOU !