Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel...
-
Upload
charleen-freeman -
Category
Documents
-
view
213 -
download
0
Transcript of Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel...
![Page 1: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/1.jpg)
Madagascar Software Package Technology for
Reproducible Computational Experiments
Sergey FomelBureau of Economic GeologyJackson School of GeosciencesThe University of Texas at Austin
http://rsf.sf.net/
Scientific Software Day 2007
![Page 2: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/2.jpg)
Outline
Past Seismic Imaging
MIT, Industry, Stanford, …
Present Test-driven development
Reproducible computational experiments
Future Asking for help
Automatic testing, parallel processing, graphics, …
![Page 3: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/3.jpg)
Seismic Imaging Computations 45 clusters on the top 500 list
many missing Europe’s most powerful supercomputer
(CGG/IBM) Data size
Gb (2-D; 1980x) Tb (3-D; 1990x) ?? (4-D, 4-C, multi-azimuth, …; 200x)
![Page 4: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/4.jpg)
History of Seismic Imaging
Norbert Wiener (© RLE/MIT) Enders Robinson
Jon Claerbout
![Page 5: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/5.jpg)
SEPlib Software Innovations Universal data file format (RSF)
Data separated from text headers Conceptual regularly gridded N-dimensional
hypercubes
If you feel an urge to design a complex binary file format, or a complex binary application protocol, it is generally wise to lie down until the feeling passes. -- Eric S. Raymond, TAUP
Universal graphics format (Vplot) Reproducible research 30+ years legacy
R. Clayton, J. Claerbout, D. Hale, S. Levin, J. Dellinger, S. Cole, D. Nichols, M. Karrenbach, B. Biondi, R. Clapp, …
Data
n1=1000 in=“/path/data.rsf@”n2=500 n3=100 d1=0.001 d2=0.1 o2=1
![Page 6: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/6.jpg)
“Madagascar” Innovations
Open source (GPL) Open community (SourceForge) Open science
Command-line modules Data processing scripts Documentation
![Page 7: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/7.jpg)
Monday Get an idea
Tuesday Implement it
Wednesday Test it
ThursdayCommunicate it
Friday Apply it
Technology Transfer
I
2
3
![Page 8: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/8.jpg)
Three Levels of Technology
Implement it (Tuesday) Command-line modules
C (C++, Fortran-77, Fortran-90, Matlab, Python)
400 modules Test it (Wednesday)
Data processing flows Python/SCons 4,000 tests
Communicate it (Thursday) Papers and reports
LaTeX + Python/SCons 60 papers
![Page 9: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/9.jpg)
Reproducible Computations
Test Driven Development Document computational experiments Old experiments become regression
tests
Within the world of science, computation is now rightly seen as a third vertex of a triangle complementing experiment and theory. However, as it is now often practiced, one can make a good case that computing is the last refuge of the scientific scoundrel. –R. LeVeque
![Page 10: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/10.jpg)
Asking for Help
Open problems Automatic reproducibility testing Parallelization Interfaces Graphics …
Open opportunities Summer students Consultants
![Page 11: Madagascar Software Package Technology for Reproducible Computational Experiments Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e905503460f94b958c1/html5/thumbnails/11.jpg)
Information
http://rsf.sourceforge.net/ Wiki, blog, e-mail lists, forums
RSF/Madagascar School in Austin April 20, 2007 Bureau of Economic Geology
http://rsf.sourceforge.net/RSF_Austin_School_2007