© Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound,...
-
Upload
pamela-chandler -
Category
Documents
-
view
216 -
download
1
Transcript of © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound,...
![Page 1: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/1.jpg)
© Geodise Project, University of Southampton, 2003.
Geodise Middleware& Optimisation
Graeme Pound, Hakki Eres, Gang Xue
& Matthew Fairman
Summer 2003
![Page 2: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/2.jpg)
© Geodise Project, University of Southampton, 2003.
Computational Toolbox
• Access to Globus compute resources from Matlab
• Proxy certificate management, job submission and file transfer
• Complex grid-enabled workflows may be coordinated from Matlab
• Uses the Java CoG 1.1
![Page 3: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/3.jpg)
© Geodise Project, University of Southampton, 2003.
Computational Toolbox
gd_createproxy.m Creates a Globus proxy certificate for the user's credentials
gd_destroyproxy.m Destroys the local copy of the user's Globus proxy certificate
gd_certinfo.m Returns information about the user's certificate
gd_proxyinfo.m Returns information about the user's proxy certificate
gd_proxyquery.m Queries whether a valid proxy certificate exists
gd_jobsubmit.m Submits a compute job to a Globus GRAM job manager
gd_jobstatus.m Gets the status of a Globus GRAM job
gd_jobpoll.m Queries the status of a Globus GRAM job until complete
gd_jobkill.m Kills a Globus GRAM specified by job handle
gd_putfile.m Puts a remote file using GridFtp
gd_getfile.m Retrieves a remote file using GridFtp
gd_rmfile.m Deletes a remote file using GridFtp
gd_makedir.m Creates a remote directory using GridFtp
gd_rmdir.m Deletes a remote directory using GridFtp
![Page 4: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/4.jpg)
© Geodise Project, University of Southampton, 2003.
Jython Computational Toolbox
• Python interface to functionality of Matlab Computational Toolbox
• Jython is a 100% Java implementation of the Python scripting language
• Lightweight alternative to the Matlab Computational Toolbox
![Page 5: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/5.jpg)
© Geodise Project, University of Southampton, 2003.
Job Submission Service Client
• Matlab client to job submission web service • User detached from compute endpoint – Condor pool• Machines publish available resources:
– HasProEngineer = TRUE– ProEngineerVersion = "2001400"– ProEngineerPath = "C:\Program Files\proe2001\bin
• DIME file transfer over http• MATLAB Client Functions:
– grid_platform – Describes the platform requirements of the job– grid_submit – Submits the job to the web service, returns a job ID– grid_poll – Polls job ID– grid_status – Queries job status– grid_results – Retrieves the output files
![Page 6: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/6.jpg)
© Geodise Project, University of Southampton, 2003.
OptionsMatlab
• Matlab interface to the Options design exploration system
• State of the art design search and optimisation algorithms
• Objective and constraint functions exposed as Matlab functions
• Grid-enabled job brokers easily incorporated
![Page 7: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/7.jpg)
© Geodise Project, University of Southampton, 2003.
OptionsMatlab Parallel Architecture
• optjob.m allows evaluation of multiple design points
• Concurrent evaluation of optfun.m and optcon.m using Objective Function Service Client
• Single evaluation via optjob.m from the optfun and optcon subroutines
Matlab environment
OptionsMatlab.dll
optjob.m
optjob optfun optcon
optfun.moptcon.m
![Page 8: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/8.jpg)
© Geodise Project, University of Southampton, 2003.
Objective Function Client
• Grid-enabled objective function evaluation• Enables multiple concurrent evaluations• Matlab functions provided:
– gd_objsubmit - Transfers required files, submits job and returns handle
– gd_objvalue - Retrieves function evaluation, prints standard output of failed jobs and cleans up files
– gd_objcleanup - Removes remote files of specified jobs
• Multiple Matlab processes, one Matlab licence • Configuration file defines available objective function
servers
![Page 9: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/9.jpg)
© Geodise Project, University of Southampton, 2003.
Objective Function Client
Matlab
Geodisefile archive
Globus server
gd_archive.m
gd_objsubmit.m
gd_jobpoll.m
gd_objvalue.m
Matlab
objfun.m
1
2
3
4
5
![Page 10: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/10.jpg)
© Geodise Project, University of Southampton, 2003.
A Level 2 Grid Application
• The aim is to find enhanced vibration isolation performance for satellite designs
• Matlab is the scripting environment• Geodise toolkit is utilized• The GA toolbox for Matlab is used for
optimization• UK e-Science Grid resources used
– Cambridge e-Science Centre– CLRC e-Science Centre– London e-Science Centre– Oxford e-Science Centre– Southampton e-Science Centre– Southampton Information Systems Services
![Page 11: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/11.jpg)
© Geodise Project, University of Southampton, 2003.
Parallel Job Submission (11x11)
![Page 12: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/12.jpg)
© Geodise Project, University of Southampton, 2003.
Results (Sample points = 11x11)
![Page 13: © Geodise Project, University of Southampton, 2003. Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f285503460f94c40470/html5/thumbnails/13.jpg)
© Geodise Project, University of Southampton, 2003.
Conclusions
• Submitting compute jobs is the easy bit.• Complexity of grid computing includes:
– applying for authorisation at a numerous locations– managing the transfer of files– keeping track of heterogeneous compute resources
• Requirements for:– Grid enabled data management– Transparent Grid-enabled tools– Consistent computing environments– Reservation and co-allocation of resources– Grid debugging