Grid MP at ISIS
description
Transcript of Grid MP at ISIS
![Page 1: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/1.jpg)
Grid MP at ISIS
Tom Griffin, ISIS Facility
![Page 2: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/2.jpg)
Introduction
• About ISIS
• Why Grid MP?
• About Grid MP
• Examples
• The future
![Page 3: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/3.jpg)
Introduction
• ISIS - World's brightest pulsed neutron and muon
source• Running since 1984; 1600 user base; ~200 staff• Neutrons and muons for condensed matter science
• Based at the Rutherford Appleton Laboratory
• Driven by scientific output, not computer science
![Page 5: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/5.jpg)
Why does ISIS need faster processing?• Many compute intensive programs and problems
• Many applications which could be compute
intensive
• Large range of application domains
• Simulations
• Analysing results
• Designing new instruments
• Visualising ideas
![Page 6: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/6.jpg)
Why Grid MP?
• Neutron science not computer science
• Run by data analysis group
• Long interest in distributed computing
• Sun Workstations
• Condor
• Entropia DC Grid
• United Devices
![Page 7: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/7.jpg)
Why Grid MP?
• ‘Out of box’ solution
• United Devices Grid MP is the ‘market leader’
• Customers include:
• Johnson & Johnson
• GlaxoSmithKline
• Novartis
• National Physical Laboratory
![Page 8: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/8.jpg)
Why Grid MP?
• Set up and running real applications in under 3
hours
• Excellent service level
• Get what you pay for!
• Queries and problems always answered promptly
• Active development
![Page 9: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/9.jpg)
The United Devices System• Server hardware
• We use two, dual Xeon servers + 280 client licenses• Could (will) easily cope with more clients
• Software• Servers run RedHat Linux Advanced Server / DB2• Clients available for Windows, Linux, SPARCs and Macs
•Programming• MGSI – Web Services interface – XML, SOAP• Accessed with C++ and Java classes etc
• Management Console• Web browser based• Can manage services, jobs, devices etc
![Page 10: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/10.jpg)
Installing and Deploying the System• Servers
• Complete set up in under 3 hours
• Virtually self maintaining
• Clients• Windows only so far
• MSI Installer
• approx 20 seconds
• SMS
• MP Agent User
• Install to other OSs looks straightforward
![Page 11: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/11.jpg)
Grid MP Mode of operation• Client – Server
• Submitting PC breaks up Job
• Uploads data to server
• Programs reside on the server
• Server distributes Workunits to clients
• Clients return results to server
• User downloads results from server
![Page 12: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/12.jpg)
Getting an Application Running
• Don’t touch executable code
• Wrap and upload executable
• Write ‘Application Service’
• Typically only a few hours work
![Page 13: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/13.jpg)
Some Features of the Grid MP System• Device to Job matching
• Jobs have requirements – CPU, RAM, free disk space etc
• Programs have requirements
• Devices have preferences
• choose what programs they want to run
• Administrator can choose which users may submit to which
machines
![Page 14: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/14.jpg)
Some Features of the Grid MP System• Timing options
• When to execute
• When to communicate
![Page 15: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/15.jpg)
Some Features of the Grid MP System• Good access/privileges control
• Very powerful
• Read, update, create, delete
• Can hook into Active Directory, Kerberos etc
• Built in basic reporting
• Job / Program CPU hours
• Performance stats
![Page 16: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/16.jpg)
What do we use it for?
• 16 Applications running so far:
• Several molecular dynamics apps
• several neutron instrument simulation packages
• crystal structure solution
• maximum entropy calculations
• POVRay image rendering
![Page 17: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/17.jpg)
What performance do we get?
• Roughly linear for current applications
• Running on 100 machines is 100 times faster
• Average CPU 2.03Ghz
• High percentage of AMD Athlons
![Page 18: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/18.jpg)
Web Interface - Monitoring Jobs
![Page 19: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/19.jpg)
![Page 20: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/20.jpg)
![Page 21: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/21.jpg)
![Page 22: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/22.jpg)
Problems Encountered
• Mercifully few
• Only once the Grid’s fault
• Memory thresholding
• Odd program interactions
• Fortran and OpenGL
![Page 23: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/23.jpg)
• Easiest thing to blame
• Too abstract for some users (no big box)• Stealing my cycles
• Expansion leads to political problems
‘Social’ Issues
![Page 24: Grid MP at ISIS](https://reader036.fdocuments.in/reader036/viewer/2022081506/56814b63550346895db8523a/html5/thumbnails/24.jpg)
• Expansion• Applied for 400 licenses (£60k)
• Change in licensing model
The Future
• Bottom line• Unlimited licenses
• Total $250K
• No technical issues
• Plenty of political issues
• Single sign-on