The ‘Otago Experience’: Insights Gained from the Otago Millennium Graduate Project
091020 E Research Otago
-
Upload
nick-jones -
Category
Technology
-
view
774 -
download
1
description
Transcript of 091020 E Research Otago
BeSTGRID: Developing NZ research infrastructure
www.bestgrid.orgNick JonesDirector, BeSTGRIDCo-Director eResearch, Centre for eResearchUniversity of [email protected]
... was a Pilot$2.5million: Sep 2006 – March 2008
“BeSTGRID aims to enhance e-Research capability in New Zealand by providing the skill base to help the various research disciplines engage with new eResearch services”
“BeSTGRID aims to kick start centralised infrastructure with some capital investment at key institutions”
BeSTGRID: intial three years
• Advanced Video Conferencing (Access GRID, EVO - Enabling Virtual Organizations, HD, H.323)
• Telemedicine – APAN 26
Collaboration Services
• Sakai VRE (Virtual Research Environment) – collaboration space (wikis, mailing lists, event calendars, file sharing)
• “VRE” development by UK e-Science group at Hartree Centre, Daresbury
• BeSTGRID IAM Federation• Pilot integration with Australian AAF• Auckland, Canterbury, Lincoln, Open• Wikis, SLCS (Grid Certificates)
Collaboration Services
Computation Grid Tools• Grid Computing interfaces
– Grisu GUI, Grisu CLI– Bioportal (Phylogenetics)
• x.509 Certificates– Grix certificate issuing tools– SLCS Shibboleth issued
certificates (Canterbury)
Bioinformatics
Steven Stones-Havas1 and Allen Rodrigo2
1Biomatters Ltd 2The Bioinformatics Institute (New Zealand)
• 30TB storage at each foundation BeSTGRID site• Auckland, Canterbury, Massey
• Local mirrors of research databases• NZ BioMirror• Ecology and Human Behaviour• Whole Genome Association Studies• NZ Social Sciences Data Service• Ocean Biogeographic Information System• Austronesian Basic Vocabulary and Bantu Language Database
• Data services (GridFTP, NFS, Samba, FTP)
Data Services
Successes: within NZ• First grid infrastructure in NZ, focused on BioInformatics, with Industry
collaboration with BioMatters and New Zealand Supercomputing Centre
• Implementation of first federated identity management framework in NZ– contribution to a multi-sector working group to further evolve identity
management across research and education (IMAGER, http://www.morst.govt.nz/current-work/science-infrastructure/imager/)
• Established the foundations of a shared infrastructure service delivery model across institutions
• Recognition from MoRST of need for eResearch support, with $4.2M in ‘08/’09 budget for eResearch infrastructure
What did we learn?• Change difficult within Institutions
– Medium to Long term timeframes required with researchers, ITS, SMT, research support units– “Final Report: A Workshop on Effective Approaches to Campus Research Computing Cyberinfrastructure” Klingenstein,
Morooney, Olshansky. 2006
• Collaboration seen as critical yet complex and expensive– Working with user communities essential to mutual understand of needs and opportunities, and developing capabilities– Need significant capability before international collaborations (and related learning) possible– New forms of organisation and support required– “Beyond Being There: A Blueprint for Advancing the Design, Development, and Evaluation of Virtual Organization”
Cummings, Finholt, Foster, Kesselman, Lawrence. 2008– “The Importance of Long-Term Science and Engineering Infrastructure for Digital Discovery” Wilkins-Diehr, Alameda,
Droegemeier, Gannon. 2008
• Government and Institutional Investment well below international levels– Australia NCRIS: $540M ++ over 5 years alongside State and Institutional investments– EU FP7 e-Infrastructure call + National Grid Initiatives + Identity + HPC + Networks in member states– US Office of Cyberinfrastructure, NSF, NIH, DoE all funding major Cyberinfrastructure initiatives
• Education programs lacking– Who is training the next generation?
BeSTGRID 2009 - 2010
MoRST GRID Computing Middleware Initiative
MoRST eResearch programme
Review:
In Vote RS&T 2008/09, funding was appropriated to develop capability within the Kiwi Advanced Research and Education Network user group to make effective use of KAREN:
• Advanced Video Conferencing Collaboration and Support Centre• Federated Identity and Access Management
• Towards a Federated Approach – strategy development• Technical Support and Resources
• Semantic Data and Public Access to Research• Grid Middleware Initiative• Hardship fund for remote site connection to KAREN
Advanced Video Collaboration Centre
• Advanced Video Collaboration Centre• Helpdesk 0800 666 889• http://avcc.karen.net.nz/• Event calendar• Quality Assurance for end points• Nathan Gardiner
NZ Identity Federation
• Towards a Federated Identity– https://nzfed.auckland.ac.nz/– Strategy consultation – Q3 – Q4 2009– Resources, Toolkit, and examplar services – 2010– Governance and Ownership discussion underway
– NZVCC IT, CRI IT, supported by MoRST– IMAGER – http://www.morst.govt.nz/current-
work/science-infrastructure/imager/– Matthew Cocker
Semantic public research data
• Public research data – semantic interoperability
• Integration of Geoscience related data holdings
• University of Auckland, Landcare Research
• Mark Gahegan, Robert Gibb
BeSTGRID 2009 – 2010
• MoRST Dec 2008 funding request for Middleware development for eResearch in NZ
$840k approved 24 Dec 2008, initiated April 2009
• Provides ongoing support for BeSTGRID– Widened from founders: Auckland, Canterbury & Massey– by active participation of new Universities and CRIs Victoria, Otago,
Lincoln, Landcare Research, AUT
• Aim: coordinate national grid middleware; use best-practice tools & services; focus on sustainability
BeSTGRID 2009 – 2010
Form collaborations with related Science Infrastructure programmes• SCENZ Grid• New Zealand Genomics Limited• Virtual Institute of Statistical Genetics and MapNet• Social Sciences Data Archive and Services• Auckland Cancer Society Research Centre
Support Science Gateways, applications and servicesinfrastructure supports science communities with own specialised needs
• Bio – Extension of scope from phylogenetics to other areas of Bioinformatics and Biomolecular sciences
• Geo – collaboration with Canadian geoscience infrastructure
Add in new Data and HPC sites and services:• Otago University• Victoria University Wellington• Landcare Research• Auckland University Technology
Massey, Albany
Auckland
Otago
Canterbury
AUT
Victoria
Landcare Research
CurrentPlanned
Lincoln
Identity provider
Virtual Applications
ComputationCluster
Storage
Computation Grid ResourcesResource name Location Architecture OS #CPU cores Freq Notes
Auckland Cluster The University of Auckland x86 Xeon Linux 80 (176) 2.8GHz Operational, since June 2008. Current Status
AUT Cluster AUT Linux 32 Connected end September 2009
BeSTGRIDPrototype Cluster University of Canterbury x86 Xeon Linux 4 3.0GHz
Prototype cluster, serving the community since June 2007.
BlueFern p575 University of Canterbury Power5+ AIX/Linux 128 1.9GHz 8 nodes, 16 CPU cores each.
BlueFern BlueGene University of Canterbury PowerPC Linux 4096 770MHz Available only together with a local account.
Maggie University of Otago x86 Pentium 4 Linux 10 3.0GHz No MPI available
Massey Cluster Massey University, Albany x86 Xeon Linux 160 Infiniband interconnect
NZ eResearch Infrastructure
Key Milestones• Winter Retreat 2009
– Geo, Bio, Grid strategy workshops – Auckland– 29 June – 1 July
• Spring Retreat 2009– Technical Working Group working session – Canterbury– 14 September
• Strategies and plans– Geo and Bio strategy drafts – near completion– Initial projects kicking off in parallel
• National workshop – planning underway for 2010– Coordinating with KAREN, Federated IAM, and others
• Grid Middleware strategy – Q3 2010
BeSTGRID Winter Retreat 2009
Geoscience WG Middleware WG
Bioscience themeLeadership
Dr. Mik Black, Biochemistry Department, University of Otago
The TeamMik Black, Anette Becher, Cris Print, Jack Flanagan, Mike Cooling, Mike
Eccles, Neil Gemmell, Phil Wilcox, Robert Hickson, George Slim, Mark Gahegan, Nick Jones, Richard Li
Research CommunitiesVirtual Institute of Statistical Genetics, MapNet, NZ Genomics Ltd, Centre for Reproduction and Genomics, Auckland Bioengineering Institute, Auckland
Cancer Society Research Centre, Bioinformatics Institute
Initial Goals – Genetics/Genomics• Storage and management of high-throughput data (e.g., microarray and sequence data).• Distributed analysis of large data sets (analysis via GenePattern).
GenePattern: a platform for integrative genomics
Clinical Research
PathologyMolecular Biology
Imaging
GenePattern provides access to a broad array of computational methods used to analyze genomic and proteomic data. It is designed to be easy for computational biologists to add their own analysis and visualizations, which enables access to new computational methods on a regular basis.
Slide adapted from Ted LiefeldThe Broad Institute of MIT and Harvard
Geoscience themeLeadership
Prof. Mark Gahegan, Centre for eResearch, University of Auckland
The TeamMark Gahegan, Kevin Buckley, Isabella Cawthorn, Robert Gibb, Niels Hoffmann, Femke Reitsma, Nick Jones, Richard Li, Mik Black
Research CommunitiesInvasive species modelling, surface and ground water modelling, ecosystem services modelling, carbon emissions, landcover and landuse databases, drill
core integration, taxonomy reconcilliation, nationally significant databases developmentInitial Goals• Invasive Species Modelling (R on the grid with workflows, mapping)• Surface and Ground Water modelling (Lidar processing and workflows, mapping)• Taxonomy Reconcilliation (Data and Collections Management)
WSData
Paul Grimwood & colleagues, GNS
User Interface: Portal
Robert Gibb, Niels Hoffmann, Landcare
Technology (future)
• Workflow– Kepler
• currently issue with proxies
– Triana– Taverna
• No spatial components
Robert Gibb, Niels Hoffmann, Landcare
Grid Middleware Planning
Technical Working Group
• Deliver on needs of Bio and Geo themes
• Meet monthly, to present and discuss proposals, progress reports, and services and applications deployed
Membership• Institutions formally
contracted to BeSTGRID + collaborators
Terms of Reference, Meeting notes, etc on BeSTGRID website
http://www.bestgrid.org/index.php/Grid_Technical_Working_Group
Summary
What do we have, where are we going?
What do we have?
• A good high-speed network with capacity for growth
• The foundations of a sophisticated infrastructure, some HPC and massive storage silos,
• Authentication, some grid middleware and basic services (but not nationally coordinated)
• Good links to e-research development communities: e.g. Australian National Collaborative Research Infrastructure Committee (NCRIS), Argonne (Globus), San Diego Supercomputer Center (SDSC), eScience UK, UK NGS, OMII-UK (Edinburgh, Manchester)
BeSTGRID 2009 – 2010
• MoRST Dec 2008 funding request for Middleware development for eResearch in NZ
$840k approved 24 Dec 2008, initiated April 2009
• Provides ongoing support for BeSTGRID– Widened from founders: Auckland, Canterbury & Massey– by active participation of new Universities and CRIs Victoria, Otago,
Lincoln, Landcare Research, AUT
• Aim: coordinate national grid middleware; use best-practice tools & services; focus on sustainability
BeSTGRID 2009 – 2010
Form collaborations with related Science Infrastructure programmes• SCENZ Grid• New Zealand Genomics Limited• Virtual Institute of Statistical Genetics and MapNet• Social Sciences Data Archive and Services• Auckland Cancer Society Research Centre
Support Science Gateways, applications and servicesinfrastructure supports science communities with own specialised needs
• Bio – Extension of scope from phylogenetics to other areas of Bioinformatics and Biomolecular sciences
• Geo – collaboration with Canadian geoscience infrastructure
Add in new Data and HPC sites and services:• Otago University• Victoria University Wellington• Landcare Research• Auckland University Technology
What next?Aligning with the NZ science community
Developing a RIAG bid for a long-term integrated landscape of eResearch including Federated IAM and HPC, as a National eResearch Infrastructure
What are the research needs, tools, applications, environments, computing capabilities that we will need in New Zealand, over the next 10 - 20 years?
Please get in touch if you would like to include your ideas and needs:
Thanks !!!
End
Questions, comments