Status of IU’s E10000 and recent activities with Sun

Sun NDA VisitJune 14 2000Craig Stewart, Ph.D.

stewart@iu.edu

Please cite as: Stewart, C.A. Status of IU’s E10000 and recent activities with Sun. Presented at Indiana University Wrubel Computing Center, 14 June 2000, Bloomington, IN. Available from http://hdl.handle.net/2022/14009.

IU's Goal

“To be a leader in absolute termsin information technology”

- IU President Myles Brand, 1996

IU in a nutshell

• Founded in 1820• $2B Annual Budget• 8 campuses• >90,000 students• 3,900 faculty• 878 degree programs; >1,000 majors; > 60

programs ranked within top 20 of their type nationally

• University highly regarded as research and teaching institution

IT@IU in a nutshell

• Academic programs in IT through computer science, library and information sciences, engineering and technology, and most notably through new School of Informatics

• CIO and Vice President for Information Technology Michael A. McRobbie– Office of the Vice President for Information Technology and

University Information Technology Services (UITS): ~$70M annual budget

– Technology services offered university-wide– UITS comprises ~500 FTE staff, organized into four technology

divisions(Teaching & Learning Information Technology,Telecommunications, University Information Systems, Research and Academic Computing) and crosscutting units (e.g. finance, planning, HR)

IU IT Strategic Plan

• 10 recommendations, 68 Actions covering all campuses and all IT areas

• Completed in May 1998, approved and implementation commenced January 1999

• Total cost of implementation: $205M over 5 years

• A unique charter for Information Technology at a large university that sets the strategic course for the next five years

Two new initiatives

• IPCRES: the Indiana Pervasive Computing Research Initiative – IPCRES Laboratories (3 at IUPUI; 3 at IUB)– Established by $30M grant from Lilly Endowment

• School of Informatics– One of the first such schools at a major University – The School's mission is to educate students broadly in the

technical,psychological and social aspects of information technology

– Includes the already-successful New Media program– Offers B.S. in Informatics, graduate programs in Chemical

Informatics, Bioinformatics, HCI, Health Informatics

Other IT highlights

• Network operations center for Abilene Internet2 network

• Lead US Institution on TransPAC High-Speed connection to APAN

• Major reengineering of University Information Systems underway

• Major Distributed Education Initiative underway• Recognized leader in use of IT in education• Recognized leader in support (IU leveraged support

model).

Advanced Computation Accomplishments

• Three HPC systems: – IBM RS/6000 SP - 139 processors, 147 GFLOPS

(largest supercomputer in Indiana) – 64-processor Sun E10000– 64-processor Parallel PC Cluster (Compaq)

• Prominently involved in Grid projects: CCAT, CCA Forum, Globus, iGrid, GOIN99

• Expanded support for High Performance Computing

Sun E10000

• Purchase negotiated as competitive upgrade to IU’s SGI Origin2000, and part of a purchase that also included equipment to completely refurbish IU’s messaging infrastructure

• Announced by Sun CEO Scott McNealy at IU Business Conference on 2/23/2000, simultaneous with announcement of Sun’s partnership with IU as a Sun Center of Excellence.

Sun E10000 timeline

• 3/27 Hardware arrives• 3/30 System turned over to IU• 3/30-4/16 Software installation & testing• 4/17 first availability for users (in hardy user

mode)• 4/24-4/27 Training by Sun• 5/24-5/25 Training by KAI representatives• 6/15 our 64P O2000 to be decommissioned

Software summary

• HPC 3.1, Workshop 6, PBS 2.3 beta• KAI suite: compilers (currently

recommended as default), KAP/Pro Toolset

• IMSL Fortran & libraries, LAPACK, NAG• Pallas tools• Radiance• The usual suite of GNU Unix tools

Performance comparisons

Performance comparisons, cont'd.

Successes with E10000• System stood up in good order and extremely reliable

(only 2 hardware failures thus far)• Excellent performance for some large codes

managed by sophisticated users (e.g. does some hard problems well)

• Good response by Sun in regards to problem with PBS andSun MPI libraries (although significant issues remain). Papakhian, Burks (IU) and Duncan (Sun) plan a paper for SuPerG.

• IU’s E10000 is 447th on current Top500 list

Problems to date

• Compilers (OpenMP, MPI, general robustness)

• 64-bit performance questions

• Performance obtained by naïve users

• Lack of control of /tmp’s use of swap

• DFS clients for Solaris 7 just now available

Concerns for future

• Sun security notification policy (we need to know about exposures when Sun knows, not when a fix is available)

• Effects of purchase of KAI by Intel (in our view, this places some added pressure on Sun to increase the rate of improvement in Sun compilers)

• Fixes for current software issues • Availability of DFS clients • MPI/Compiler issues • PBS support • Maintenance

Detail on Software issues

• Most of what we need has been promised in previous nondisclosures or other communications. Current issues include:– /tmp’s use of swap prevents us from using PFS – Need to get current versions in synch

• We cannot implement a new OS version (e.g. Solaris 8) until a DFS client is available for it.

• Compilers– OpenMP and Interval arithmetic for Fortran & C– Better performance, robustness, responsiveness to bugs

• PBS– MPI/PBS issues not yet completely resolved– PBS support should be a high priority; Sun seems to be moving in

the right direction on this

Self-Maintenance Issues

• IU has had successful self-maintenance agreements with three different vendors: Intel, IBM, SGI.

• Sun hpc.com model for ‘capacity on demand’ may contain the key elements of the required business model.

• At Sun’s HPC Government/Higher Ed Summit several Sun leaders emphasized their understanding that the research HPC market is different than the commercial market.

• IU’s success with self-maintenance makes it a good test case for Sun.

• This is IU’s #1 priority issue in regards to HPC as we move forward in our relationship with Sun.

Sun/IU relationship history & ongoing events

• IU site licenses: Solaris, related products, Star Office(obtained prior to Star Division’s acquisition by Sun)

• IU participation in Sun HPC Consortium

• Purchase of E3500 as Web data publishing/exchange server

• Flybase (http://flybase.bio.indiana.edu/)

Sun/IU current & pending activities

• Sarpa visit -- two proposals pending – Leake: Task-Based, Proactive Retrieval and Communication

(IU, NW, Sun)– Meglicki: Remote HPSS mover (IU, NCSA, Sun)

• SunRay Pilot• Key labs programs -- UITS Student technology

centers to be refurbished with Sun WSs, 7 other IU orders

• AEG application: WeBWorKs *Just approved*• Finalization of Sun TREC:HPC-Grid CE agreement

(Cheminformatics?) pending

In Summary

• So far, so good. • IU/Sun partnership in HPC area off to an

excellent start• IU/Sun partnership is being broadened and

deepened successfully– Relationship in HPC is strengthening – Building researcher-to-researcher relationships– IT relationship being broadened in several

directions,particularly in regards to teaching, learning, and student use

References

IU IT Strategic Plan:http://www.indiana.edu/~ovpit/strategic/

IPCRES:http://www.indiana.edu/~ovpit/ipcres/

IU University Information Technology Services:http://www.indiana.edu/~uits/

• Except where otherwise noted, the contents of this presentation are © by the Trustees of Indiana University. This content is released under the Creative Commons Attribution 3.0 Unported license (http://creativecommons.org/licenses/by/3.0/). This license includes the following terms: You are free to share – to copy, distribute and transmit the work and to remix – to adapt the work under the following conditions: attribution – you must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). For any reuse or distribution, you must make clear to others the license terms of this work.