SCD Computing Center Expansion

Aaron Andersen

SCD User Forum

5/18/2005

Phase 1: SCD Internal Analysis

1/1/2003 1/1/20045/5/2002 7/1/2004

SCD Executive RetreatInfrastructure Impacts

Computing In AtmosphericSciences

Discussions with other Peer Centers

Analysis of Computing Futures& Infrastructure Needs

Initial DiscussionsLoft & Andersen

SCD Strategic Planning Effort

Growing Model ComplexityGrowing Model Complexity

Chips: Faster, Cheaper but Hotter

• Power problem: • P = C V2 f

– Frequency (clock) increase driving Moore’s Law

– Decreasing voltage increases gate delay (hurts performance)

– Capacitance increases as gate count increases.

• Chip power consumption will increase 30% over 5 years.

• Processor costs decreasing at 29%/year.

Mesa Lab Power Limit

The Problem with the Mesa Lab Facility

• NCAR’s machine room was built during the era of low power/high cost computing systems (1976).– 13000 sq ft– 1.2 MW computer power limitation– 80 W/sq ft cooling design

• We live in an era in which high power/low cost computing systems are available.

• It is not cheap to modify the power infrastructure of Mesa Lab Computing Facility, nor does it make sense without expanding the machine room floor.

• Site situated on bedrock.• If we do nothing, the Mesa Lab power and design

limitations will become a serious constraint on NCAR’s scientific computing capacity. This is true for both future vector and microprocessor systems.

Recap of Phase 1 Conclusions

• Three Driving Issues for a Facility Expansion– Geoscience: Exponential demand for supercomputing– Technology: CMOS is getting hotter and cheaper

– Engineering: Mesa Lab facility design is obsolescent • Recommended responses on three timescales

– Short term (1 year): • Mesa Lab risk mitigation (RMH, Inc.)

– Medium term (2 years):• Lease options (Staubach Real Estate)• Colocation (Inflow, Inc., Level 3)

– Long term (3-5 years):• Refurbishment of existing structures (RNL Design, Inc.)• New construction (Uptime Institute, Inc.)

Phase 2: Analysis of Facility Options

8/1/2004 9/1/2004 10/1/20047/1/2004 10/22/2004

Feasibility StudyCenter Green 3

Final Report on CG3$26 M and No Expansion

UCAR Board of TrusteesUpdate

Uptime Institute SeminarUCAR Executive Management

Planning and Justifyinga Data Center Project

Evaluation of Leased SpaceOptions

NCAR ExecutiveUpdate on Data Center Options

Phase 2 Conclusions

• Center Green 3 was not as capable as hoped– $26M to create a CG3 data center smaller than ML facility.– No room for expansion

• Existing Computer Room Spaces on Market– Most not adequate for anticipated power densities or loads.– One that was acceptable was purchased by another entity.

• Strategy Moving Forward– Act to maximize life and capabilities of Mesa Lab facility,– Move forward on building a data center expansion.

Preservation of theMesa Lab Facility

• Perform a Risk Assessment– Ensure capability of reliably housing

~ 1 – 1.2 MW of computing– Evaluate the capabilities of infrastructure

equipment

Phase 3: Facility Design Process

1/1/200510/22/2004 6/1/2005

Conceptual DesignRFP for Data

Center Expansion

Risk AssessmentReport

RiskAssessment for

Mesa Lab

Statement of Work for PeerReview & Conceptual Design

Develop Conceptual Design

Peer Review RFPPeer Review Award

Conceptual Design Award

Preliminary SiteSelection

Update NCAR Execon Risk Assessment

Conceptual Design& Cost Estimate Due

Peer Review ofConceptual Design

Today

Develop ReportRecomendations

Design Concepts

Land for 20 Years

Building Shell for 10

Outfit for 5Strategy preservesinitial capital investmentas computing hardwareevolves rapidly.

Majority of cost in thistype of facility in outfittingmechanical and electricalsystems.

Facility Design Concept

20,000 sq. ft.4 MW

20,000 sq. ft.4 MW

20,000 sq. ft.

M/E Space

Computers

Staff/Support

20,000 sq. ft.4 MW

20,000 sq. ft.4 MW

20,000 sq. ft.

Initial ModuleModule 2 Module 3

20,000 sq. ft.4MW

20,000 sq. ft.4 MW

20,000 sq. ft.

Expand laterally

Architectural Detail

Office Space

Network Operation Center

Data Center

Mechanical/Electrical

Architectural View

Phase 2 Addition

Phase 3Addition

Next Steps

• Conceptual Design & Cost Estimate is due as I present this

• Developing full report on findings

• Develop funding strategies/options – UCAR Management– Board of Trustees– NSF