Power considerations for the SKA

Peter Hall

ICRAR/Curtin

Co-Chair, SKA Power Investigation Task Force

AU PEP workshop, Perth, 24 Nov. 2011

Outline

• See conference paper for more technical discussion

• SKA background and power history

• Power Investigation Task Force (PITF)

– Topics and contributions

• Path to SKA power system in pre-construction era

– SKA Project Office and Industry

– Renewable energy: photovoltaic example

• SKA precursor contributions

Background

• SKA is now designing a system, selecting technologies and entering pre-construction)

– SKA1 (10% sensitivity) defined: sparse aperture arrays + dishes

– Advanced instrumentation package defined as possible path to SKA2

• Two candidate central sites: Murchison (WA) and Karoo (RSA)

– 2011, Q1 selection

• Power is a pivotal consideration in system design, and in site selection and development

– Availability and cost determines scientific capability of instrument

– Remote locations, high temperatures, advanced electronics, huge computing

demands, long facility lifetime, …

• Capital and operating dimensions to SKA design + site choice

• Key SKA considerations:

– Low-power telescope design

– Efficient environmental conditioning (passive cooling etc)

– Innovative energy delivery and costing arrangement, e.g.

• Buffer from world-parity energy pricing?

History: SKA power estimates

• Before 2001: blissful ignorance

• c. 2001: estimates of ~10 MW

– Mainly antennas and related systems

• c. 2003: ~20 MW

– Included computing etc.

• 2003-2004: the information explosion!

– (Very) wide fields-of-view to observed and processed

• 2005-2006: scaling from existing installations hundreds of MW panic, denial, …

– “Mr Fusion”

• 2007-present: < 100 MW limit (€ 100M p.a. @ € 0.12 per kWh)

– Pathfinders confront reality

– Capped SKA demands (and capabilities)

– Optimized system design

– Search for innovative funding solutions for power

Power Investigation Task Force (PITF)

• PITF active in refining SKA power estimates – Current numbers of order 30-40 MW for array, 50-60 MW for computing

• Only achieved with substantial innovation

– Recognize demand may be capped and science limited at a given epoch

• Provide starting material for expert pre-construction power infrastructure consultants

• PITF has informed SKA Program Development Office (SPDO) system design

– Incl. new supply technologies: scaling, breakpoints, …

• PITF has promoted information exchange between SKA

Pathfinders, Design Studies, … – Understand and reconcile various power estimates

Recent PITF topics

• Best estimates of SKA power requirements – Capex and opex implications

• Major expenses which help dimension a feasible SKA project

• SKA Pathfinder power technology demonstrations • Trends in generation and supply technologies

• Demand-minimization and trade-offs in major sub-systems, and

environmental conditioning

• Possible solutions to SKA central and remote power needs

– Grid and non-grid

– Fossil and renewable

• Scalability and life-cycle costing of potential power solutions

• Carbon trading and its effect on SKA power optimizations

Mix of information depth – SKA project just entering detailed infrastructure studies

Example: DSP flexibility vs power trade-off

*Plus cost of cooling and delivering power

• ASIC approach:

– 22nm :

2.5 nW/MHz/Gate

> 40 T MACS (4 bit) per device => 25,000 devices

Assuming < 50 % gates switching at any one time: 600kW

Operating cost $600k per annum*

• For a 1018 MACS processing requirement.

• FPGA approach:

– Virtex 6:

2016 x DSP slices clocked at 600 MHz -> 1200 G MACS

~ 25 G MACs per Watt => Requires ~ 106 FPGAS

@ 48 W per device and ~ 48 M Watts for 1018 MACS

Operating cost 1$ per Watt per year => $48M per annum*

Courtesy P. Dewdney

Path to SKA power system

• On entry to pre-construction phase, we need:

– SKA1 design; electronics system design framed to minimize power

– SKA2 physical layouts and representative electrical loads

– Top-level EMC and RFI standards

– Operational model for SKA1, and representative operational model for

SKA2

– Pool of qualified and briefed consultant engineers and suppliers from which

to select prime contractors, sub-contractors and suppliers.

• Default: SKA power generation and transmission infrastructure

will be delivered and operated via agreement with host nation – Possible international industry participation via e.g. equipment supply

contracts

– Variation: large-scale renewable energy collaboration

• Need mechanisms to feed SKA-specific needs to power industry designers and operators

– SKA adds another layer to traditional “electronics – power” divide

Example inner and intermediate layout. 180 dishes on each spiral

arm. 16 AA-hi and 16 AA-lo stations on each spiral arm (not shown).

Dish core: 1500 dishes 5 km diameter.

AA-hi and AA-lo core 167 stations, 5 km diameter.

-10

-8

-6

-4

-2

0

2

4

6

8

10

-10 -8 -6 -4 -2 0 2 4 6 8 10

AAhi core AAlo core Dish core Dishes mid range

SKA 2012 activities • Representative SKA cores • Detailed configuration

analysis • Site specific information

– e.g. grid or non-grid

• Basic load assumptions for dishes and AA stations

– e.g. low / likely / high

• Basic central DSP model • Post-processing computing

located according to successful site submission

• Simple models for remote stations

• Basic operational and lifecycle model

• Brief consultants for more detailed study

Diagram courtesy R. Bolton (U. Camb.)

SKA power in the PEP

• Five work packages defined in Project Execution Plan (PEP)

• WP10.1 Engineering and Management – Coordination activity; led by SKA Project Office (SPO)

• WP10.2 Intra-system Power Design

– Review SKA design proposal with aim to minimize power; led by SPO with strong industry participation; detailed design report addressing light-heavy electrical system interface

• WP10.3 Power System Design

– Major SKA power task; led by industry with SPO input in specialist areas (e.g. EMC); produces complete, costed design for SKA1 and expansion plan for SKA2

• WP10.4 Power System Operation

– Develop and prosecute a plan for SKA power systems operation; led by industry with SPO input; considers power quality, demand evolution, etc.

• WP10.5 Strategic Power Planning

– Assess applicability of emerging power sources and technologies to the SKA; led by industry with SPO input

SKA and power industry - 1

• Current gap between SKA and detailed design mandate needed by industry

• Gap is beginning to close

– Pathfinders are tackling major design issues (e.g. low-RFI supplies)

– 2012 Project activities will bring serious-scale industry interaction

• Superficially SKA is not hugely challenging

– c.f. hundreds of MW at remote natural resources sites in RSA and AU

– Many similarities to supplying a town, its suburbs and outlying areas

• But SKA is idiosyncratic

– 50 year facility lifetime, non-commercial customer

– Geographical diversity

– Dominance of RFI / EMC in power system design

– “Soft” performance - cost targets for remote stations

– Relatively flat load versus time-of-day

– Simultaneous SKA operation and expansion

SKA and power industry - 2

• Need to combine power cost modelling (incl. lifecycle costing) with SKA performance vs cost modelling

– New SKA design models track power requirements

– First-order studies are important; many trade-offs

• e.g. larger built-area vs (more power-hungry) lower Tsys receptors

• Many factors: capex and opex of receptors, DSP, computing ….

• Recognize political aspects of infrastructure provision and operation could confound engineering analysis

• Generic SKA models are invaluable – Allows power experts to begin thinking about important specifics

– Generates key questions for system design and pre-construction

– Dimensions the challenge for governments and funding agencies

• Recognize site-specific issues will rapidly dominate from 2012 – Possibly: grid + renewables in RSA; islanded gas + renewables in Aust

SKA sites have high solar potential

Courtesy Dr Eicke Weber

Courtesy Dr Eicke Weber

SKA target is < 0.12 €/kWh

Solar PV example: Several other technologies possible

SKA precursor: 1.5 MW supply at MRO

•Modular design using the same construction

technique as the ASKAP correlator building

•Identical RFI mitigation techniques

•Two levels of attenuation in the room design

•Power station located > 1 km from antennas

•All RFI emissions reduced to > 20 dB below

MIL Spec 461F

SKA precursor: Radio-quiet 33 kV feeder in Karoo

• Specialized hardware - Mid-span conductor joints require automatic line splices - Only crimped T-connections and pistol grips allowed to connect conductor at ruling spans - All stay wires are equipped with silicon composite long rod insulator

• Specialized techniques - All fuse links selected with soldered tails to prevent fraying 0 1 2 3 4 5

x 108

30

40

50

60

70

80

90

Frequency (Hz)

Magnitude (

dB

uV

/m)

Frequency spectrum for 4mm on a short line

Background noise

4mm gap

Thorough analysis of conventional lines

Solar power for SKA-low?

Courtesy Budi Juswardy

PV panels and EMC

Where does the radiation come from?

Data centre power - example

20

CRAC: Computer Room Air Conditioner

Liam Newcombe “Energy Efficient Data Centres” British Computer Society Data Centre Specialist Group

http://dcsg.bcs.org//component/option,com_docman/task,cat_view/gid,17/

SKA Data Centre Cooling Demonstration

Courtesy Dr Klaus Regenauer-Liebe

Summary

• SKA faces both power demand and supply challenges

• Only achieve < 100 MW with substantial innovation

– Gains from industry trends (e.g. computing)

– Much SKA development needed (e.g. system design, receptor cooling)

• 100 MW € 100M p.a.

– Large component of SKA opex

– Likely limits science re-investment capacity

• Politics and reality of renewable energy could be favourable to

SKA

– But present SKA budget does not allow Project alone to champion this

• SKA capacity will evolve, even for fixed power ceiling

– Reflect in science design reference mission

• Power is a high-stakes game

– Technical and funding advances translate directly into science performance

and capacity