0

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

U.S. Department of Energy’sOffice of Science

High Performance Networks & Associated Research

Dan HitchcockDaniel.Hitchcock@science.doe.gov

301-903-6767

U.S. Department of Energy’sOffice of Science

1

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Networks Support Science Mission

2

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

FY 2007 Budget Request Supports Office of Science

Network Budgets

0

5000

10000

15000

20000

25000

30000

FY 2005 FY 2006 FY 2007

Fiscal Year

k$

ResearchUSnetESnet

3

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Networking Workshops

4

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

ESnet Connects SC Assets to Scientists worldwide (since 1987)

ESne

t Sc

ienc

e Dat

a Net

work

(S

DN) co

re

TWC

SNLL

YUCCA MT

BECHTEL-NV

PNNLLIGO

INEEL

SNLAAlliedSignal

PANTEX

ARM

KCP

NOAA

OSTI ORAU

SRS

JLAB

PPPLLab DCOffices

MIT

ANL

BNL

FNALAMESLLNL

GA

DOE-ALB

OSC GTNNNSA

International (high speed)10 Gb/s SDN core10G/s IP core2.5 Gb/s IP coreMAN rings (= 10 G/s)OC12 / GigEthernetOC3 (155 Mb/s)45 Mb/s and less

Office Of Science Sponsored (22)NNSA Sponsored (12)Joint Sponsored (3)Other Sponsored (NSF LIGO, NOAA)Laboratory Sponsored (6)

42 end user sites

ESnet IP core

CA*net4FranceGLORIAD(Russia, China)

Korea (Kreonet2

Japan (SINet)Australia (AARNet)Canada (CA*net4Taiwan (TANet2)Singaren

ESnet IP core: Packet over SONET Optical Ring and Hubs

ELP

ATL

DC

commercial and R&E peering points

MAE-E

PAIX-PAEquinix, etc.

PNW

GPo

P/PA

cific

Wav

e

ESnet core hubsIP

Abilene high-speed peering points with Internet2/Abilene

Abilene

CERN(USLHCnet

CERN+DOE funded)

GÉANT- France, Germany,

Italy, UK, etc

NYC

StarlightChi NAP

CH

I-SL

SNV

Abilene

SNV SDN

JGILBNL

SLACNERSC

SDSC

Equinix

MAXGPoP

SoXGPoP

SNV

ALB

ORNL

CHI

MRENNetherlandsStarTapTaiwan (TANet2)UltraLight

NA

SAA

mes

AU

AU

SEA

SINet (Japan)Russia (BINP)

LANL

NREL

AMPATH(S. America)

AMPATH

MAN

LAN

Abile

ne

MAN rings

Abile

ne

USN

USN

ESne

t Sc

ienc

e Dat

a Net

work

(S

DN) co

re

TWCTWCTWC

SNLLSNLLSNLL

YUCCA MTYUCCA MTYUCCA MT

BECHTEL-NV

BECHTEL-NV

BECHTEL-NV

PNNLPNNLPNNLLIGOLIGOLIGOLIGO

INEELINEELINEELINEEL

SNLASNLASNLAAlliedSignalAlliedSignalAlliedSignalAlliedSignal

PANTEX

PANTEX

PANTEX

PANTEX

ARMARMARM

KCPKCPKCP

NOAANOAANOAANOAA

OSTIOSTIOSTIOSTI ORAUORAUORAUORAU

SRSSRSSRSSRS

JLABJLABJLAB

PPPLPPPLPPPLLab DCOfficesLab DCOfficesLab DCOffices

MITMITMIT

ANLANLANL

BNLBNLBNL

FNALFNALFNALAMESAMESAMESLLNLLLNL

GAGA

DOE-ALBDOE-ALBDOE-ALB

OSC GTNNNSA

OSC GTNNNSA

International (high speed)10 Gb/s SDN core10G/s IP core2.5 Gb/s IP coreMAN rings (= 10 G/s)OC12 / GigEthernetOC3 (155 Mb/s)45 Mb/s and less

Office Of Science Sponsored (22)NNSA Sponsored (12)Joint Sponsored (3)Other Sponsored (NSF LIGO, NOAA)Laboratory Sponsored (6)

42 end user sites

ESnet IP core

CA*net4FranceGLORIAD(Russia, China)

Korea (Kreonet2

Japan (SINet)Australia (AARNet)Canada (CA*net4Taiwan (TANet2)Singaren

ESnet IP core: Packet over SONET Optical Ring and Hubs

ELP

ATL

DCDC

commercial and R&E peering points

MAE-E

PAIX-PAEquinix, etc.

PNW

GPo

P/PA

cific

Wav

e

ESnet core hubsIP

AbileneAbilene high-speed peering points with Internet2/Abilene

AbileneAbilene

CERN(USLHCnet

CERN+DOE funded)

GÉANT- France, Germany,

Italy, UK, etc

GÉANT- France, Germany,

Italy, UK, etc

NYC

Starlight

StarlightChi NAP

Chi NAP

CH

I-SLC

HI-SL

SNV

AbileneAbilene

SNV SDN

JGIJGILBNLLBNL

SLACSLACNERSCNERSC

SDSC

Equinix

MAXGPoP

MAXGPoP

SoXGPoPSoXGPoP

SNVSNV

ALB

ORNLORNLORNL

CHI

MRENNetherlandsStarTapTaiwan (TANet2)UltraLight

NA

SAA

mes

NA

SAA

mes

AU

AU

SEASEA

SINet (Japan)Russia (BINP)SINet (Japan)Russia (BINP)

LANLLANLLANL

NRELNREL

AMPATH(S. America)

AMPATH

MAN

LAN

Abile

ne

MAN

LAN

Abile

ne

MAN rings

Abile

neAb

ilene

USNUSN

USNUSN

5

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Ultra Science Net Ultra High-Speed Network

Testbed

• GoalExplore advanced cost-effective optical network technologies that can be used to distribute petabytes-scale data, interconnect distributed terascale computing resources and instrument, and support distributed high-end applications • Features:

20 Gpbs with node at: ORNL, StarLight, FNAL, PNNL, Sunnyvale, Caltech, ANL*, BNL*Circuit peering at Starlight with Internet2 HOPI and EU GEANT

* Planned Sites

• Capabilities• Guaranteed end-to-end VLAN circuits• On-demand and reservation bandwidth services• Transport protocols independent (non-IP)• Hybrid mode: Seamless co-existence with best-effort IP network

•Tech Transfer• LHCnet, Interet2 HOPI, ESNet Science Data

Network, LHC & ATLAS tier1 network technologies

6

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Requirements Grow RapidlyNetworks for LHC

CER

N-1

GÉA

NT-1

GÉA

NT-2

CER

N-2

Tier 1 CentersESnet IP core hubs

ESnet SDN/NLR hubsCross connects with Internet2/Abilene

Abilene/GigaPoP nodes

USLHC nodes

New York

Tier 2 Sites

Denver

Sunn

yval

e

LA

KC

Dallas

Albuq.

CER

N-

3

ESnetSDN

Abilene / Gigapop Footprint

Seattle

FNAL (CMS T1)

BNL (Atlas T1)

Wash DC

Jacksonville

Boise

San DiegoAtlanta

Vancouver

Toronto

Chicago

ESnetIP Core

TRIUMF (Atlas T1, Canada)

CANARIE

GÉANT

USLHCNet

Virtual Circuits

• Direct connectivity T0-T1-T2

• USLHCNet to ESnet to Abilene

• Backup connectivity

• SDN, GLIF, Virtual Circuits

7

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Everything is Integrated in the Future

8

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

ESnet Traffic Characterization

9

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

ESnet links USto the World

10

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Disruptive Changes in Networks for Science

11

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Future ESnet Proposal

12

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

ESnet External Review Feb 21-23

• Same, gigapops, network research, SC laboratories Process as used for other major SC projects

• Broad Committee representing scientific disciplines, universities

• Broad endorsement of ESnet approach

• Recognition of increased complexity and need for outreach to partners, especially Regional Optical Networks and end sites

13

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Implications for Sites

• Hybrid Networks in site infrastructure

• Network “Peering” at Layers 1,2,3

• New Cybersecurity Challenges

• Defining Next Generation DMZ

14

Advanced Scientific Computing Research Program

ASCAC Meeting: March 15-16, 2006

U.S. Department of Energy

Office of Science

Network Environment Research

• Inter-Domain Interfaces

• End-to-end performance

• Cyber security

• High-Performance Middleware

• Integrated testbeds and networks