Supporting Advanced Scientific Computing Research • Basic Energy Sciences • Biological and Environmental Research • Fusion Energy

Sciences • High Energy Physics • Nuclear Physics

ESnetUpdate

Feb3,2009JointTechs,SaltLakeCity

SteveCo;er,DeptHeadsteve@es.net

LawrenceBerkeleyNaFonalLab

NetworkUpdate

ESnet4,OSCARS,OtherProjects

ESnet4Network

3

EquipmentUpgrades/Installs

Peeringupgrades:•  EQX‐SJ:installedMX480onOct15th

•  EQX‐ASH:installedMX480onNov30th

•  EQX‐CHI:PendingMX480installonFeb18th

Site/hubupgrades:•  10GconnecFonatBOISwithPNNLforbackuppeeringNov10th•  10GpeeringatPNWG‐HUBwithKorea(KSTAR&KISTI)Nov11th

•  CombineofLOSA‐SUNN&ELPA‐LOSAintonewELPA‐SUNNSDN(priortothedecommissionofLOSA‐HUB)Dec.3rd

•  OC12betweenDENV‐HUBandPantexJan28th

Jan, 1990 A

pr, 1990 Jul, 1990 O

ct, 1990 Jan, 1991 A

pr, 1991 Jul, 1991 O

ct, 1991 Jan, 1992 A

pr, 1992 Jul, 1992 O

ct, 1992 Jan, 1993 A

pr, 1993 Jul, 1993 O

ct, 1993 Jan, 1994 A

pr, 1994 Jul, 1994 O

ct, 1994 Jan, 1995 A

pr, 1995 Jul, 1995 O

ct, 1995 Jan, 1996 A

pr, 1996 Jul, 1996 O

ct, 1996 Jan, 1997 A

pr, 1997 Jul, 1997 O

ct, 1997 Jan, 1998 A

pr, 1998 Jul, 1998 O

ct, 1998 Jan, 1999 A

pr, 1999 Jul, 1999 O

ct, 1999 Jan, 2000 A

pr, 2000 Jul, 2000 O

ct, 2000 Jan, 2001 A

pr, 2001 Jul, 2001 O

ct, 2001 Jan, 2002 A

pr, 2002 Jul, 2002 O

ct, 2002 Jan, 2003 A

pr, 2003 Jul, 2003 O

ct, 2003 Jan, 2004 A

pr, 2004 Jul, 2004 O

ct, 2004 Jan, 2005 A

pr, 2005 Jul, 2005 O

ct, 2005 Jan, 2006 A

pr, 2006 Jul, 2006 O

ct, 2006 Jan, 2007 A

pr, 2007 Jul, 2007 O

ct, 2007 Jan, 2008 A

pr, 2008 Jul, 2008 O

ct, 2008 Jan, 2009 A

pr, 2009 Jul, 2009 O

ct, 2009 Jan, 2010 A

pr, 2010 Jul, 2010 O

ct, 2010

0.0

0.1

1.0

10.0

100.0

1000.0

10000.0

100000.0

Actual

Exponential regression extended 12 months beyond actualESnet Accepted Traffic (TB/mo) - Log Scale

Projected volume for Dec, 2010: 10442 TB

Actual volume for Dec, 2009: 3562 TB

Newtrend?ESnetTraffic

MonitoringtrafficonprimaryandsecondaryESnet/USLHCnetinterconnectsatStarLight

8Gprimary

6GSecondary

Dec13‐2.36TeVLHCRun

•  SuccessfullydeployedwithinESnetSDN•  OSCARSSoiwareisOpen‐source(oscars‐idc.googlecode.com)

–  Aresourceforthecommunity–  Example:Internet2IONleveragesOSCARS

•  Ongoingchallenge:Builddual‐purposesoiware–  Enableresearcherstoinnovateusingthisframework–  Providerobustproduct‐gradesoiware–  TakeadvantageofnewinnovaFonsandresearchinthisfield

•  DirecFonforward:BuildlargercriFcalmassaroundtheopen‐sourceeffort•  Collaboratewithlike‐mindedresearchersandopen‐source

projects(likeOpen‐DRAC)

OSCARS:“MulF‐Domain,VirtualCircuits”asaService

Jan

, 20

00

Ap

r, 20

00

Jul, 2

00

0

Oct, 2

00

0

Jan

, 20

01

Ap

r, 20

01

Jul, 2

00

1

Oct, 2

00

1

Jan

, 20

02

Ap

r, 20

02

Jul, 2

00

2

Oct, 2

00

2

Jan

, 20

03

Ap

r, 20

03

Jul, 2

00

3

Oct, 2

00

3

Jan

, 20

04

Ap

r, 20

04

Jul, 2

00

4

Oct, 2

00

4

Jan

, 20

05

Ap

r, 20

05

Jul, 2

00

5

Oct, 2

00

5

Jan

, 20

06

Ap

r, 20

06

Jul, 2

00

6

Oct, 2

00

6

Jan

, 20

07

Ap

r, 20

07

Jul, 2

00

7

Oct, 2

00

7

Jan

, 20

08

Ap

r, 20

08

Jul, 2

00

8

Oct, 2

00

8

Jan

, 20

09

Ap

r, 20

09

Jul, 2

00

9

Oct, 2

00

9

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Accepted

OSCARS AcceptedESnet Accepted Traffic (TB/mo)

ESnetIP&OSCARSTraffic

No-fica-onBroker• ManageSubscripFons• ForwardNoFficaFons

AuthN• AuthenFcaFon

PathSetup• NetworkElement

Interface

Coordinator• WorkflowCoordinator

PCE• ConstrainedPathComputaFons

TopologyBridge• TopologyInformaFon

Management

WSAPI• ManagesExternalWS

CommunicaFons

ResourceManager• ManageReservaFons

• AudiFng

Lookup• Lookupservice

AuthZ*• AuthorizaFon

• CosFng

*Dis%nct Data and Control Plane Func%ons 

WebBrowserUserInterface

50%

80%

50%

95%

50%

95%

20%

50%70%

90%

60%

OSCARS0.6–TargetRelease3/10

•  ESnetisakeymemberoftheperfSONARcollaboraFon–h;p://www.perfsonar.net

•  Numeroustesthostsdeployed,automatedtestsarerunregularly(h;p://stats1.es.net)

•  TesthostsareavailabletoESnetsitesandR&Ecollaboratorsforbwctl/iperftests

•  TestandmeasurementisveryhelpfulinlocaFngthecauseofnetworkperformanceproblems

perfSONAR

AdvancedNetworkingIniFaFve

PrototypeNetworkandTestbed

• Prototypenetwork:–  Acceleratethedeploymentof100Gbpstechnologies–  BuildapersistentinfrastructurethatwilltransiFontotheproducFonnetwork~2012

•  KeysteptowardDOE’svisionofa1‐TerabitnetworklinkingDOEsupercompuFngcentersandexperimentalfaciliFes

• Testbed:–  BuildanexperimentalnetworkresearchenvironmentatsufficientscaletousefullytestexperimentalapproachestonextgeneraFonnetworks

•  Fundedfor3years,thenrollintotheESnetprogram•  Breakable,reserveable,configurable,rese;able•  EnableR&Datspeedsupto100Gbps

12

ANIProjectGoals

OpFcalclientsideconnecFons,i.e.1x100GE

man‐wdm

BayExpressBackBone

bb‐wdm

NERSC

pe‐wdm

ce‐rtr

1x100GE

3x100GE

man‐wdm

ChiExpressBackBone

bb‐wdm

pe‐rtr1x100GE

ANL

pe‐wdm

ce‐rtr

1x100GE

ORNL

pe‐wdm

ce‐rtr

1x100GE

OpFcalclientsideconnecFons,i.e.1x100GE

bb‐wdmAofA

aofa‐wdm

newy‐wdm

bnl‐wdm1 bnl‐wdm2

LIMAN

1x100GE

StarlightPeeringExchange

1x100GE

MANLANPeeringExchange

1x100GE

ANIBaselineDesignTestbed

• Tabletoptestbedequipment–ordered,rackedatLBL• 100Gbpstechnologyresearch&evaluaFonphase‐ongoing

–  MeeFngs/briefingswithvendors–  EquipmentinESnetlab

• TransportRFPwri;en–goingthroughreviews–  Acquire100Gbpswaveservicefromacarrier–  Don’tneedtoown/controlopFcalgear–  PlantorunOSCARSlayer2/3servicesacrossnetwork–  DarkfiberispartofDOE’slong‐termresearchagenda

• RouFng/SwitchRFP–Summer2010–  ESnetwillpurchasethisequipment–  WillconducttesFng/evaluaFonaspartofselecFonprocess

14

ANIProgresstoDate

ESnetProjects

Fenius,SiteOutreach,OtherProjects

•  Atthe9thAnnualGlobalLambdaGridWorkshopinDaejeon,Korea.–  ESnet,KISTI,AISTandtheEU‐fundedPhosphorusprojectsuccessfullydemonstratedinteroperabilitybetweentheirnetworkresourceschedulingsystems

•  CoordinatedwithintheacFviFesoftheGLIFconsorFumGNIAPITaskForce–  Developedspecializedsoiwaretoenablethedifferentnetworkschedulingservicestobeusedandmonitoredthroughonecommoninterface.

•  DemonstratedagainatSupercompuFng2009

Fenius

LongIslandMAN

RFPresponsesdue:Feb22nd

•  Spectrum–  AllJuniperroutersarebeingpolledwithandsendtrapsusingIPv6–  TheSpectrumMPLStransportmanagerauto‐discoverstheOSCARS

circuittopology.

–  OSCARSLSPalarmshavenowbeenintegratedintothedailyoutagesonthePlannedMaintenanceCalendar(PMC)

•  DNSSEC–completedaheadofOMBmandate

•  Cfengine–provideautomatedconfiguraFonandmaintenanceofservers,fromapolicyspecificaFon

•  Nagios&OpsView–forautomatedmonitoringserversandotherequipment

•  Bladeservers–reducespaceandpowerconsumpFon,labor

InternalProjects

•  StartedJan1,ledbyEliDart•  GoalistoincreaseeffecFveuseofnetworksforscience–  LeverageESnet’sexperienceinhelpingsitessolveproblemsandincreaseperformance

–  Understandsitenetworkinfrastructure,drivers,andlong‐termplans

–  Helpsitesanddisciplinesbuildnetworkswell‐matchedtotheirneeds

SiteOutreachProgram

•  Pilotunderway:SLAC•  Lookingat:

– Networkarchitecture•  Impactof“convergednetworks”onhighbandwidthdatatransfers–andpossibleneedforseparaFonofScienceandEnterprisenetworks

•  Adequatebufferingonswitchesandrouters– HostandsystemconfiguraFon

•  Dedicatedhostsforwideareadatatransfer•  ProperTCPtuning

–  Testandmeasurementinfrastructure(e.g.perfSONAR)

SiteOutreachProgram

•  Planned Maintenance Calendar •  Network measurement •  Web site update •  weathermap.es.net •  ESnet View •  Twitter feeds

OtherESnetProjects