HENP Networks and Grids for Global Science HENP Networks and Grids for Global Science Harvey B....
-
date post
21-Dec-2015 -
Category
Documents
-
view
223 -
download
6
Transcript of HENP Networks and Grids for Global Science HENP Networks and Grids for Global Science Harvey B....
HENP HENP Networks and Grids for Global ScienceNetworks and Grids for Global Science
Harvey B. NewmanHarvey B. Newman California Institute of TechnologyCalifornia Institute of Technology33rdrd International Data Grid Workshop International Data Grid Workshop
Daegu, Korea August 26, 2004Daegu, Korea August 26, 2004
Challenges for Global HENP ExperimentsChallenges for Global HENP Experiments
Major Challenges (Shared with Other Fields)Major Challenges (Shared with Other Fields) Worldwide Communication and Collaboration Worldwide Communication and Collaboration Managing Globally Distributed Computing & Data Resources Managing Globally Distributed Computing & Data Resources Cooperative Software Development and Data AnalysisCooperative Software Development and Data Analysis
LHC Example- 20075000+ Physicists 250+ Institutes 60+ Countries
BaBar/D0 Example - 2004500+ Physicists 100+ Institutes 35+ Countries
First Beams: Summer 2007Physics Runs: from Fall 2007
TOTEM pp, general purpose; HI
pp, general purpose; HI
LHCb: B-physics
ALICE : HI
pp s =14 TeV L=1034 cm-2 s-1
27 km Tunnel in Switzerland & France
Large Hadron Collider (LHC) CERN, Geneva: 2007 Start
Large Hadron Collider (LHC) CERN, Geneva: 2007 Start
CMS
Atlas
Higgs, SUSY, QG Plasma, CP Violation, … the Unexpected
Challenges of Next Generation Science in the Information AgeChallenges of Next Generation Science in the Information Age
Flagship Applications Flagship Applications High Energy & Nuclear Physics, AstroPhysics Sky Surveys:High Energy & Nuclear Physics, AstroPhysics Sky Surveys:
TByte to PByte “block” transfers at 1-10+ Gbps TByte to PByte “block” transfers at 1-10+ Gbps Fusion Energy:Fusion Energy: Time Critical Burst-Data Distribution; Time Critical Burst-Data Distribution;
Distributed Plasma Simulations, Visualization and Analysis; Distributed Plasma Simulations, Visualization and Analysis; Preparations for Fusion Energy Experiment Preparations for Fusion Energy Experiment
eVLBI:eVLBI: Many real time data streams at 1-10 Gbps Many real time data streams at 1-10 Gbps BioInformatics, Clinical Imaging:BioInformatics, Clinical Imaging: GByte images on demand GByte images on demand
Provide results with rapid turnaround, coordinating Provide results with rapid turnaround, coordinating large but limited computing and data handling resources,large but limited computing and data handling resources,over networks of varying capability in different world regionsover networks of varying capability in different world regions
Advanced integrated applications, such as Data Grids, Advanced integrated applications, such as Data Grids, rely on seamless operation of our LANs and WANsrely on seamless operation of our LANs and WANs With reliable, quantifiable high performanceWith reliable, quantifiable high performance
Petabytes of complex data explored and analyzed by Petabytes of complex data explored and analyzed by 1000s of globally dispersed scientists, in hundreds of teams1000s of globally dispersed scientists, in hundreds of teams
LHC Data Grid Hierarchy:Developed at Caltech
LHC Data Grid Hierarchy:Developed at Caltech
Tier 1
Tier2 Center
Online System
CERN Center PBs of Disk;
Tape Robot
FNAL CenterIN2P3 Center INFN Center RAL Center
InstituteInstituteInstituteInstitute
Workstations
~100-1500 MBytes/sec
~10 Gbps
1 to 10 Gbps
Tens of Petabytes by 2007-8.An Exabyte ~5-7 Years later.Physics data cache
~PByte/sec
10 - 40 Gbps
Tier2 CenterTier2 CenterTier2 Center
~10 Gbps
Tier 0 +1
Tier 3
Tier 4
Tier2 Center Tier 2
Experiment
CERN/Outside Resource Ratio ~1:2Tier0/( Tier1)/( Tier2) ~1:1:1
Emerging Vision: A Richly Structured, Global Dynamic System
ICFA and Global Networks ICFA and Global Networks for Collaborative Sciencefor Collaborative Science
ICFA and Global Networks ICFA and Global Networks for Collaborative Sciencefor Collaborative Science
National and International Networks, with sufficient National and International Networks, with sufficient (rapidly increasing) capacity and seamless end-to-end (rapidly increasing) capacity and seamless end-to-end capability, are essential forcapability, are essential forThe daily conduct of collaborative work in bothThe daily conduct of collaborative work in both
experiment and theory experiment and theoryExperiment development & construction Experiment development & construction
on a global scale on a global scaleGrid systems supporting analysis involving Grid systems supporting analysis involving
physicists in all world regions physicists in all world regions The conception, design and implementation of The conception, design and implementation of
next generation facilities as “global networks” next generation facilities as “global networks” ““Collaborations on this scale would never have been Collaborations on this scale would never have been
attempted, if they could not rely on excellent networks”attempted, if they could not rely on excellent networks”
History of Bandwidth Usage – One Large Network; One Large Research Site
History of Bandwidth Usage – One Large Network; One Large Research Site
ESnet Monthly Accepted Traffic 1/90-1/04
0
50
100
150
200
250
300
Jan,
90Oc
t, 90
Jul,9
1Ap
r, 92
Jan,
93Oc
t,93
Jul, 9
4Ap
r, 95
Jan,
96Oc
t, 96
Jul, 9
7Ap
r, 98
Jan,
99Oc
t, 99
Jul, 0
0Ap
r, 01
Jan,
02Oc
t, 02
Jul, 0
3
TByte
/Mon
th
SLAC Traffic ~300 Mbps; ESnet Limit
Growth in Steps: ~ 10X/4 YearsProjected: ~2 Terabits/s by
~2014
ESnet Accepted Traffic 1/90 – 1/04Exponential Growth Since ’92;
Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 Years
Int’l Networks BW on Major Links for HENP: US-CERN Example
Int’l Networks BW on Major Links for HENP: US-CERN Example
Rate of Progress >> Moore’s Law (US-CERN Example)Rate of Progress >> Moore’s Law (US-CERN Example) 9.6 kbps Analog9.6 kbps Analog (1985) (1985) 64-256 kbps Digital (1989 - 1994) [X 7 – 27]64-256 kbps Digital (1989 - 1994) [X 7 – 27] 1.5 Mbps Shared (1990-3; IBM) [X 160]1.5 Mbps Shared (1990-3; IBM) [X 160] 2 -4 Mbps2 -4 Mbps (1996-1998) [X 200-400] (1996-1998) [X 200-400] 12-20 Mbps (1999-2000) [X 1.2k-2k]12-20 Mbps (1999-2000) [X 1.2k-2k] 155-310 Mbps155-310 Mbps (2001-2) [X 16k – 32k] (2001-2) [X 16k – 32k] 622 Mbps622 Mbps (2002-3) [X 65k] (2002-3) [X 65k] 2.5 Gbps 2.5 Gbps (2003-4) [X 250k] (2003-4) [X 250k] 10 Gbps 10 Gbps (2005) [X 1M] (2005) [X 1M] 44x10 Gbps or 40 Gbps (2007-8)x10 Gbps or 40 Gbps (2007-8) [X 4M][X 4M]
A factor of ~1M Bandwidth Improvement over A factor of ~1M Bandwidth Improvement over 1985-2005 (a factor of ~5k during 1995-2005)1985-2005 (a factor of ~5k during 1995-2005)
A prime enabler of major HENP programsA prime enabler of major HENP programs HENP has become a leading applications driver, HENP has become a leading applications driver,
and also a co-developer of global networks and also a co-developer of global networks
Internet Growth in the World At Large
Internet Growth in the World At Large
Amsterdam Internet Exchange Point Example
20 Gbps
30 Gbps
Average
5 Minute
Max
Some Annual Growth Spurts;
Typically In Summer-Fall
The Rate of HENP Network Usage Growth (~100% Per Year) is Similar to the World at Large
11.08.04
Judged on product of transfer speed Judged on product of transfer speed and distance end-to-end, using and distance end-to-end, using standard Internet (TCP/IP) protocols.standard Internet (TCP/IP) protocols.
IPv6 record: IPv6 record: 4.0 Gbps4.0 Gbps between between Geneva and Phoenix (SC2003)Geneva and Phoenix (SC2003)
IPv4 Multi-stream record with IPv4 Multi-stream record with Windows & Linux: Windows & Linux: 6.6 Gbps6.6 Gbps between between Caltech and CERN (16 kkm; Caltech and CERN (16 kkm; “Grand Tour d’Abilene”) June 2004“Grand Tour d’Abilene”) June 2004 Exceeded 100 Petabit-m/secExceeded 100 Petabit-m/sec
Single Stream 7.5 Gbps X 16 kkmSingle Stream 7.5 Gbps X 16 kkm with with Linux Achieved in JulyLinux Achieved in July
Concentrate now on reliable Concentrate now on reliable Terabyte-scale file transfersTerabyte-scale file transfersNote System Issues: CPU, PCI-XNote System Issues: CPU, PCI-X
Bus, NIC, I/O Controllers, Drivers Bus, NIC, I/O Controllers, Drivers
Monitoring of the Abilene traffic in LA:Monitoring of the Abilene traffic in LA:
6.6 Gbps16500km
4.2 Gbps16343km5.6 Gbps
10949km
5.4 Gbps7067km
2.5 Gbps10037km0.9 Gbps
10978km0.4 Gbps12272km
0
20
40
60
80
100
120
Internet 2 Land Speed Record (LSR)Internet 2 Land Speed Record (LSR)
Pe
tab
itm
ete
r (1
0^1
5 b
it*m
ete
r)
LSR History – IPv4 single stream
http://www.guinnessworldrecords.com/
June 2004 Record Network
Evolving Quantitative Science Requirements for Networks (DOE High Perf. Network Workshop)
Evolving Quantitative Science Requirements for Networks (DOE High Perf. Network Workshop)
Science AreasScience Areas Today Today End2EndEnd2End
ThroughputThroughput
5 5 years years End2End End2End
ThroughputThroughput
5-10 5-10 Years Years End2End End2End
ThroughputThroughput
RemarksRemarks
High Energy High Energy PhysicsPhysics
0.5 0.5 Gb/sGb/s 100 100 Gb/sGb/s 1000 1000 Gb/sGb/s High bulk High bulk throughputthroughput
Climate (Data & Climate (Data & Computation)Computation)
0.5 0.5 Gb/sGb/s 160-200 160-200 Gb/sGb/s N x 1000 Gb/sN x 1000 Gb/s High bulk High bulk throughputthroughput
SNS NanoScienceSNS NanoScience Not yet startedNot yet started 1 1 Gb/sGb/s 1000 1000 Gb/s + QoS Gb/s + QoS for Control for Control
ChannelChannel
Remote control Remote control and time critical and time critical
throughputthroughput
Fusion EnergyFusion Energy 0.066 0.066 Gb/sGb/s(500 MB/s (500 MB/s
burst)burst)
0.198 0.198 Gb/sGb/s(500MB/(500MB/
20 sec. burst)20 sec. burst)
N x 1000 Gb/sN x 1000 Gb/s Time critical Time critical throughputthroughput
AstrophysicsAstrophysics 0.013 0.013 Gb/sGb/s(1 TByte/week)(1 TByte/week)
N*N multicastN*N multicast 1000 1000 Gb/sGb/s Computational Computational steering and steering and
collaborationscollaborations
Genomics Data & Genomics Data & ComputationComputation
0.091 0.091 Gb/sGb/s(1 TBy/day)(1 TBy/day)
100100s of userss of users 1000 1000 Gb/s + QoS Gb/s + QoS for Control for Control
ChannelChannel
High throughput High throughput and steeringand steering
HENP Lambda Grids:Fibers for Physics
HENP Lambda Grids:Fibers for Physics
Problem: Extract “Small” Data Subsets of 1 to 100 Terabytes Problem: Extract “Small” Data Subsets of 1 to 100 Terabytes from 1 to 1000 Petabyte Data Storesfrom 1 to 1000 Petabyte Data Stores
Survivability of the HENP Global Grid System, with Survivability of the HENP Global Grid System, with hundreds of such transactions per day (circa 2007)hundreds of such transactions per day (circa 2007)requires that each transaction be completed in a requires that each transaction be completed in a relatively short time. relatively short time.
Example: Take 800 secs to complete the transaction. ThenExample: Take 800 secs to complete the transaction. Then Transaction Size (TB)Transaction Size (TB) Net Throughput (Gbps)Net Throughput (Gbps) 1 101 10 10 10010 100 100 1000 (Capacity of 100 1000 (Capacity of
Fiber Today) Fiber Today) Summary: Providing Switching of 10 Gbps wavelengthsSummary: Providing Switching of 10 Gbps wavelengths
within ~2-4 years; and Terabit Switching within 5-8 years within ~2-4 years; and Terabit Switching within 5-8 years would enable “Petascale Grids with Terabyte transactions”,would enable “Petascale Grids with Terabyte transactions”,to fully realize the discovery potential of major HENP programs, to fully realize the discovery potential of major HENP programs, as well as other data-intensive research.as well as other data-intensive research.
SCIC in 2003-2004 http://cern.ch/icfa-scic SCIC in 2003-2004 http://cern.ch/icfa-scic
Three 2004 Reports; Presented to ICFA in FebruaryThree 2004 Reports; Presented to ICFA in February
Main Report: “Networking for HENP”Main Report: “Networking for HENP” [H. Newman et al.] [H. Newman et al.] Includes Brief Updates on Monitoring, the Digital Divide Includes Brief Updates on Monitoring, the Digital Divide
and Advanced Technologies [*]and Advanced Technologies [*] A World Network Overview (with 27 Appendices): A World Network Overview (with 27 Appendices):
Status and Plans for the Next Few Years of National & Status and Plans for the Next Few Years of National & Regional Networks, and Optical Network InitiativesRegional Networks, and Optical Network Initiatives
Monitoring Working Group ReportMonitoring Working Group Report [L. Cottrell] [L. Cottrell]
Digital Divide in RussiaDigital Divide in Russia [V. Ilyin] [V. Ilyin]
August 2004 Update Reports at the SCIC Web Site:August 2004 Update Reports at the SCIC Web Site:See http://icfa-scic.web.cern.ch/ICFA-SCIC/documents.htmSee http://icfa-scic.web.cern.ch/ICFA-SCIC/documents.htm
Asia Pacific, Latin America, GLORIAD (US-Ru-Asia Pacific, Latin America, GLORIAD (US-Ru-KoKo-China);-China);Brazil, Korea, etc. Brazil, Korea, etc.
SCIC Main Conclusion for 2003Setting the Tone for 2004
SCIC Main Conclusion for 2003Setting the Tone for 2004
The disparity among regions in HENP could increase The disparity among regions in HENP could increase even more sharply, as we learn to use advanced even more sharply, as we learn to use advanced networks effectively, and we develop dynamic Grid networks effectively, and we develop dynamic Grid systems in the most favored” regions systems in the most favored” regions
We must take action, and workWe must take action, and work to Close the Digital Divideto Close the Digital Divide To make Physicists from All World Regions Full To make Physicists from All World Regions Full
Partners in Their Experiments; and in the Process Partners in Their Experiments; and in the Process of Discoveryof Discovery
This is essential for the health of our global This is essential for the health of our global experimental collaborations, our plans for future experimental collaborations, our plans for future projects, and our field.projects, and our field.
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (2)General Conclusions (2)
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (2)General Conclusions (2)
Reliable high Reliable high End-to-end PerformanceEnd-to-end Performance of networked applications such as large of networked applications such as large file transfers and Data Grids is required. Achieving this requires:file transfers and Data Grids is required. Achieving this requires: End-to-end monitoring extending to all regions serving our community.End-to-end monitoring extending to all regions serving our community.
A coherent approach to monitoring that allows physicists throughout A coherent approach to monitoring that allows physicists throughout our community to extract clear information is required. our community to extract clear information is required.
Upgrading campus infrastructures.Upgrading campus infrastructures. These are still not designed to support Gbps data transfers in most HEP These are still not designed to support Gbps data transfers in most HEP centers. One reason for under-utilization of national and Int’l backbones, is centers. One reason for under-utilization of national and Int’l backbones, is the lack of bandwidth to end-user groups in the campus. the lack of bandwidth to end-user groups in the campus.
Removing local, last mile, and nat’l and int’l bottlenecks Removing local, last mile, and nat’l and int’l bottlenecks end-to-end, whether technical or political in origin. end-to-end, whether technical or political in origin.While National and International backbones have reached 2.5 to 10 Gbps While National and International backbones have reached 2.5 to 10 Gbps speeds in many countries, the bandwidths speeds in many countries, the bandwidths across borders, the countryside across borders, the countryside or the city may be much less. or the city may be much less.
This problem is very widespread in our community, This problem is very widespread in our community, with examples stretching from the Asia Pacific to with examples stretching from the Asia Pacific to Latin America Latin America to the Northeastern U.S. Root causes for this vary, to the Northeastern U.S. Root causes for this vary, from lack from lack of local infrastructure to unfavorable pricing policies. of local infrastructure to unfavorable pricing policies.
ICFA Report (2/2004) Update: ICFA Report (2/2004) Update: Main Trends Continue, Some AccelerateMain Trends Continue, Some Accelerate
ICFA Report (2/2004) Update: ICFA Report (2/2004) Update: Main Trends Continue, Some AccelerateMain Trends Continue, Some Accelerate
Current generation of 2.5-10 Gbps network backbones and major Int’l links Current generation of 2.5-10 Gbps network backbones and major Int’l links arrived in the last 2-3 Years [US+Europe+Japan; Now arrived in the last 2-3 Years [US+Europe+Japan; Now Korea and ChinaKorea and China]] Capability: 4 to Hundreds of Times; Much Faster than Moore’s LawCapability: 4 to Hundreds of Times; Much Faster than Moore’s Law
Proliferation of 10G links across the Atlantic Now; Will Begin use of Multiple Proliferation of 10G links across the Atlantic Now; Will Begin use of Multiple 10G Links (e.g. US-CERN) Along Major Paths by Fall 200510G Links (e.g. US-CERN) Along Major Paths by Fall 2005 Direct result of Falling Network Prices: $ 0.5 – 1M Per Year for 10GDirect result of Falling Network Prices: $ 0.5 – 1M Per Year for 10G
Ability to fully use long 10G paths with TCP continues to advance: Ability to fully use long 10G paths with TCP continues to advance: 7.5 Gbps X 16kkm (August 2004)7.5 Gbps X 16kkm (August 2004)
Technological progress driving equipment costs in end-systems lowerTechnological progress driving equipment costs in end-systems lower ““Commoditization” of Gbit Ethernet (GbE) ~complete: ($20-50 per port)Commoditization” of Gbit Ethernet (GbE) ~complete: ($20-50 per port)
10 GbE commoditization (e.g. < $ 2K per NIC with TOE) underway 10 GbE commoditization (e.g. < $ 2K per NIC with TOE) underway Some regions (US, Europe) moving to owned or leased dark fiber Some regions (US, Europe) moving to owned or leased dark fiber Emergence of the “Hybrid” Network Model:Emergence of the “Hybrid” Network Model: GNEW2004; UltraLight, GLIF GNEW2004; UltraLight, GLIF Grid-based AnalysisGrid-based Analysis demands end-to-end high performance & management demands end-to-end high performance & management The rapid rate of progress is confined mostly to the US, Europe, Japan and The rapid rate of progress is confined mostly to the US, Europe, Japan and
Korea, as well as the major Transatlantic routes; this Korea, as well as the major Transatlantic routes; this threatens to threatens to cause cause the Digital Divide to become a Chasmthe Digital Divide to become a Chasm
Work on the Digital Divide:Several Perspectives
Work on the Digital Divide:Several Perspectives
Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, …Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, … Find Ways to work with vendors, NRENs, and/or Gov’tsFind Ways to work with vendors, NRENs, and/or Gov’ts Exploit Model Cases: e.g. Poland, Slovakia, Czech RepublicExploit Model Cases: e.g. Poland, Slovakia, Czech Republic
Inter-Regional Projects Inter-Regional Projects GLORIAD,GLORIAD, Russia-China-US Optical Ring Russia-China-US Optical Ring South America: CHEPREO (US-Brazil); EU CLARA ProjectSouth America: CHEPREO (US-Brazil); EU CLARA Project Virtual SILK Highway Project (DESY): FSU satellite linksVirtual SILK Highway Project (DESY): FSU satellite links
Workshops and Tutorials/Training SessionsWorkshops and Tutorials/Training Sessions For Example: Digital Divide and HEPGrid Workshop,For Example: Digital Divide and HEPGrid Workshop,
UERJ Rio, February 2004; Next Daegu May 2005UERJ Rio, February 2004; Next Daegu May 2005 Help with Modernizing the Infrastructure Help with Modernizing the Infrastructure
Design, Commissioning, DevelopmentDesign, Commissioning, Development Tools for Effective Use: Monitoring, CollaborationTools for Effective Use: Monitoring, Collaboration
Participate in Standards Development; Open ToolsParticipate in Standards Development; Open Tools Advanced TCP stacks; Grid systemsAdvanced TCP stacks; Grid systems
Grid and Network Workshopat CERN March 15-16, 2004
Grid and Network Workshopat CERN March 15-16, 2004
WORKSHOP GOALS Share and challenge the lessons learned by nat’l and
international projects in the past three years; Share the current state of network engineering and
infrastructure and its likely evolution in the near future; Examine our understanding of the networking needs of
Grid applications (e.g., see the ICFA-SCIC reports); Develop a vision of how network engineering and
infrastructure will (or should) support Grid computing needs in the next three years.
CONCLUDING STATEMENT
"Following the 1st International Grid Networking Workshop(GNEW2004) that was held at CERN and co-organized byCERN/DataTAG, DANTE, ESnet, Internet2 & TERENA, there is a wide consensus that hybrid network services capable ofoffering both packet- and circuit/lambda-switching as well as highly advanced performance measurements and a newgeneration of distributed system software, will be required inorder to support emerging data intensive Grid applications, Such as High Energy Physics, Astrophysics, Climate andSupernova modeling, Genomics and Proteomics, requiring 10-100 Gbps and up over wide areas."
National Lambda Rail (NLR)National Lambda Rail (NLR)
15808 Terminal, Regen or OADM siteFiber route
NLRComing Up NowInitially 4 10G
WavelengthsNorthern Route
Operation by 4Q04 Internet2 HOPI Initiative (w/HEP) To 40 10G Waves in Future
Transition beginning now to optical, multi-wavelength Community owned or leased
“dark fiber” networks for R&E
PITPIT
PORPOR
FREFRE
RALRAL
WALWAL
NASNASPHOPHO
OLGOLGATLATL
CHICHI
CLECLE
KANKAN
OGDOGD
SACSAC BOSBOSNYCNYC
WDCWDC
STRSTR
DALDAL
DENDEN
LAXLAX
SVLSVL
SEASEA
SDGSDG
JACJAC nl, de, pl, cz,jp18 US States
JGN2: Japan Gigabit Network (4/04 – 3/08)20 Gbps Backbone, 6 Optical Cross-Connects
Kanazawa Sendai
Sapporo
Nagano
Kochi Nagoya
Fukuoka
Okinawa
Okayama
< 1 G> ・ Teleport Okayama (Okayama) ・ Hiroshima University (Higashi
Hiroshima) < 100 M> ・ Tottori University of
Environmental Studies (Tottori) ・ Techno Ark Shimane (Matsue) ・ New Media Plaza Yamaguchi
(Yamaguchi)
< 10 G> ・ Kyoto University (Kyoto) ・ Osaka University (Ibaraki) < 1 G> ・ NICT Kansai Advanced Research Center (Kobe) < 100 M> ・ Lake Biwa Data Highway AP * (Ohtsu) ・ Nara Prefectural Institute of Industrial
Technology (Nara) ・ Wakayama University (Wakayama) ・ Hyogo Prefecture Nishiharima Technopolis
(Kamigori-cho, Hyogo Prefecture) < 10 G> ・ Kyushu University (Fukuoka) < 100 M> ・ NetCom Saga (Saga) ・ Nagasaki University
(Nagasaki) ・ Kumamoto Prefectural Office
(Kumamoto) ・ Toyonokuni Hyper Network
AP *(Oita) ・ Miyazaki University (Miyazaki) ・ Kagoshima University
(Kagoshima)
< 100 M> ・ Kagawa Prefecture Industry Promotion Center (Takamatsu) ・ Tokushima University (Tokushima) ・ Ehime University (Matsuyama) ・ Kochi University of Technology
(Tosayamada-cho, Kochi Prefecture)
< 100 M> ・ Nagoya University (Nagoya) ・ University of Shizuoka (Shizuoka) ・ Softopia Japan (Ogaki, Gifu Prefecture) ・ Mie Prefectural College of Nursing (Tsu)
< 10 G> ・ Ishikawa Hi-tech Exchange Center
(Tatsunokuchi-machi, Ishikawa Prefecture) < 100 M> ・ Toyama Institute of Information Systems (Toyama) ・ Fukui Prefecture Data Super Highway AP * (Fukui)
< 100 M> ・ Niigata University
(Niigata) ・ Matsumoto Information Creation Center
(Matsumoto, Nagano Prefecture)
< 10 G> ・ Tokyo University
(Bunkyo Ward, Tokyo) ・ NICT Kashima Space Research Center
(Kashima, Ibaraki Prefecture) <1G> ・ Yokosuka Telecom Research Park
(Yokosuka, Kanagawa Prefecture) <100M> ・ Utsunomiya University (Utsunomiya) ・ Gunma Industrial Technology Center
(Maebashi) ・ Reitaku University
(Kashiwa, Chiba Prefecture) ・ NICT Honjo Information and Communications Open Laboratory
(Honjo, Saitama Prefecture) ・ Yamanashi Prefecture Open R&D Center
(Nakakoma-gun, Yamanashi Prefecture)
< 1 G> ・ Tohoku University (Sendai) ・ NICT Iwate IT Open Laboratory
(Takizawa-mura, Iwate Prefecture) < 100 M> ・ Hachinohe Institute of Technology
(Hachinohe, Aomori Prefecture) ・ Akita Regional IX * (Akita) ・ Keio University Tsuruoka Campus
(Tsuruoka, Yamagata Prefecture) ・ Aizu University (Aizu Wakamatsu)
< 100 M> ・ Hokkaido Regional Network Association AP * (Sapporo)
NICT Tsukuba Research Center
20Gbps10Gbps1GbpsOptical testbeds
Core network nodes
Access points
Osaka
Otemachi
USA
NICT Keihannna Human Info-Communications
Research Center
NICT Kita Kyushu IT Open Laboratory
NICT Koganei Headquarters
[Legends ]
*IX:Internet eXchange
AP:Access Point
Integrated hybrid experimental network, leveraging Transatlantic Integrated hybrid experimental network, leveraging Transatlantic R&D network partnerships; packet-switched + dynamic optical paths R&D network partnerships; packet-switched + dynamic optical paths 10 GbE across US and the Atlantic: NLR, DataTAG, TransLight, 10 GbE across US and the Atlantic: NLR, DataTAG, TransLight,
NetherLight, UKLight, etc.; Extensions to Japan, Taiwan, BrazilNetherLight, UKLight, etc.; Extensions to Japan, Taiwan, Brazil End-to-end monitoring; Realtime tracking and optimization; End-to-end monitoring; Realtime tracking and optimization;
Dynamic bandwidth provisioningDynamic bandwidth provisioning Agent-based services spanning all layers of the system,Agent-based services spanning all layers of the system, from the from the
optical cross-connects to the applications.optical cross-connects to the applications.
UltraLight Collaboration:UltraLight Collaboration:http://ultralight.caltech.eduhttp://ultralight.caltech.edu
Caltech, UF, FIU, UMich, SLAC,FNAL,MIT/Haystack,CERN, UERJ(Rio), NLR, CENIC, UCAID,Translight, UKLight, Netherlight, UvA, UCLondon, KEK, Taiwan
Cisco, Level(3)
GLIF: Global Lambda Integrated Facility
DWDM SURFnet
lambda service path
IP service path
10 Gbit/s
SURFnet10 Gbit/s
SURFnet10 Gbit/s
IEEAF10 Gbit/s
DwingelooASTRON/JIVE
DwingelooASTRON/JIVE
PragueCzechLight
PragueCzechLight
2.5 Gbit/s
NSF10 Gbit/s
LondonUKLightLondonUKLight
StockholmNorthernLightStockholm
NorthernLight
2.5 Gbit/s
New YorkMANLANNew YorkMANLAN
TokyoWIDETokyoWIDE
10 Gbit/s
10Gbit/s
10 Gbit/s
2x10 Gbit/s
IEEAF10 Gbit/s
2x10 Gbit/s
10 Gbit/s
2.5 Gbit/s
2.5 Gbit/s
TokyoAPANTokyoAPAN
GenevaCERN
GenevaCERN
ChicagoChicago AmsterdamAmsterdam
“ GLIF is a World Scale Lambda based Lab for Application and Middleware development, where Grid applications ride on dynamically configured networks based on optical wavelengths ... Coexisting with more traditional packet-switched network traffic
4th GLIF Workshop:Nottingham UK Sept. 2004
10 Gbps Wavelengths For R&E Network Development Are Prolifering,
Across Continents and Oceans
1660 km of Dark Fiber CWDM Links, up to 112 km.
1 to 4 Gbps (GbE)
August 2002: First NREN in Europe to establish Int’l GbE Dark Fiber Link, to Austria
April 2003 to Czech Republic.
Planning 10 Gbps Backbone; dark fiber link to Poland this year.
PROGRESS in SE Europe (Sk, Pl, Cz, Hu, …)
Dark Fiber in Eastern Europe Poland: PIONIER Network
Dark Fiber in Eastern Europe Poland: PIONIER Network
ŁÓDŹ
TORUŃ
P OZNAŃ
B YDGOS ZCZ
OLS ZTYN
B IAŁYS TOK
GDAŃS K
KOS ZALIN
S ZCZECIN
ZIELONAGÓRA
WROCŁAW
CZĘS TOCHOWA
KRAKÓW RZES ZÓW
LUB LIN
KIELCE
P UŁAWY
RADOM
KATOWICEGLIWICE
B IELS KO-B IAŁA
OP OLE
GUB IN WARS ZAWA
CIES ZYN
S IEDLCE
P IO NIE R n o d e sIn s ta lle d fib e r
F ib e rs p la n n e d in 2 0 0 4P IO NIE R n o d e s p la n n e d in 2 0 0 4
2650 2650 km Fiber km Fiber ConnectingConnecting
16 MANs; 5200 km 16 MANs; 5200 km and 21 MANs by 2005 and 21 MANs by 2005
Support Support Computational GridsComputational Grids
Domain-Specific Domain-Specific Grids Grids
Digital LibrariesDigital Libraries Interactive TVInteractive TV
Add’l Fibers for Add’l Fibers for e-Regional Initiatives e-Regional Initiatives
The Advantage of Dark Fiber CESNET Case Study (Czech Republic)
The Advantage of Dark Fiber CESNET Case Study (Czech Republic)
Case Study ResultCase Study ResultWavelength ServiceWavelength Service
Vs. Fiber Lease:Vs. Fiber Lease:
Cost Savings of Cost Savings of 50-70% Over 4 Years50-70% Over 4 Years
for Long 2.5G for Long 2.5G or 10G Links or 10G Links
2513 2513 km km Leased Fibers Leased Fibers (Since 1999) (Since 1999)
1 x 2,5G Leased 1 x 2,5G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 7,000
about 300km (e.g. Praha - Brno) 8,000
***
4 x 2,5G Leased 4 x 2,5G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 14,000
about 300km (e.g. Praha - Brno) 23,000
***
1 x 10G Leased 1 x 10G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 14,000
about 300km (e.g. Praha - Brno) 16,000
***
4 x 10G Leased 4 x 10G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 29,000
about 300km (e.g. Praha - Brno) 47,000
***
2 x booster 24dBm, 2 x DCF, DWDM 10G2 x (booster +In-line + preamplifier), 6 x DCF, DWDM 10G
Leased fibre with own equipment (EURO/Month)
5 000 *7 000 **
2 x booster 18dBm 2 x booster 27dBm + 2 x preamplifier + 6 x DCF
Leased fibre with own equipment (EURO/Month)
12 000 *14 000 **
8 000 **
2 x booster 21dBm, 2 x DCF2 x (booster 21dBm + in-line + preamplifier) + 6 x DCF
2 x booster 24dBm, DWDM 2,5G2 x (booster +In-line + preamplifier), 6 x DCF, DWDM 2,5G
Leased fibre with own equipment (EURO/Month)
5 000 *
Leased fibre with own equipment (EURO/Month)
8 000 *11 000 **
ICFA/SCIC Network Monitoring
Prepared by Les Cottrell, SLAC, forICFA
www.slac.stanford.edu/grp/scs/net/talk03/icfa-aug04.ppt
PingER: World View from SLAC
Important for policy makers
TCP throughput measured from N. America to World Regions
1
10
100
1000
10000J
an
-95
Ja
n-9
6
De
c-9
6
De
c-9
7
De
c-9
8
De
c-9
9
De
c-0
0
De
c-0
1
De
c-0
2
De
c-0
3
De
c-0
4
Deri
ved
TC
P t
hro
ug
hp
ut
in K
Byte
s/s
ec
1
10
100
1000
10000
China (13)
S.E. Europe (21)
Europe(150) Canada (27)
Russia(17)
Edu (141)
Latin America (37)
India(7) Africa (30)
Mid East (16)
50% Improvement/year~ factor of 10 in < 6 years
C. Asia (8)
From the PingER project, Aug 2004
Caucasus (8)
C. Asia, Russia, SE Europe, L. America, M. East, China:
4-5 yrs behindIndia, Africa: 7 yrs behind
View from CERNConfirms This View
Now monitoring 650 sites in 115 countriesIn last 9 months:
Several sites in Russia (GLORIAD)Many hosts in Africa (27 of 54 Countries) Monitoring sites in Pakistan, Brazil
Research Networking in Latin America: August 2004
The only Countries with research The only Countries with research network connectivity now in network connectivity now in Latin America:Latin America:Argentina, Brazil, Chile, Argentina, Brazil, Chile, Mexico, Venezuela Mexico, Venezuela
AmPathAmPath Provided connectivity for Provided connectivity for some South American countriessome South American countries
New Sao Paolo-Miami Link at 622 MbpsNew Sao Paolo-Miami Link at 622 MbpsStarting This MonthStarting This Month
New: New: CLARA CLARA (Funded by EU)(Funded by EU) Regional Network Connecting 19 Countries:Regional Network Connecting 19 Countries:
AmPath
Argentina Argentina Dominican Republic Dominican Republic Panama Panama Brasil Brasil Ecuador Ecuador ParaguayParaguayBolivia Bolivia El Salvador El Salvador PeruPeruChile Chile Guatemala Guatemala UruguayUruguayColombia Colombia Honduras Honduras VenezuelaVenezuelaCosta Rica Mexico Costa Rica Mexico NicaraguaNicaraguaCubaCuba
155 Mbps Backbone with 10-45 Mbps Spurs;
4 Mbps Satellite to Cuba; 622 Mbps to Europe
Also NSF Proposals To
Connect at 2.5G to US
HEPGRID (CMS) in BrazilHEPGRID-CMS/BRAZIL is a project to build a Grid thatAt Regional Level will include CBPF,UFRJ,UFRGS,UFBA, UERJ & UNESPAt International Level will be integrated with CMS Grid based at CERN; focal points include iVGDL/Grid3 and bilateral projects with Caltech Group
France Italy USAGermany BRAZIL622 Mbps
UFRGS
UERJUFRJ
T1
IndividualMachines
On line systems
Brazilian HEPGRID
CBPF
UNESP/USPSPRACE-Working
Gigabit
CERN 2.5 - 10 Gbps
UFBA
UERJ RegionalTier2 Ctr
T4
T0 +T1
T2 T1
T3 T2
UERJ: T2T1,
100500 Nodes;
Plus T2s to
100 Nodes
Subject Argentina Brazil Chile Colombia Costa Rica
Equator Mexico
Astrophysics
e-VLBI
High Energy Physics
Geosciences
Marine sciences
Health and Biomedical applications
Environmental studies
Latin America Science Areas Interested in Improving Connectivity ( by Country)
Networks and Grids: The Potential to Spark a New Era of Science in the Region
Asia Pacific Academic Network ConnectivityAsia Pacific Academic Network Connectivity
Access PointExchange PointCurrent Status2004 (plan)
US
Europe
ID
VN
LK
KR
TH
MY
SG
RU
PH
CN
HK
JP2G
12M
34M
45M
20.9G
155M
777M
1.2G
TW
200M
310M
AU
45M`722M
IN
16M
1.5M
1.5M
7.5M2M
2.5M
2M
90M
155M
622M
155M
9.1G
932M(to 21 G)
Better Better North/South Linkages within AsiaNorth/South Linkages within AsiaJP-SG link: 155Mbps in 2005 is proposed to NSF by JP-SG link: 155Mbps in 2005 is proposed to NSF by CIRENCIRENJP- TH link: 2Mbps JP- TH link: 2Mbps 45Mbps in 2004 is being 45Mbps in 2004 is being studied.studied.CIREN is studying an extension to IndiaCIREN is studying an extension to India
Connectivity Connectivity to US from JP, to US from JP, KO, AU is KO, AU is Advancing Advancing Rapidly.Rapidly.
Progress in Progress in the Region, the Region, and to Europe and to Europe is Much is Much Slower Slower
APAN Status July APAN Status July 20042004
APAN Link Information 2004.7.7 [email protected]
Countries Network Bandwidth (Mbps) AUP/RemarkAU-US AARNet 310 to 2 x 10 Gbps soon R&E + CommodityAU-US (PAIX) AARNet 622 R&E + CommodityCN-HK CERNET 622 R&E + CommodityCN-HK CSTNET 155 R&ECN-JP CERNET 155 R&ECN-JP CERNET 45 Native IPv6CN-US CERNET 155 R&E + CommodityCN-US CSTNET 155 R&EHK-US HARNET 45 R&EHK-TW HARNET/TANET/ASNET 100 R&EIN-US/UK ERNET 16 R&EJP-ASIA UDL 9 R&EJP-ID AI3(ITB) 0.5/1.5 R&EJP-KR APII 2Gbps R&EJP-LA AI3 (NUOL) 0.128/0.128 R&EJP-MY AI3(USM) 1.5/0.5 R&EJP-PH AI3(ASTI) 1.5/0.5 R&EJP-PH MAFFIN 6 Research JP-SG AI3(TP) 1.5/0.5 R&EJP-TH AI3(AIT) (service interrupted) R&EJP-TH SINET(ThaiSarn) 2 R&EJP-US TransPac 5 Gbps to 2x10 Gbps soon R&E
APAN Link Information 2004.7.7 [email protected]
Countries Network Bandwidth (Mbps) AUP/Remark(JP)-US-EU SINET 155 R&E / No TransitJP-US SINET 5 Gbps R&E / No TransitJP-US IEEAF 10Gbps R&E wave serviceJP-US IEEAF 622 R&EJP-US Japan-Hawaii 155 R&EJP-VN AI3(IOIT) 1.5/0.5 R&EKR-FR KOREN/RENATER 34 Research (TEIN) KR-SG APII 8 R&EKR-US KOREN/KREONet2 1.2Gbps R&ELK-JP LEARN 2.5 R&E MY-SG NRG/SICU 2 Experiment (Down)SG-US SingaREN 90 RÐ-US Uninet 155 R&E TW-HK ASNET/TANET/TWAREN 622 R&ETW-JP ASNET/TANET 622 R&ETW-SG ASNET/SingAREN 155 R&ETW-US ASNET/TANET/TWAREN 6.6 Gbps R&E(TW)-US-NL ASNET/TANET/TWAREN 2.5 Gbps R&E
APAN Recommendations(at July 2004 Meeting in CAIRNS, Au)
APAN Recommendations(at July 2004 Meeting in CAIRNS, Au)
Central Issues for APAN this decadeCentral Issues for APAN this decade Stronger linkages between applications and infrastructure - Stronger linkages between applications and infrastructure -
neither can exist independentlyneither can exist independently Stronger application and infrastructure linkages among APAN Stronger application and infrastructure linkages among APAN
members. members. Continuing focus on APAN as an organization that represents Continuing focus on APAN as an organization that represents
infrastructure interests in Asiainfrastructure interests in Asia Closer connection between APAN the infrastructure & Closer connection between APAN the infrastructure &
applications organization and regional political organizations applications organization and regional political organizations (e.g. APEC, ASEAN)(e.g. APEC, ASEAN)
New issues demand attentionNew issues demand attention Application measurement, particularly end-to-end network Application measurement, particularly end-to-end network
performance measurement is increasingly critical (deterministic performance measurement is increasingly critical (deterministic networking)networking)
Security must now be a consideration for every application and Security must now be a consideration for every application and every network.every network.
KR-US/CA Transpacific connectionKR-US/CA Transpacific connection
Participation in Global-scale Lambda NetworkingParticipation in Global-scale Lambda Networking Two STM-4 circuits (1.2G) : KR-CA-USTwo STM-4 circuits (1.2G) : KR-CA-US Global lambda networking : North America, Europe, Global lambda networking : North America, Europe,
Asia Pacific, etc.Asia Pacific, etc.
STM-4 * 2
Seattle
APII-testbed/KREONet2
KREONET/SuperSIReNCA*Net4
Global Lambda Networking
StarLight
Chicago
PacWave
New Record!!! 916 Mbps from CHEP to Caltech New Record!!! 916 Mbps from CHEP to Caltech (22/06/’04)(22/06/’04)
Subject: UDP test on KOREN-TransPAC-CaltechSubject: UDP test on KOREN-TransPAC-Caltech
Date: Tue, 22 Jun 2004 13:47:25 +0900Date: Tue, 22 Jun 2004 13:47:25 +0900
From: "Kihwan Kwon" <[email protected]>From: "Kihwan Kwon" <[email protected]>
To: <[email protected]>To: <[email protected]>
[root@sul Iperf]# ./iperf -c socrates.cacr.caltech.edu -u -b 1000m [root@sul Iperf]# ./iperf -c socrates.cacr.caltech.edu -u -b 1000m
------------------------------------------------------------------------------------------------------------------------
Client connecting to socrates.cacr.caltech.edu, UDP port 5001Client connecting to socrates.cacr.caltech.edu, UDP port 5001
Sending 1470 byte datagrams; UDP buffer size: 64.0 KByte (default)Sending 1470 byte datagrams; UDP buffer size: 64.0 KByte (default)
------------------------------------------------------------------------------------------------------------------------
[ 5] local 155.230.20.20 port 33036 connected with 131.215.144.227[ 5] local 155.230.20.20 port 33036 connected with 131.215.144.227
[ ID] Interval Transfer Bandwidth[ ID] Interval Transfer Bandwidth
[ 5] 0.0-2595.2 sec 277 GBytes 916 Mbits/sec[ 5] 0.0-2595.2 sec 277 GBytes 916 Mbits/sec
USA TransPACUSA TransPAC
G/H-JapanG/H-JapanKNU/KoreaKNU/Korea
Max. 947.3Mbps
OC3 circuits Moscow-Chicago-OC3 circuits Moscow-Chicago-Beijing since January 2004Beijing since January 2004
OC3 circuit Moscow-Beijing July OC3 circuit Moscow-Beijing July 2004 (completes the ring)2004 (completes the ring)
Korea (KISTI) joining US, Russia, Korea (KISTI) joining US, Russia, China as full partner in GLORIADChina as full partner in GLORIAD
Plans developing for Central Asian Plans developing for Central Asian extension (w/Kyrgyz Government)extension (w/Kyrgyz Government)
Rapid traffic growth with heaviest Rapid traffic growth with heaviest US use from DOE (FermiLab), US use from DOE (FermiLab), NASA, NOAA, NIH and Universities NASA, NOAA, NIH and Universities (UMD, IU, UCB, UNC, UMN, PSU, (UMD, IU, UCB, UNC, UMN, PSU, Harvard, Stanford, Wash., Oregon, Harvard, Stanford, Wash., Oregon, 250+ others) 250+ others)
GLORIAD 5-year Proposal Pending (with US NSF) for expansion: 2.5G Moscow-Amsterdam-Chicago-Seattle-Hong Kong-Pusan-Beijing circuits early 2005; 10G ring around northern hemisphere 2007; multiple wavelength service 2009 – providing hybrid circuit-switched (primarily Ethernet) and routed services
> 5TBytes now transferred monthly via GLORIAD to US, Russia, China
Global Ring Network for Advanced Applications Development
Aug. 8 2004: P.K. Young, Korean IST
Advisor to President Announces
Korea Joining GLORIAD TEIN gradually to 10G, connected to GLORIAD Asia Pacific Info. Infra- Structure (1G) will be
backup net to GLORIAD
Internet in China (J.P.Wu APAN July 2004)
Internet users in China: from 6.8 Million to 78 Million within 6 months
IP Addresses: 32M ( 1A+233B+146C )Backbone : 2.5-10G DWDM+RouterInternational links : 20GExchange Points : > 30G ( BJ , SH , GZ )Last Miles Ethernet , WLAN , ADSL , CTV , CDMA , ISDN
, GPRS , Dial-upNeed IPv6
TotalWirelin
eDial Up ISDN
Broadband
68M 23.4M 45.0M 4.9M 9.8M
China: CERNET Update1995, 64K Nation wide backbone connecting 8 cities, 100 Universities1998, 2M Nation wide backbone connecting 20 cities, 300 Universities2000, Own dark fiber crossing 30+ major cities and 30,000 kilometers2001, CERNET DWDM/SDH network finished2001, 2.5G/155M Backbone connecting 36 cities, 800 universities2003,1300 + universities and institutes, over 15 million users
CERNET2 and Key TechnologiesCERNET 2: Next Generation Education and Research Network in ChinaCERNET 2 Backbone connecting 15-20 GigaPOPs at 2.5G-10Gbps (I2-like Model)Connecting 200 Universities and 100+ Research Institutes at 1Gbps-10GbpsNative IPv6 and Lambda Networking Support/Deployment of the following technologies: E2E performance monitoring Middleware and Advanced Applications Multicast
AFRICA: Key TrendsAFRICA: Key Trends
Growth in traffic and lack of infrastructure Growth in traffic and lack of infrastructure Predominance of Satellite;Predominance of Satellite;But these satellites are heavily subscribed But these satellites are heavily subscribed
Int’l Links: Only ~1% of traffic on links is for Internet connections;Int’l Links: Only ~1% of traffic on links is for Internet connections;Most Internet traffic (for ~80% of countries) via satelliteMost Internet traffic (for ~80% of countries) via satellite Flourishing Grey market for Internet & VOIP traffic using VSAT dishesFlourishing Grey market for Internet & VOIP traffic using VSAT dishes
Many Regional fiber projects in “planning phase” (some languished in Many Regional fiber projects in “planning phase” (some languished in the past); Only links from South Africa to Nimibia, Botswana done so far the past); Only links from South Africa to Nimibia, Botswana done so far
Int’l fiber Project: SAT-3/WASC/SAFE Cable from South Africa to PortugalInt’l fiber Project: SAT-3/WASC/SAFE Cable from South Africa to PortugalAlong West Coast of AfricaAlong West Coast of Africa Supplied by Alcatel to Worldwide Consortium of 35 CarriersSupplied by Alcatel to Worldwide Consortium of 35 Carriers 40 Gbps by Mid-2003; Heavily Subscribed. Ultimate Capacity 120 Gbps40 Gbps by Mid-2003; Heavily Subscribed. Ultimate Capacity 120 Gbps Extension to Interior Mostly by Satellite: < 1 Mbps to ~100 Mbps typical Extension to Interior Mostly by Satellite: < 1 Mbps to ~100 Mbps typical
M. Jensen and P. Hamilton Infrastructure Report, March 2004
Note: World Conference on Physics and Sustainable Development,
10/31 – 11/2/05 in Durban South Africa; Part of World Year of Physics 2005.
Sponsors: UNESCO, ICTP, IUPAP, APS, SAIP
AFRICA:AFRICA: Nectar Net Initiative Nectar Net Initiative
Growing Need to connect academic researchers, medicalGrowing Need to connect academic researchers, medicalresearchers & practitioners to many sites in Africaresearchers & practitioners to many sites in Africa
ExamplesExamples: : CDC & NIH: Global AIDS Project, Dept. of Parasitic Diseases,CDC & NIH: Global AIDS Project, Dept. of Parasitic Diseases,
Nat’l Library of Medicine (Ghana, Nigeria)Nat’l Library of Medicine (Ghana, Nigeria) Gates $ 50M HIV/AIDS Center in BotswanaGates $ 50M HIV/AIDS Center in Botswana; Project Coord at Harvard; Project Coord at Harvard Africa Monsoon AMMA Project, Dakar Site [cf. East US Hurricanes]Africa Monsoon AMMA Project, Dakar Site [cf. East US Hurricanes] US Geological Survey: Global Spatial Data InfrastructureUS Geological Survey: Global Spatial Data Infrastructure Distance Learning: Emory-Ibadan (Nigeria); Research Distance Learning: Emory-Ibadan (Nigeria); Research
Channel ContentChannel ContentBut Africa is HardBut Africa is Hard: : 11M Sq. Miles, 600 M People, 54 Countries11M Sq. Miles, 600 M People, 54 Countries
Little Little Telecommunications InfrastructureTelecommunications Infrastructure
Approach:Approach: Use SAT-3/WASC Cable (to Portugal), GEANT Use SAT-3/WASC Cable (to Portugal), GEANT Across Europe, Amsterdam-NY Link Across the Atlantic, Across Europe, Amsterdam-NY Link Across the Atlantic, then Peer with R&E Networks such as Abilene in NYCthen Peer with R&E Networks such as Abilene in NYC Cable Landings in 8 West African Countries and South AfricaCable Landings in 8 West African Countries and South Africa Pragmatic approach to reach end points: Pragmatic approach to reach end points:
VSAT satellite, ADSL, microwave, etc.VSAT satellite, ADSL, microwave, etc.W. MatthewsGeorgia Tech
Sample Bandwidth Costs for African Universities
$0.27
$3.00
$6.77
$9.84
$11.03
$20.00
$0.00 $5.00 $10.00 $15.00 $20.00 $25.00
USA
IBAUD Target
Ghana
Uganda
Average
Nigeria
$/kbps/month
Sample size of 26 universitiesAverage Cost for VSAT service: Quality, CIR, Rx, Tx not distinguished
Bandwidth prices in Africa vary dramatically; are in general many times what they could be if universities purchase in volume
Roy Steiner Internet2 Workshop
27 sites (U.S., Korea) 2300-2800 CPUs 700-1100 Concurrent Jobs
Korea
www.ivdgl.org/grid2003
Grid2003: An Operational Grid2003: An Operational Production Grid, Since October 2003Production Grid, Since October 2003
Trillium: PPDGGriPhyNiVDGL
Prelude to Open Science Grid: www.opensciencegrid.org
The original Computational and Data Grid concepts are The original Computational and Data Grid concepts are largely stateless, open systems largely stateless, open systems
Analogous to the WebAnalogous to the WebThe classical Grid architecture had a number of implicit The classical Grid architecture had a number of implicit
assumptionsassumptions The ability to locate and schedule suitable resources,The ability to locate and schedule suitable resources,
within a tolerably short time (i.e. resource richness) within a tolerably short time (i.e. resource richness) Short transactions with relatively simple failure modesShort transactions with relatively simple failure modes
HENP Grids are HENP Grids are Data Intensive & Resource-ConstrainedData Intensive & Resource-Constrained Resource usage governed by local and global policiesResource usage governed by local and global policies Long transactions; some long queuesLong transactions; some long queues Analysis:Analysis: 1000s of users competing for resources 1000s of users competing for resources at dozens of sites: complex scheduling; management at dozens of sites: complex scheduling; management
HENP HENP Stateful, End-to-end Monitored and Tracked ParadigmStateful, End-to-end Monitored and Tracked Paradigm Adopted in OGSA [Now WS Resource Framework]Adopted in OGSA [Now WS Resource Framework]
HENP Data Grids, and Now HENP Data Grids, and Now Services-Oriented GridsServices-Oriented Grids
The Move to OGSA and then The Move to OGSA and then Managed Integrated SystemsManaged Integrated Systems
Incr
ease
d f
un
ctio
nal
ity,
stan
dar
diz
atio
n
Time
Customsolutions
Open GridServices Arch
GGF: OGSI, …(+ OASIS, W3C)
Multiple implementations,including Globus Toolkit
Web services + …
Globus Toolkit
Defacto standardsGGF: GridFTP, GSI
X.509,LDAP,FTP, …
App-specificServices~~Integrated SystemsIntegrated Systems
Stateful; ManagedWeb Services
Resrc Framwk
Managing Global Systems: Dynamic Managing Global Systems: Dynamic Scalable Services ArchitectureScalable Services Architecture
MonALISA: http://monalisa.cacr.caltech.edu
““Station Server” Station Server” Services-enginesServices-engines at sites host many at sites host many“Dynamic Services”“Dynamic Services” Scales to Scales to
thousands of thousands of service-Instancesservice-Instances
Servers autodiscover Servers autodiscover and interconnect and interconnect dynamically to form dynamically to form a robust fabric a robust fabric
Autonomous agentsAutonomous agents
+ CLARENS: Web Services Fabric and Portal Architecture
24 24 X 7 OperationsX 7 OperationsMultiple Orgs.Multiple Orgs.
Grid2003Grid2003 US CMSUS CMS CMS-DC04CMS-DC04 ALICEALICE STARSTAR VRVSVRVS ABILENEABILENE Soon: GEANTSoon: GEANT + GLORIAD+ GLORIAD
52Caltech GAE Team
Grid Analysis EnvironmentGrid Analysis Environment
Analysis Clients talk standard protocols to the CLARENS “Grid Services Web Server”, with a simple Web service API
The secure Clarens portal hides the complexity of the Grid Services from the client
Key features: Global Scheduler, Catalogs, Monitoring, and Grid-wide Execution service;Clarens servers forma Global Peer to peer Network
SchedulerCatalogs
Analysis Client
Grid ServicesWeb Server
ExecutionPriority
Manager
Grid WideExecutionService
DataManagement
Fully-ConcretePlanner
Fully-AbstractPlanner
Analysis Client
AnalysisClient
Virtual Data
Replica
ApplicationsMonitoring
Partially-AbstractPlanner
Metadata
HTTP, SOAP, XML/RPC
CLARENS: Web Services Architecture
World Summit on the Information Society(WSIS): Geneva 12/2003 and Tunis in 2005 World Summit on the Information Society
(WSIS): Geneva 12/2003 and Tunis in 2005 The UN General Assembly adopted in 2001 a resolution The UN General Assembly adopted in 2001 a resolution
endorsing the organization of the endorsing the organization of the World Summit on the World Summit on the Information Society (WSIS),Information Society (WSIS), under UN Secretary-General, under UN Secretary-General, Kofi Annan, with the ITU and host governments taking Kofi Annan, with the ITU and host governments taking the lead role in its preparation. the lead role in its preparation.
GOAL: To Create an Information Society:GOAL: To Create an Information Society: A Common Definition was adoptedA Common Definition was adoptedin the “Tokyo Declaration” of January 2003:in the “Tokyo Declaration” of January 2003:“… One in which highly developed ICT networks, equitable “… One in which highly developed ICT networks, equitable and ubiquitous access to information, appropriate content and ubiquitous access to information, appropriate content in accessible formats and effective communication can in accessible formats and effective communication can help people achieve their potential”help people achieve their potential”
Kofi Annan Challenged the Scientific Community to Help (3/03) Kofi Annan Challenged the Scientific Community to Help (3/03) CERN and ICFA SCIC have been quite active in the WSIS in CERN and ICFA SCIC have been quite active in the WSIS in
Geneva (12/2003) Geneva (12/2003)
Role of Science in the Information Society; WSIS 2003-2005
Role of Science in the Information Society; WSIS 2003-2005
HENP Active in WSIS HENP Active in WSIS CERN RSIS EventCERN RSIS EventSIS Forum & CERN/Caltech SIS Forum & CERN/Caltech
Online Stand at WSIS I Online Stand at WSIS I (Geneva 12/03) (Geneva 12/03)
Visitors at WSIS I Visitors at WSIS I Kofi AnnanKofi Annan, UN Sec’y General , UN Sec’y General John H. MarburgerJohn H. Marburger, ,
Science Adviser to US PresidentScience Adviser to US PresidentIon IliescuIon Iliescu, President of Romania; , President of Romania;
and Dan Nica, Minister of ICTand Dan Nica, Minister of ICTJean-Paul Hubert, Ambassador Jean-Paul Hubert, Ambassador
of Canada in Switzerland of Canada in Switzerland……
Planning Underway forPlanning Underway for WSIS II: Tunis 2005 WSIS II: Tunis 2005
NEWS: Bulletin: ONE TWOWELCOME BULLETIN General InformationRegistrationTravel Information Hotel RegistrationParticipant List How to Get UERJ/Hotel Computer AccountsUseful Phone Numbers ProgramContact us: Secretariat Chairmen
CLAF CNPQ FAPERJ UERJ
SPONSORS
HEPGRID and Digital Divide Workshop
UERJ, Rio de Janeiro, Feb. 16-20 2004Theme: Global Collaborations, Grids
and Their Relationship to the Digital Divide
ICFA, understanding the vital role of these issues for our field’s future, commissioned the Standing Committee on Inter-regional Connectivity (SCIC) in 1998, to survey and
monitor the state of the networks used by our field, and identify problems. For the past
three years the SCIC has focused on understanding and seeking the means of reducing or eliminating the Digital Divide, and proposed to ICFA that these issues, as
they affect our field of High Energy Physics, be brought to our community for discussion. This led to ICFA’s approval, in July 2003, of the Digital Divide and HEP Grid Workshop.
More Information: http://www.lishep.uerj.br
TutorialsC++Grid TechnologiesGrid-Enabled
AnalysisNetworksCollaborative Systems
Sessions &Tutorials Available
(w/Video) on the Web
Computing Model Progress CMS Internal Review of Software and Computing
International ICFA Workshop on HEP International ICFA Workshop on HEP Networking, Grids and Digital Divide Networking, Grids and Digital Divide Issues for Global e-ScienceIssues for Global e-Science
Proposed Workshop Dates: May 23-27, 2005
Venue: Daegu, Korea
Dongchul SonCenter for High Energy PhysicsKyungpook National University
ICFA, Beijing, ChinaAug. 2004 ICFA Approval
Requested Today
고에너지물리연구센터 CENTER FOR HIGH ENERGY PHYSICS
International ICFA Workshop on HEP International ICFA Workshop on HEP Networking, Grids Networking, Grids and Digital Divide Issues for Global e-Scienceand Digital Divide Issues for Global e-Science
ThemesNetworking, Grids, and Their Relationship to the Digital Divide for
HEP as Global e-ScienceFocus on Key Issues of Inter-regional Connectivity
Mission Statement
ICFA, understanding the vital role of these issues for our field’s future, commissioned the Standing Committee on Inter-regional Connectivity (SCIC) in 1998, to survey and monitor the state of the networks used by our field, and identify problems. For the past three years the SCIC has focused on understanding and seeking the means of reducing or eliminating the Digital Divide, and proposed to ICFA that these issues, as they affect our field of High Energy Physics, be brought to our community for discussion. This workshop, the second in the series begun with the the 2004 Digital Divide and HEP Grid Workshop in Rio de Janeiro (approved by ICFA in July 2003) will carry forward this work while strengthening the involvement of scientists, technologists and governments in the Asia Pacific region.
고에너지물리연구센터 CENTER FOR HIGH ENERGY PHYSICS
International ICFA Workshop on HEP International ICFA Workshop on HEP Networking, Grids Networking, Grids and Digital Divide Issues for Global e-Scienceand Digital Divide Issues for Global e-Science
Workshop Goals Review the current status, progress and barriers to the effective use
of the major national, continental and transoceanic networks used by HEP
Review progress, strengthen opportunities for collaboration, and explore the means to deal with key issues in Grid computing and Grid-enabled data analysis, for high energy physics and other fields of data intensive science, now and in the future
Exchange information and ideas, and formulate plans to develop solutions to specific problems related to the Digital Divide in various regions, with a focus on Asia Pacific, Latin America, Russia and Africa
Continue to advance a broad program of work on reducing or eliminating the Digital Divide, and ensuring global collaboration, as related to all of the above aspects.
Networks and Grids, GLORIAD, ITER and HENP
Networks and Grids, GLORIAD, ITER and HENP
Network backbones and major links used by major experimentsNetwork backbones and major links used by major experiments in HENP and other fields are advancing rapidly in HENP and other fields are advancing rapidly
To the 10 G range in < 2 years; much faster than Moore’s LawTo the 10 G range in < 2 years; much faster than Moore’s Law New HENP and DOE Roadmaps: a factor ~1000 improvement per decadeNew HENP and DOE Roadmaps: a factor ~1000 improvement per decade
We are learning to use long distance 10 Gbps networks effectively We are learning to use long distance 10 Gbps networks effectively 2003-2004 Developments: to 7.5 Gbps flows over 16 kkm2003-2004 Developments: to 7.5 Gbps flows over 16 kkm Important advances in Asia-Pacific, notably KoreaImportant advances in Asia-Pacific, notably Korea
A transition to community-owned and operated R&E networks A transition to community-owned and operated R&E networks is beginning (us, ca, nl, pl, cz, sk …) or considered (de, ro, …) is beginning (us, ca, nl, pl, cz, sk …) or considered (de, ro, …)
We Must Work to Close to Digital DivideWe Must Work to Close to Digital Divide Allowing Scientists and Students from All World Regions Allowing Scientists and Students from All World Regions
to Take Part in Discoveries at the Frontiers of Science to Take Part in Discoveries at the Frontiers of Science Removing Regional, Last Mile, Local Bottlenecks and Removing Regional, Last Mile, Local Bottlenecks and
Compromises in Network Quality are now On Compromises in Network Quality are now On the Critical Path the Critical Path GLORIADGLORIAD is A Key Project to Achieve These Goals is A Key Project to Achieve These Goals
Synergies Between the Data-Intensive Missions of HENP & ITERSynergies Between the Data-Intensive Missions of HENP & ITER Enhancing Partnership and Community among the US, Russia Enhancing Partnership and Community among the US, Russia
and China: both in Science and Education and China: both in Science and Education
SC2004: HEP Network LayoutPreview of Future Grid systems
SC2004: HEP Network LayoutPreview of Future Grid systems
Joint Caltech, CERN, SLAC, Joint Caltech, CERN, SLAC, FNAL, UKlight, HP, Cisco… DemoFNAL, UKlight, HP, Cisco… Demo
6 to 8 10 Gbps waves to HEP6 to 8 10 Gbps waves to HEP setup on the show floor setup on the show floor
Bandwidth challenge: aggregate Bandwidth challenge: aggregate throughput goal throughput goal of 40 to 60 Gbps of 40 to 60 Gbps
3*10GbpsTeraGrid
Australia
JapanBrazil
SLACUK
10 GbpsAbilene
10 GbpsNLR
2*10 GbpsNLR
Caltech CACR CERN
Geneva
StarLight
LA
10 GbpsLHCNet 2 Metro
10 Gbps Waves LA-Caltech
FNAL
SCIC in 2003-2004http://cern.ch/icfa-scic
SCIC in 2003-2004http://cern.ch/icfa-scic
Strong Focus on the Strong Focus on the Digital DivideDigital Divide Continues Continues A Striking Picture Continues to Emerge: Remarkable A Striking Picture Continues to Emerge: Remarkable
Progress in Some Regions, and a Deepening Digital Progress in Some Regions, and a Deepening Digital Divide Among NationsDivide Among Nations
IntensiveIntensive Work in the Field: > 60 Meetings and Workshops: Work in the Field: > 60 Meetings and Workshops: E.g., Internet2, TERENA, AMPATH, APAN, CHEP2003, SC2003, E.g., Internet2, TERENA, AMPATH, APAN, CHEP2003, SC2003,
Trieste, Telecom World 2003, SC2003, WSIS/RSIS, GLORIAD Trieste, Telecom World 2003, SC2003, WSIS/RSIS, GLORIAD LaunchLaunch, Digital Divide and HEPGrid Workshop (Feb. 16-20 in , Digital Divide and HEPGrid Workshop (Feb. 16-20 in Rio), Rio), GNEW2004, GridNets2004, NASA ONT Workshop, … etc. GNEW2004, GridNets2004, NASA ONT Workshop, … etc.
33rdrd Int’l Grid Workshop in Daegu (August 26-28, 2004); Plan for Int’l Grid Workshop in Daegu (August 26-28, 2004); Plan for 22ndnd ICFA Digital Divide and Grid Workshop in Daegu (May 2005) ICFA Digital Divide and Grid Workshop in Daegu (May 2005)
HENP increasingly visible to governments; heads of state:HENP increasingly visible to governments; heads of state: Through Network advances (records), Grid developments,Through Network advances (records), Grid developments,
Work on the Digital Divide and issues of Global Collaboration Work on the Digital Divide and issues of Global Collaboration Also through the World Summit on the Information Society Also through the World Summit on the Information Society
Process. Next Step is Process. Next Step is WSIS II in TUNIS November 2005WSIS II in TUNIS November 2005
Coverage Now monitoring 650 sites in 115 countries In last 9 months added:
Several sites in Russia (thanks GLORIAD)Many hosts in Africa (5 36 now; in 27 out of 54 countries)Monitoring sites in Pakistan and Brazil (Sao Paolo and Rio)
Working to install monitoring host in Bangalore, India
Monitoring siteRemote site
Latin America: CLARA Network (2004-2006 EU Project)
Significant contribution from European Comission and Dante through ALICE project
NRENs in 18 LA countries forming a regional network for collaboration traffic
Initial backbone ring bandwidth f 155 Mbps
Spur links at 10 to 45 Mbps (Cuba at 4 Mbps by satellite)
Initial connection to Europe at 622 Mbps from Brazil
Tijuana (Mexico) PoP soon to be connected to US through dark fibre link (CUDI-CENIC) access to US, Canada and Asia -
Pacific Rim
To East Coast
ClaraClara
To West Coast
to West Coast to East Coast
to Europ
eNSF IRNC 2004: Two Proposals to
Connect CLARA to the US (and Europe)1st Proposal:
FIU and CENIC2nd Proposal:
Indiana and Internet2
Note: CHEPREO (FIU, UF, FSU Caltech, UERJ, USP, RNP)
622 Mbps Sao Paolo – Miami Link Started in August
GIGA Project: Experimental Gbps Network: Sites in Rio and Sao Paolo
FapesptelcosUnesp
USP – IncorUSP - C.Univ.
CPqDLNLS
Unicamp
LNCC
CPTEC UFF
CTAINPE
CBPFLNCC
FiocruzIME
IMPA-RNP
PUC-Rio telcosUERJUFRJ
UniversitiesIMEPUC-RioUERJUFFUFRJUnespUnicampUSP
R&D CentresCBPF - physicsCPqD - telecomCPTEC - meteorologyCTA - aerospaceFiocruz - healthIMPA - mathematicsINPE - space sciencesLNCC - HPCLNLS - physics
About 600 km extension - not to scale
Slide from M. Stanton
Trans-Eurasia Information NetworkTEIN (2004-2007)
Trans-Eurasia Information NetworkTEIN (2004-2007)
Circuit between KOREN(Korea) and RENATER(France). Circuit between KOREN(Korea) and RENATER(France). AP Beneficiaries: AP Beneficiaries:
China, Indonesia, Malaysia, Philippines, Thailand, VietnamChina, Indonesia, Malaysia, Philippines, Thailand, Vietnam (Non-beneficiaries: (Non-beneficiaries: Brunei, Japan, Korea, SingaporeBrunei, Japan, Korea, Singapore EU partners: EU partners: NRENs of France, Netherlands, UKNRENs of France, Netherlands, UK The scope expanded to South-East Asia and China recently. The scope expanded to South-East Asia and China recently. Upgraded to 34 Mbps in 11/2003. Upgrade to 155Mbps planned Upgraded to 34 Mbps in 11/2003. Upgrade to 155Mbps planned
12M Euro EU Funds12M Euro EU Funds Coordinating PartnerCoordinating Partner
DANTE DANTE Direct EU-AP Link; Direct EU-AP Link;
Other Links go Other Links go Across the USAcross the US
APAN China ConsortiumAPAN China Consortium Has been established in 1999. The China Education and Research Has been established in 1999. The China Education and Research
Network (Network (CERNETCERNET), the Natural Science Foundation of China ), the Natural Science Foundation of China Network (Network (NSFCNETNSFCNET) and the China Science and Technology ) and the China Science and Technology Network (Network (CSTNETCSTNET) are the main three advanced networks. ) are the main three advanced networks.
CERNet
2.5 Gbps
NSFCnetTsinghua --- Tsinghua UniversityPKU --- Peking UniversityNSFC --- Natural Science Foundation of ChinaCAS --- China Academy of SciencesBUPT --- Beijing Univ. of Posts and Telecom.BUAA --- Beijing Univ. of Aero- and Astronautics
GLORIAD: Global Optical Ring (US-Ru-Cn; Korea Now Full Partner )
GLORIAD: Global Optical Ring (US-Ru-Cn; Korea Now Full Partner )
Also Important for Intra-Russia Connectivity;Education and Outreach
DOE: ITER Distributed Ops.;Fusion-HEP Cooperation
NSF: Collaboration of ThreeMajor R&E Communities
Aug. 8 2004: P.K. Young, Korean IST
Advisor to President Announces
Korea Joining GLORIAD TEIN gradually to 10G, connected to GLORIAD Asia Pacific Info. Infra- Structure (1G) will be
backup net to GLORIAD
GLORIAD and HENP Example:GLORIAD and HENP Example:Network Needs of IHEP Beijing Network Needs of IHEP Beijing
ICFA SCIC Report: Appendix 18, on Network Needs ICFA SCIC Report: Appendix 18, on Network Needs for HEP in China (See for HEP in China (See http://cern.ch/icfa-scichttp://cern.ch/icfa-scic))
““IHEP is working with the Computer Network Information Center (CNIC) and other universities and institutes to build Grid applications for the experiments. The computing resources and storage management systems are being built or upgraded in the Institute. IHEP has a 100 Mbps link to CNIC, so it is quite easy
to connect to GLORIAD and the link could be upgraded as needed.” Prospective Network Needs for IHEP Bejing
Experiment Year 2004-2005 Year 2006 and on
LHC/LCG 622Mbps 2.5Gbps
BES 100Mbps 155Mbps
YICRO 100Mbps 100Mbps
AMS 100Mbps 100Mbps
Others 100Mbps 100Mbps
Total (Sharing) 1Gbps 2.5Gbps
Role of Sciences in Information Society. Palexpo, Geneva 12/2003
Role of Sciences in Information Society. Palexpo, Geneva 12/2003
Demos at the CERN/Caltech Demos at the CERN/Caltech RSIS Online StandRSIS Online Stand
Advanced network and Advanced network and Grid-enabled analysis Grid-enabled analysis
Monitoring very large scale Grid Monitoring very large scale Grid farms with MonALISA farms with MonALISA
World Scale multisite multi-protocol World Scale multisite multi-protocol videoconference with VRVSvideoconference with VRVS
(Europe-US-Asia-South America) (Europe-US-Asia-South America) Distance diagnosis and surgery using Distance diagnosis and surgery using
Robots with “haptic” feedback Robots with “haptic” feedback (Geneva-Canada)(Geneva-Canada)
Music Grid: live performances with Music Grid: live performances with bands at St. John’s, Canada and the bands at St. John’s, Canada and the Music Conservatory of Geneva on Music Conservatory of Geneva on stagestage
VRVS 37k hostsVRVS 37k hosts106 Countries106 Countries
2-3X Growth/Year2-3X Growth/Year