Layer 2 High Performance Networking
Challenges and Breakthroughs: Lessons From
GLIF 2012 and SC12 Demonstrations
Marc Lyonnais, Ciena
Joe Mambretti International Center for Advanced Internet Research Northwestern Univ
Linda Winkler, Argonne National Laboratory
Special Collaboration: Rod Wilson, Ciena
TIP 2013
January 14, 2013
2
Introduction
This presentation describes experiments and demonstrations staged
during the Global Lambda Grid Workshop (GLIF2012) and the SC12
Supercomputing Conference.
These activities resulted from a collaboration among Ciena, iCAIR at
Northwestern, Argonne National Lab, the StarLight Consortium, the
Metropolitan Research and Education Network (MREN), and multiple
other research partners.
These experiments and demonstrations utilized software and
hardware prototypes based on closely integrated Ethernet and
photonic transport technologies.
The demonstrations presented techniques designed to scale and to
reduce network complexity of SDN at 100G Rates.
These techniques are still In early stages of deployment.
3
In This Presentation…
We Will Look Into:
What drives bandwidth at and beyond 100G.
Complexities observed at this early stage of building next generation
Optical Open Exchanges
What should be done to scale Networks and Open Exchanges?
What are the next steps in this process?
What Drives Bandwidth At and Beyond
100G? – Data Intensive Science
5
Sloan Digital Sky
Survey
www.sdss.org
Globus Alliance
www.globus.org
LIGO
www.ligo.org
ALMA: Atacama
Large Millimeter Array
www.alma.nrao.edu
CAMERA
metagenomics
camera.calit2.net
Comprehensive
Large-Array
Stewardship System
www.class.noaa.gov
DØ (DZero)
www-d0.fnal.gov
ISS: International
Space Station
www.nasa.gov/station
IVOA: International
Virtual Observatory
www.ivoa.net
BIRN: Biomedical
Informatics Research
Network
www.nbirn.net
GEON: Geosciences
Network
www.geongrid.org
ANDRILL:
Antarctic
Geological
Drilling
www.andrill.org
GLEON: Global Lake
Ecological
Observatory Network
www.gleon.org Pacific Rim
Applications and Grid
Middleware Assembly
www.pragma-grid.net
CineGrid
www.cinegrid.org
Carbon Tracker
www.esrl.noaa.gov/
gmd/ccgg/carbontracke
r
www.xsede.org
LHCONE
www.lhcone.net
WLCG
lcg.web.cern.ch/LCG/public/
OOI-CI
ci.oceanobservatories.org
OSG
www.opensciencegrid.org SKA
www.skatelescope.org
NG Digital
Sky Survey
ATLAS
Compilation By Maxine Brown
www.ligo.org
StarLight Supports All Major Data Intensive Science Projects
6
Global Lambda Integrated Facility: A World Wide Facility
Supporting Multiple Networks and Testbeds
Visualization courtesy of Bob Patterson, NCSA; data compilation by Maxine Brown, UIC. www.glif.is
7
GLIF 2012 Hosted By EVL and iCAIR, Supported By StarLight 14
Major Demonstrations, Including (4k Uncompressed Streaming
Transport)
Chicago
Ottawa
MEN LAB 10 • 40G Connectivity from Ciena
Ottawa to Starlight Gigapop in
Chicago
• 2 x10G DWDM link to EVL Lab
40G through 1100km
EVL Lab-Quad-HD
710NLSD UIC
DWDM
2x10G
8
High Performance Digital Media Network (HPDMnet): Dynamically
Provisioned Inter-Domain International Service for High Performance Digital Media and Other Data Intensive Applications
The HPDMnet Consortium is working with the GLIF
community on developing and implementing architecture for new types of advanced network services
The HPDMnet Consortium is designing and developing new dynamically provisioned L1/L2 capabilities for large scale HPDM services, which can be used for any data intensive application
Including world-wide L1/L2 services for high performance, high quality, large volume stream digital media
Key attributes: capacity, flexibility, quality, reliability, “green”
The Consortium is developing services specifically for implementation at GLIF GOLES, and related facilities
9
10
An Advanced International Distributed Programmable
Environment for Experimental Network Research:
“Slice Around the World” Demonstration
A Demonstration and Presentation By the
Consortium for International Advanced Network Research
Leads for Participating Organizations: Ilia Baldine, Andy Bavier, Scott Campbell, Jeff Chase,
Jim Chen, Cees de Laat, Dongkyun Kim, Te-Lung Liu, Luis Fernandez Lopez, Mon-Yen Lou,
Joe Mambretti, Rick McGeer, Paul Muller, Aki Nakao, Max Ott, Ronald van der Pol, Martin
Reed, Rob Ricci, Ruslan Smeliansky, Marcos Rogerio Salvador, Myung-Ki Shin, Michael
Stanton, Jungling Yu
11
Overview
This International Research Consortium is creating a large scale
distributed environment (platform) for basic network science
research, experiments, and demonstrations.
This initiative Is designing, implementing, and demonstrating an
international highly distributed environment (at global scale) that
can be used for advanced next generation communications and
networking.
The environment is much more than “A Network” – other resources
include programmable clouds, sensors, instruments, specialized
devices, and other resources.
This environment is the world’s most extensive international
OpenFlow research network.
The US National Science Foundation’s Global Environment for
Network Innovations (GENI) Is a major partner in this initiative.
13
14
16
Other SC12 Demonstrations At SC12, the 100 Gbps Consortium demonstrated multiple architectural
approaches, tools and methods related to using 100 Gbps channels
across multiple LAN and WAN environments.
Devices from multiple vendors integrating available components and
emerging beta components not yet commercialized.
The WAN environment included:
A 100 Gbps channel established on the Sidera Network between the
NASA Goddard Space Flight Center in Greenbelt Maryland, the Mid-
Atlantic Crossroads (MAX) on the east coast and the StarLight
International/National Communications Exchange in Chicago (the
MREN StarWave Facility)
Between StarLight On ESnet and the SC12 Conference Center in
Salt Lake City, interconnecting to a100 Gbps ring among booths on
the SC12 Showfloor.
17
100 Gbps Services for Petascale Science Project
Partners
NASA Goddard Space Flight Center
International Center for Advanced Internet Research (iCAIR) at
Northwestern University
MidAtlantic Crossroads (MAX)
Argonne National Laboratory
StarLight International/National Communications Exchange Facility
Sidera Networks
Ciena
The Metropolitan Research and Education Network (MREN)
Energy Science Network (ESnet)
Laboratory for Advanced Computing
The Open Cloud Consortium (OCC)
18
19
SC12 Version of Playing with Light Using Video
Sources and Sinks
CC
MD
12
SM
D
8x
2 W
SS
CC
MD
12
SM
D
8x
2 W
SS
Sink-Source
Sink-Source
A Software Define Network
that could reflect the Application
Interconnection at Layer 1
CC
MD
12
SM
D
8x
2 W
SS
Sink
20
Early Observations
High bandwidth requirements & deployments beyond 10G are increasing
rapidly to 100G.
100G capacity provides major opportunities for new services (i.e., more
than mere aggregation, increased L1/L2 deployments vs L3).
New optical technology is emerging: e.g, using phase modulation instead
of amplitude modulation when exceeding 10G.
This capacity is required for new segmentation techniques, e.g., Software
Define Networks (SDN), which allows for much more flexibility,
customization, new standards, new techniques (e.g., OpenFlow).
Demonstrations for GLIF 2012 and SC12 transiting the Starlight exchange
facility performed as expected – i.e, very well.
Last minute component provisioning posed some problems.
LR4 vs LR10 issues.
Testing paths using a test Set @ 100GE saved a lot of time.
Optimizations, e.g, L1 vs L2 scalability are still unclear.
What Issues Need To Be Resolved In
Order to Scale the Network and Open
Exchanges?
22
Layer1 v. Layer 2 as
Interconnection Option For Open Light Exchange
Unresolved to date: when are Layer 2 interconnections more optimal for an
Open Light Exchanges compared to Layer 1 interconnections? Both
approaches have positive attributes. Does one preclude the other?
Interconnecting many point to point sites brings up this question:
Should/can a simple OTN Layer 1 cross connection do the trick?
Or should Layer2, which Is VLAN sensitive In the intermediate interconnections
point e.g., at StarLight?
Layer 1 greatly reduces the complexity of interconnections, e.g., fewer steps.
Layer 2 provides stat muxing, but at 100G; is this needed? Yes, but primarily at
the endpoints really.
Should a hybrid fabric be implemented that introduces the flexibility of both for
the same network element?
Choice is highly dependent on app requirements.
More investigation of these issues is required – especially for scalability.
These are the questions we are looking at for SC13.
23
What will be the Submarine interconnect? OTN or Ethernet?
Visualization courtesy of Bob Patterson, NCSA; data compilation by Maxine Brown, UIC.
www.glif.is
24
Why OTN
OTN by definition, protects, the synchronization domain of the
payload.
In the initial interconnection of high bandwidth, we need to reflect on
how a slower clock rate will slowly influence the buffer outflow Rate.
In other words, if you fill the buffer with a slower speed than you are
emptying it, you will run into buffer overrun.
The idea? Same buffer rate in and out.
The Ethernet frame fits inside OTN. In fact, this is what is being used
in the transponders to transport Ethernet.
By using a Layer 1 protocol to transport or switch bandwidth, you may
be able to simplify your network point to point connections by
introducing SOME EPL circuit.
By using Ethernet switching at each end and using OTN throughout
the path, the interconnects are by far simpler to manage.
Conclusion
26
What We Talked About
Bandwidth at and beyond 100G is required.
Some solutions are emerging
However, complexities can be observed at this early stage of building
next generation optical open exchanges based on 100G services
Scaling L1/L2 services at open exchanges is a key issue
The next steps in this process
Implementing production 100G services
Preparing for more complex 100G demonstrations as proof points for
next generation 100G services at future events, including SC13
Thank you
Top Related