1

SINET3 L1-OndemandSINET3 L1-OndemandService InterfaceService Interface

MIddleware, 27th APAN MeetingMarch 3rd, 2009

Motonori Nakamura, Shigeo UrushidaniNational Institute of Informatics (NII)

SINET3: Science Information Network 3

SINET3 is the new Japanese academic backbone network launched in April 2007 for more than 700 universities and research institutions.

It has 63 edge and 12 core nodes and deploys Japan’s first 40 Gbps lines between Tokyo, Nagoya, and Osaka.

2

10 Gbps

10 Gbps622 M

bps

622 Mbps

: 40 Gbps: 10 to 20 Gbps: 1 to 20 Gbps: Core Node: Edge Node

Japan’s first 40 Gbps (STM256) lines

Los Angeles

New York

Tokyo

Nagoya

Osaka

3

Service Features in SINET3

SINET3 emphasizes four service aspects: transfer layer, virtual private network (VPN), quality-of-service (QoS), and bandwidth on demand.

It provides all services on a single network platform, and users can freely choose the best transfer layer services for their applications.

Services Examples

★ Multiple Layer Services • L3 (IP), L2 (Ethernet), & L1 (dedicated line)

★ Enriched VPN Services• Support for collaborative research among distant

sites with closed user group environment

★ Enhanced QoS Services • Support for performance-sensitive applications

★ Bandwidth-on-demand (BoD) Services

• Support for data-intensive applications

4

SINET3 provides bandwidth-on-demand (BoD) services as part of layer-1 services. Users can specify the destinations, duration, bandwidth with granularity of about

150Mbps, and route option, via simple Web pages. BoD server receives path setup requests from users, calculates the appropriate

routes, schedules accepted reservations, and triggers layer-1 path setup.

1 Gbps(13:00-14:00)

2 Gbps

(17:00-18:00)

1 Gbps

(15:00-16:00)

Bandwidth on Demand (BoD) Services

User

Web-based Interface(Destination, Duration, Bandwidth, & Route option)

On-demand layer-1 path

Layer-1 path setup trigger

SINET3

Layer-1BoD Server

Tokyo

Hokkaido

OsakaFukuoka

5

Architecture for BoD Services

L2MUX

GMPLS control and management plane

L1SW L1SWL1SWL1SW

Layer-1BoD Server

GMPLS

BoD server receives reservation requests, performs path calculation, schedules accepted requests, and triggers layer-1 path setup to source layer-1 switch.

Source layer-1 switch sets up layer-1 path toward destination using GMPLS. BoD server changes path bandwidth for L2/L3 traffic by LCAS via L1-OPS as

needed.

User

L2MUX

IP Router IP Router

Path setup trigger

Hitless bandwidthchange by LCAS

Destinations, Duration, Bandwidth, & Route Option

Path setup request

Scheduling

Path control

Route calculation

Resource managementF

ront

-end

IP

Ethernet

On-demand

L1-OPS

6

Service Parameters of L1 BoD Services

BoD server allows users to specify connection style + destinations, duration, bandwidth, & route option via Web-based interface.

VPN Extranet Public

Connection Style + Destinations : VPN-A

: Non-VPN

: VPN-B

Pre-configuredinterfaces

Duration

- Start Time &- Finish Time (in 15 minute intervals)

Bandwidth

GE

STM-16

STM-64

GE

10GE

VC-4-7v

VC-4-17v

VC-4-Av

VC-4-Bv 1 ≤ A ≤ 71 ≤ B ≤ 64

STM-64

Lambda (Full bandwidth) Bandwidth-specified

VC-4 Granularity (approx. 150 Mbps)Route Option

- “Minimum Delay” or- “Unspecified”

L1SW

Considerations on Path Calculation BoD server selects path (route and links) by taking into account following conditions.

(1) Each link has different available bandwidth for L1 services which varies over time. (2) Each link has different delay which is a fixed value.(3) There are parallel links between core nodes.(4) There are multiple routes between source and destination nodes

Link

Ban

dw

idth Available bandwidth for L1 services

(1) Available bandwidth for L1

L2/L3 Traffic Pattern

(3) Parallel Links (4) Multiple Routes

L1SW

RouterRouterLink Aggregation& Load Balancing

L1 Path

VCAT

Mon Tue Wed Thu Fri Sat Sun

Hiroshima Kyoto

Osaka Nagoya Tokyo1

Fukuoka

Matsuyama

Kanazawa

Tokyo2VCAT

1.05 Gbps (VC-4-7v)

0.6 Gbps (VC-4-4v)

0.45 Gbps (VC-4-3v)

Tokyo1 Tsukuba Sendai

Kanazawa

Tokyo2

Sapporo12ms

7ms

7ms

3ms 5ms1ms

(2) Delay

7

Doshisha

Univ.L1SW

Backbone Topology and Current BoD User Sites　

: Core L1SW

: Edge L1SW

Hiroshima

L1SWKyotoL1SW

FukuokaL1SW

Kanazawa

L1SW

Tokyo2L1SW

SapporoL1SW

TsukubaL1SW

SendaiL1SW

Matsuyama

L1SWNagoyaL1SW-1

OsakaL1SW-2

OsakaL1SW-1

Tokyo1L1SW-1

Tokyo1L1SW-2

Tokyo1L1SW-3

NagoyaL1SW-2

NAOJL1SW

OsakaUniv.L1SW

NIIL1SW

KEKL1SW

KyushuUniv.L1SW

Yamaguchi

Univ.L1SW

HokkaidoUniv.L1SW

NIFSL1SW

• SINET3 has 16 core layer-1 switches and 63 edge layer-1 switches, and has multiple routes and parallel links between core layer-1 switches

8

Sample Reservation Screen

9

createSNC REQ Path registration REQ

Path registration RESP

Path setup REQ

createSNC RESP

Path setup CMPLD

Path info retrieving REQ

Path info retrieving RESPNotification (create CMPLD)

Path setup RESP

L1-BoDServer

L1-OPS L1SW

deleteSNC REQ Path release REQ

deleteSNC RESP

Path release CMPLD

Path deregistration REQ

Path deregistration RESPNotification (delete CMPLD)

Path release RESP

getSNC REQ

getSNC RESP

Path info retrieving REQ

Path info retrieving RESP

Interface Between BoD Server and L1-OPS

CORBA(TMF-814)

TL1

L2MUX

GMPLS control and management plane

L1SW L1SWL1SWL1SW

Layer-1BoD Server

GMPLS

L2MUX

IP Router IP Router

L1-OPS

10

11

Current Projects using L1 BoD Services (1)

Three projects (eVLBI, high-quality remote backup, and new video communication) are using L1 BoD services.

High-quality remote backup projecteVLBI project

: 2.4 Gbps : 0.15G to 1 Gbps

: L1 Switch : L1 Switch

Detected Fringe (June 12th )

Yamaguchi

Gifu

Tsukuba

TomakomaiHokkaido Univ.

Kyushu Univ.Osaka Univ.

NII&NTT (Tokyo)

NAOJ (Tokyo)

NAOJ: National Astronomical Observatory of Japan

* VLBI: Very Long Baseline Interferometory

Current Projects using L1 BoD Services (2)

t-Room --- a room-sharing video system that allows people to simultaneously experience "distant space" and "remote time“. Users feel as if they are in the same room.

Folding the spaces of Kyoto, Atsugi, present, past onto the space where you are overlapping spaces (rooms) and overcoming time and space constraints.

PresentLocal

Room 1Room 1Room 2

Room 3Room 3

PastKyoto

PresentAtsugi

“Monolith” Building Module:side view (left) and

front view (right).

195 cm 142 cm

47 cm

65’’ LCD PanelPCs

HDV Camera

Effective Screen Size:142 cm x 80 cm

Pathway

65’’ LCD Panel

HDV Camera

3.0 m

Monolith

12

Examples of Path Setup/Release Time

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8 9 10 11 12 13 140123456789

1011

1 2 3 4 5 6 7 8 9 10 11 12 13 14

[min]

The number of transit switches

[min]

The number of transit switches

Pat

h se

tup/

rele

ase

time

Pat

h se

tup/

rele

ase

time

(17)

(17)

(17)

(17)

Setup (in parallel)

Setup (in series)Release (in parallel)Release (in series)

(7)

(4)

(1) (1)

(7)

(4)

(2)

(2)

(1)

1.05 Gbps (setup)600 Mbps (setup)150 Mbps (setup)

1.05 Gbps (release)

600 Mbps (release)

150 Mbps (release)

(a) e-VLBI project (b) High-quality remote backup project

(17)

Tsukuba - NAOJ

Gifu - NAOJ

Yamaguchi - NAOJ

NII – Hokkaido Univ.

Osaka Univ. – Hokkaido Univ.

Kyushu Univ. – Hokkaido Univ.

Setup(release) time was defined as the difference between the time at that BoD server sends “create(delete)SNC REQ” and the time at that it receives “notification (create(delete) CMPLD)”.

We first created each path in series after receiving “notification (create CMPLD)” but we refined the mechanism to create paths in parallel right after receiving “createSNC RESP.”

13

Layer-1BoD Server

More General Architecture for BoD Services

L2MUX

GMPLS Control and Management Plane

L1SW L1SWL1SWL1SWGMPLS

L1-OPS

Router Router

PC

HTTP(S)

Server

CORBA

Reservation-based Service(Destinations, Duration, Bandwidth, & Route Option)

Admission control,Scheduling

Path and bandwidth control

Path calculation

Resource managementF

ront

-end

BoDUsers

Signalling-based Service(Destination and Bandwidth)

L2MUX

Hitless BandwidthChange by LCAS

Path for L2/L3 Path for L2/L3 Path for L2/L3

We are planning to provide GMPLS-UNI-based services in addition to reservation-based services. Forwarding adjacency (FA) paths are preliminary established to manage the services.

BoD server receives the information of GMPLS-UNI paths via L1-OPS. If GMPLS-UNI paths are established on unexpected routes, BoD server forcibly tears down them.

IP

Ethernet

GMPLS-UNI

14

Open Issues

Admission control toward full-scale operations• If the total requested bandwidth exceeds the available bandwidth of a link, we try to rearrange pre-

assigned paths for “unspecified” routes to accommodate as many paths as possible.

• If the rearrangement fails, the BoD server informs the network operators about the situation. We seek negotiated solutions whereby network operators change the bandwidth and duration among users while we limit the number of users of the BoD services. We need an effective admission control algorithm that fairly selects from among the requests.

Improvement of layer-1 path setup/release times• We would like to improve the path setup/release times but this depends on the specifications of

vendor products.

Dissemination of BoD services to new scientific research areas • We would like to explore new scientific research areas which effectively utilize the properties (low

delay, no delay variance, and no packet losses ) of on-demand layer-1 paths.

15

References

1. S. Urushidani, J. Matsukata, K. Fukuda, S. Abe, Y. Ji, M. Koibuchi, S. Yamada, K. Shimizu, T. Takeda, I. Inoue, and K. Shiomoto, “Layer-1 bandwidth on demand services in SINET3,” IEEE Globecom 2007, Dec. 2007.

2. S. Urushidani, K. Fukuda, Y. Ji, S. Abe, M. Koibuchi, M. Nakamura, S. Yamada, K. Shimizu, R. Hayashi, I. Inoue, and K. Shiomoto, “Resource allocation and provision for bandwidth/networks on demand in SINET3,” 2nd IEEE International Workshop on Bandwidth on Demand, April 2008.

3. S. Urushidani, S. Abe, Y. Ji, K. Fukuda, M. Koibuchi, M. Nakamura, S. Yamada, R. Hayashi, I. Inoue, and K. Shiomoto, “Design of versatile academic infrastructure for multilayer network services,” IEEE Journal on Selected Areas in Communications, April 2009 (to appear).

4. S. Urushidani. K. Shimizu, R. Hayashi, H. Tanuma, K. Fukuda, Y. Ji, M. Koibuchi, S. Abe, M. Nakamura, S. Yamada, I. Inoue, and K. Shiomoto, “Implementation and evaluation of layer-1 bandwidth-on-demand capabilities in SINET3,” IEEE ICC2009, Jun. 2009 (to appear).

16