The Future of Optical Networking: Challenges, Pitfalls...

MNA Copyright 2016 Monarch Network Architects LLC 1

The Future of Optical Networking:

Challenges, Pitfalls, and Common Sense

Jane M. Simmons Monarch Network Architects

www.monarchna.com

May 12, 2016

Cartagena, Spain

20th International Conference on Optical Network Design and Modeling

ONDM 2016 Keynote Address

http://www.monarchna.com/


Overview

• Analysis and Modeling • Complexity vs. Long-term

Benefits

• Consider the Alternatives

• Extrapolation

• Unintended Consequences

• Performance Objectives

• Modeling Assumptions

• Design Methodologies

• Potential Optical-Networking

Innovations • Gridless/Elastic Networking

• Sleep-Mode for Energy Efficiency

• Space Division Multiplexing

• Software Defined Networking

• Mixed-Line-Rate Networks

Goals:

• Examine some of the pitfalls related to the list on the left in

the context of the innovations listed on the right

• Enumerate some of the challenges in implementing the proposed innovations


Complexity

MNA

• If a connection requires B units of bandwidth, it is assigned B units of bandwidth

• Optical-layer grooming

• B can grow / shrink Elastic

• Two major goals:

• Reduce the amount of required electronic grooming (cost & power consumption)

• Utilize bandwidth more efficiently

• In practice, B cannot be arbitrary in size

• For example: B may be limited to being a multiple of S (slot size)

(I will use the term optical slice to indicate the chunk of spectrum allocated to a

connection)

Gridless/Elastic Optical Networks (EONs)

Copyright 2016 Monarch Network Architects LLC 4

MNA

Management of a Gridless Network is Complex

•Need to track how the spectrum is sliced up on each fiber Routing and Spectrum Assignment (RSA) vs. Routing and Wavelength Assignment (RWA)


Free Assigned

•May lead to „stranded‟ spectral resources, especially in a dynamic network

The available spectrum may consist of narrow, non-contiguous spectral regions

Example: 45 GHz of spectrum is available, but not contiguous

A new demand requiring 45 GHz of spectrum is

either blocked or partitioned into multiple lower-rate

„sub-demands‟ (which adds more complexity)

25

GHz

15

GHz

50

GHz

10

GHz

20

GHz

30

GHz

ROADM

70 GHz 75 GHz

•May limit opportunities for optical bypass, especially in a dynamic network

The spectrum likely will be sliced up differently on each fiber entering a ROADM

Example: 25 GHz is free on both links, but the free regions do not align. A new 25-GHz

connection would need to be regenerated at this node.

50

GHz

20

GHz 25

GHz

15

GHz

60

GHz

25

GHz

25

GHz 50

GHz

20

GHz

50

GHz

MNA

Defragmentation is Likely Required in a Gridless Network

• Defragmentation modifies live connections; needs to be done carefully

• Transponders and ROADMs need to be reconfigured to match the new

spectral allocation, without bringing down any connections

• Modify only the assigned spectrum, or modify both the route and spectrum Push/Pull methodology – Shifts the spectrum; hitless operation has been demonstrated

• F. Cugini, et al., “Push-pull defragmentation without traffic disruption in flexible grid optical networks,”

J. Lightwave Technol., January 2013

Need to Defragment the network to create larger blocks of free spectrum

Perform defragmentation periodically or when a new connection is blocked


25

GHz

15

GHz

50

GHz

10

GHz

20

GHz

30

GHz 25

GHz

50

GHz 45

GHz

30

GHz

Defragment

• Practical Issues • Need guardbands between each „optical slice‟ Wasted bandwidth

• Many bandwidth-variable TxRx‟s are required; or, use Multi-Flow TxRx‟s

• Need bandwidth-variable ROADMs

25

GHz

50

GHz 45

GHz

30

GHz G G G

MNA

What Does All of the Added Complexity Buy You?

„Back of the Envelope‟ Benefit Analysis

• Assume that network traffic doubles every 30 months

• Assume that a Gridless/EON approach is 30% more capacity-efficient than

alternative architectures. (Note: Some studies that show a much greater benefit in capacity efficiency do not make use of any electronic grooming for the low-rate traffic in the

alternative architectures.)

• The network will be filled ~11 months later with the Gridless/EON approach

Does delaying a network upgrade by less than one year

warrant the extra complexity?

• Of course, the Gridless/EON approach provides benefits other than more

efficient capacity use

• Decreases, but does not eliminate, the need for electronic grooming

• Well suited for carrying high-bit-rate services


Do all of the network management and advanced technology

requirements justify the Gridless/EON architecture?


Consider Alternatives

MNA

Alternative Approach to Implement Elasticity


Two current approaches to grow/shrink the spectral bandwidth allocated to a connection In both approaches, the central frequency of the connection remains the same

Grow to the

right, shrink

from the right

Grow Shrink

Grow to the

right and left,

shrink from the

right and left

Grow Shrink

K. Christodoulopoulos, I. Tomkos, E. Varvarigos, “Time-varying spectrum allocation policies and blocking analysis in

flexible optical networks,” JSAC, January 2013

Alternative:

Grow to the

left, shrink

from the right

Grow Shrink

See: J. Simmons, Optical Network Design and Planning, 2nd Edition

This will have the tendency to shift connections to the left, which should result in better

packing; i.e., Simultaneously accomplishes defragmentation similar to Push/Pull. (May want to leave small gaps between connections to allow for future growth.)

MNA

Alternative Architecture 1: Two-Tier Gridless Architecture


• Partition the spectrum into the same band structure on every link

• Bands don‟t have to be of equal spectral width (as long as each link is the same)

• Utilize gridless approach within a band

• Requires effective algorithms to efficiently fill the bands

• Two-Tier ROADM Architecture

• Tier 1: Band-level ROADM - Course-granularity, Fixed-Bandwidth

• Tier 2: „Slice-level‟ ROADM - Fine-granularity, Bandwidth-Variable

See: Saleh, A.A.M. and Simmons, J.M., “All-optical networking – evolution, benefits, challenges, and future vision,” IEEE

Proceedings, May 2012.

Band 1 Band 2 Band 3 Band 4 Band 5

Tier 2

Tier 1

Eliminates „misalignment‟ across links

Greater opportunity for optical bypass

MNA

Defragmentation / Elasticity in the Two-Tier Approach


Tier 1

Fixed-Bandwidth

Coarse Granularity ROADMs

Tier 2

Bandwidth-Variable

Fine Granularity ROADMs

One band is shown, carrying two connections

Shifting the spectrum within a band, or changing a connection‟s bandwidth, requires:

• Filter reconfiguration at the endpoints, in the Tier 2 ROADMs only

• Transponder reconfiguration

• Less disruption of network equipment as compared to single-tier operation, where all of the

ROADMs and transponders along the path need to be reconfigured

Shifting the spectrum across bands may also require reconfiguring the Tier 1

ROADMs at the endpoints (but only to modify which bands are added/dropped)

Bandwidth-Variable Transponders

Node 1 Node 2 Node 3 Node 4

MNA

• Limit the possible spectral widths of the „optical slices‟

• For example, permit only 37.5 GHz, 50 GHz, and 75 GHz slices

As compared to supporting M12.5 GHz slices, for example, in the gridless approach

• Attempt to segregate the different slice sizes when assigning spectrum

• Less stranded bandwidth, more optical bypass, less need for defragmentation


Alternative Architecture 2: Limited Number of „Slice‟ Sizes

• May not be as capacity-efficient as the Gridless approach, but should be close

With a lot less management and technical complexity !

• However, less reduction in the amount of required electronic grooming

37.5 GHz Slices 50 GHz Slices 75 GHz Slices

See: J. Simmons, Optical Network Design and Planning, 2nd Edition

Also see Semi-Flex Grid proposal in: Z . Shen, H. Hasegawa, K. Sato, T. Tanaka, and A. Hirano, “A novel elastic

optical path network that utilizes bitrate-specific anchored frequency slot arrangement,” Optics Express, Feb 2014.

This is a soft partitioning of the spectrum; Not a fixed partitioning


While we’re talking about ‘Soft Partitioning’ ….

Wavelength Assignment in Mixed-Line-Rate (MLR) Networks

• 10-Gb/s wavelengths may have an adverse effect on „near-by‟

40-Gb/s and 100-Gb/s wavelengths co-propagating on the same fiber

• Depends on the modulation formats used

• The 10-Gb/s wavelengths induce XPM, especially with the 40-Gb/s wavelengths

• No problems with co-propagating 40-Gb/s and 100-Gb/s wavelengths O. Bertran-Pardo, et al., “Overlaying 10 Gb/s legacy optical networks with 40 and 100 Gb/s coherent terminals,” J.

Lightwave Technol., July 2012

• Necessitates using guardbands in between the „incompatible‟ wavelengths

• Wastes bandwidth: E.g., 300-GHz guardband between 10-Gb/s OOK and 40-Gb/s DP-QPSK

• Wavelength assignment can be strategically utilized to mitigate this effect

• Segregate the rates when possible:

• Assign 10 Gb/s connections to one end of the spectrum l1 l2 l3 … • Assign 40 Gb/s and 100 Gb/s connections to the opposite end …l78l79l80

• This is NOT a hard partitioning of the resources among the rates

Sometimes a Simple Solution Works Just Fine !

• Note: MLR is more likely to be found in small carriers. Large carriers generally light a new fiber

when they upgrade the line-rate.


Extrapolation


Beware of the Perils of Extrapolation

In a First Grade class learning multiplication, the teacher posed

the following question:

If a 9-year old boy eats 3 meals each day, how many meals

per day does an 81-year old man eat?

Half of the class responded 27 meals per day !!!


Optical Reach: Longer Is Not Always Better

0 1000 2000 3000 4000 5000 6000

Optical Reach (km)

Cap

ital

Co

st

of

Netw

ork

Typical “Cost vs. Reach” curve for a US backbone network

If 2,000-km reach is more cost effective than 1,000-km reach, then

6,000-km reach must be even better …

No !

• With ~2,000 km reach, the remaining number of regenerations is relatively small • Less opportunity to reduce cost

• More costly equipment needed to attain longer reach (e.g., better amps, lasers)

From: JM Simmons, “On

determining the optimal

optical reach for a long-haul

network,” J. Light.Technol.,

March 2005

Also see:

H.P. Sardesai, et al.,

“Optimal WDM layer

partitioning and transmission

reach in optical networks,”

OFC 2005

A.Morea, et al., “Impact of

the reach distance of WDM

systems on the cost of

translucent optical transport

networks,” Networks 2004


Extrapolating Impairment Effects Resulted in Dire Predictions

• In the 2000 timeframe, the state-of-the-art transmission systems supported

2.5-Gb/s line-rate with an optical reach of 3,000 km

• Polarization-mode Dispersion (PMD) is potentially a larger problem as the

line-rate increases

• The PMD-limited optical reach decreases with the square of the line-rate

• Led to predictions that 40-Gb/s line-rate would be limited to a reach of 12 km !

• Many concluded that 40-Gb/s line-rate in the core network was impossible !

• Reality: 40 Gb/s has been deployed with 2,500 km reach. 100 Gb/s is currently

being deployed, with full expectations that 400 Gb/s and 1 Tb/s line-rate are on

the horizon !

• Technology is constantly improving Extrapolating current results is difficult

• Modulation formats with less sensitivity to PMD

• Coherent detection – the digital signal processing used at the receiver allows for

electronic compensation of PMD

• Better fiber

“It's tough to make predictions, especially about the future !” Yogi Berra


Gridless Network Granularity: Finer Is Not Always Better

If 12.5-GHz granularity is more capacity-efficient than 50-GHz granularity,

why not use even finer granularity ?

• All of the complexity issues discussed earlier will be exacerbated

• E.g., Tracking slot usage, stranded bandwidth, optical bypass limitations

• At some point, the ROADM filters are a limiting factor

• As filters become narrower, their roll-off becomes less sharp

• Guardbands take up a larger fraction of the spectrum, which nullifies spectral benefits

• Implication

• Some amount of electronic grooming is likely still needed to efficiently transport

low-rate connections (current networks carry a lot of traffic at 10 Gb/s or below)

Note: Filter precision/resolution (e.g., 0.5 GHz) pertains to sharpness, not filter width

G 12.5 GHz

G


Unintended Consequences


Put Equipment in Sleep Mode to Save Energy

The amount of energy used in communication networks is causing concern

• One proposal to reduce energy usage:

If equipment is currently not in use, either power it down or put it in „Sleep Mode‟ Can combine this with routing/grooming algorithms to proactively drive traffic away

from some links and nodes

• This strategy has potential to save energy, but with an undesired side-effect:

Power cycling will likely shorten the life of the equipment

• A comprehensive study examined how the cost savings from power cycling

compared with the attendant additional cost of replacing equipment EDFAs were the most negatively impacted

The extra EDFA reparation costs would be higher than the energy cost savings P. Wiatr, J. Chen, P. Monti, and L. Wosinska, “Energy Efficiency Versus Reliability Performance in

Optical Backbone Networks,” JOCN, March 2015.

• Another undesired side effect:

Dynamic traffic may incur greater delays (need time to power-up the equipment)


Fiber Capacity Isn‟t „Infinite‟ After All

• The capacity of a fiber has often been viewed as being almost „infinite‟

• But, recent studies indicate that systems with 80 x 100 Gb/s wavelengths on

one fiber (8 Tb/s) are already within a factor of ~5 of the fiber capacity limit

Study Assumptions: Conventional Single-Mode Fiber (SMF)

Optical reach of 2,000 km (fiber capacity increases as the optical reach decreases)

Raman amplification

R.-J. Essiambre, et al., “Capacity limits of optical fiber networks,” J. Lightwave Technol., Feb. 2010

• Some of the proposals to address the impending limit on fiber capacity:

• Deploy multiple fiber-pairs per link instead of one

• Does not provide economies of scale w/r to cost or power consumption

• Multicore Fiber (current fibers have a single core); e.g., 7 cores in one fiber

• Ideally, capacity increases in proportion to the number of cores

• Potential problems with cross-talk between the cores

• Multimode Fiber (current long-distance fibers have a single mode)

• Very small number of modes (e.g., 6), not hundreds; „Few-Mode Fiber‟ (FMF)

• Ideally, capacity increases in proportion to the number of modes

• Combine Multicore with Multimode to multiply the benefits

These are all examples of Space Division Multiplexing (SDM)


Side Effects of Few-Mode Fiber (FMF) Solution

• The modes of a given wavelength will be coupled together

• Example: if there are 6 modes, then all 6 l1‟s will be coupled together • A ROADM will not be able to pick out just one mode or a subset of the modes

• All modes will be routed as an inseparable unit, whether desired or not

• Positive „Spin‟: FMF results in ‘Spatial Super-Channels’ • Allows for efficient transmission of very high bandwidth connections

• Ex: A 600 Gb/s connection can be carried as one 6100G super-channel

• Lower-cost ROADMs and improved transmission performance • Only 1/Nth of the number of ROADM filters (assuming N Modes)

• Better transmission performance with wider bandwidth channel

• Some impairments have less of an impact with mode-mixing

• Negative „Spin‟: Bundling modes in FMF is similar to ‘Wavebands’ in SMF • Waveband is a group of wavelengths routed as a bundle

• Wavebands were first proposed in the late 1990s to reduce switching costs

• Wavebands require more complex RWA algorithms to use the bandwidth efficiently

• Here, the size of the „wavebands‟ is determined by FMF; not necessarily the optimal size

• Coupling issues also arise with Multi-Core Fiber if significant X-talk among the cores

• Other ramifications of coupled modes/cores: • Uncoupling modes/cores requires electronic multiple-input multiple-output (MIMO) processing,

which consumes a significant amount of power

• Need to lay new fiber (current fiber is single core / single mode)


Performance Objectives (What services need to be supported by

the network)


Very Brief Overview of Software Defined Networking (SDN)

SDN originated as a paradigm for the Ethernet and IP Layers

Basic Tenets: • The network switches/routers are controlled via software that runs in a

decoupled automated control plane; i.e., separation of control & data planes

• The control plane is logically centralized

Expected Benefits: • Allows the control software to be more easily modified/customized. Less

dependency on the switch or router vendor.

• SDN is compatible with Network Virtualization, where the communication,

computing, and storage resources of a single physical network are partitioned

into multiple logically isolated networks. Customers can be assigned their own

„view‟ of the network.

• May lead to lower-cost switches and routers


Extending SDN to the Optical Layer

The data and control planes are already separate in the optical layer

Two critical aspects of extending SDN to the optical layer:

• Desirability of Unified Network Control

Is a unified multi-layer, multi-vendor, multi-domain view of the network feasible and

scalable?

• Will vendors want to expose the details of their system ?

• Latency of Centralized Network Control

What are the desired network applications and the corresponding required

connection set-up times ? Some examples of services requiring very fast setup:

• Rapid dynamic restoration (restoration path calculated & established on the fly,

after a failure occurs) Requires setup times on the order of 100 ms

• Highly interactive visualization and data fusion (network-wide)

Wavelength-connections, of very short duration, need to be established in response to

mouse clicks. Too costly to keep the wavelengths permanently established. Requires

setup times on the order of 100 ms to meet the human tolerance for delay with

interactive applications

• Global-scale distributed computing

Efficiency increases as setup time decreases

MNA

• If desired setup times cannot be met, may need alternative solutions • Deploy geographically diverse SDN controllers

• Implement fully/partially distributed network control (e.g., GMPLS) Issues with resource contention


SDN Latencies Depend on the Network Scale

When analyzing the inherent latency of SDN, the assumptions regarding

network (domain) scale are clearly important

Source

Destination

SDN

Controller

Domain is a National Network – 5,000 km in extent

Unlikely to meet 100 ms setup time

Domain is a Regional Network

300 km in extent

SDN

Controller

Source

Destination

• The most stringent network requirements may be driven by services that represent a

small percentage of the network traffic. Is it worth trying to satisfy them?

• Perhaps, “if we build it, they will come”; i.e., new services will be developed


Modeling Assumptions


The Modeled Topology Should Befit the Study

Example: „USNET‟ topology is often used in Network Studies:

24 Nodes, 43 Links Average Nodal Degree of 3.6

Three Nodes of Degree-2

Realistic US Backbone Optical Network:

Average Nodal Degree of ~ 2.7

Approximately 50% of the Nodes are Degree-2

(Representative backbone network topologies can be downloaded from: monarchna.com/topology.html)

If the scheme being analyzed is dependent upon nodal degree, the USNET

topology may skew the results

Note: The virtual topology of the IP-layer Network typically does have a higher

average nodal degree (there typically is not a core IP router at every ROADM node)


European National Network Topologies are More Dense

Topologies Commonly used in Published Network Studies:

Question for the European Carriers:

Are these accurate models of the actual networks?

Higher Average Nodal Degree, Typically Few Degree-Two Nodes

British Telecom Avg. Nodal Degree: 3.2

Two Degree-two nodes

Telefόnica Avg. Nodal Degree: 3.3

Three Degree-Two nodes

Telecom Italia Avg. Nodal Degree: 3.2

Ten Degree-two nodes

Deutsche Telekom Avg. Nodal Degree: 3.2

Five Degree-two nodes


Use a Realistic Failure Rate for the Modeled Network

In the US, a typical fiber-cut rate for backbone optical networks is approximately

2 cuts/1000 miles per year

The cut-rate for US metro networks is typically higher:

Old Telcordia statistics indicate ~5 cuts/1000 miles per year - still valid?

The fiber cut-rate for European national networks may be higher than in the US

Don‟t assume an Unrealistic Fiber-Cut Rate for a modeled network simply to make the proposed scheme looks good !

Question for the European Carriers:

What are realistic fiber-cut rates in European networks?


Use a Realistic Blocking Rate To Evaluate Dynamic Services

• Published studies that compare various network design schemes often

focus on blocking rates of 1% or higher

• This may be the only regime where differences among the schemes are

pronounced

• It is unlikely that carriers would offer a service with a blocking rate higher

than 1%

• For dynamic services, an acceptable blocking rate of 0.1% is likely more

realistic

Don‟t assume an Unrealistic Blocking Rate for a modeled network simply to make the proposed scheme looks good !


Design Methodologies


Optimization Tools

Integer Linear Programming (ILP) as a Network Design Tool:

Step 1: Present Details of the ILP

Step 2: Conclude the ILP section with text similar to:

“… the ILP model is not well-suited for a large number of traffic demands in

large-scale network topologies. Thus, we develop heuristic algorithms …”

“… the ILP formulation is not applicable for large scale and practical networks…”

“…the ILP becomes intractable and does not scale well for large problems …”

“…fails to provide a solution for large networks due to the high computational complexity…”

Step 3: Present a scalable heuristic

Step 4: Compare the solution using the heuristic approach to that of the ILP, using a

very small network (e.g., few nodes, few wavelengths, few demands)

If the solutions are close, declare Victory !

Is it always true that a heuristic that produces „good‟ results on a very small network

will work well for a realistically sized network ?

If a heuristic does not perform very well on a small network, might it not still work

well in a larger network ?

(e.g., a grooming algorithm that performs better with more traffic)

Text from some recent papers


Put a Greater Emphasis on Improving Tractability

• Look for symmetries or other imposed patterns to shrink the ILP search

space or allow for decomposition of the problem E. Yetginer, Z. Liu, and G.N. Rouskas, “Fast exact ILP decompositions for ring RWA,” J. Opt.

Commun. Netw., July 2011

• Impose intelligent restrictions that are likely to have minimal impact on

the solution • Example: Use path-based routing approaches in the ILP, where the

permitted paths between two endpoints must come from a prescribed set

(mimics K-path alternative routing)

• Other approximating techniques • Use Linear Programming (LP) instead of ILP

• Apply perturbation and rounding techniques to integralize the solution • K. Christodoulopoulos, K. Manousakis, and E. Varvarigos, “Comparison of routing and

wavelength assignment algorithms in WDM networks,” GLOBECOM‟08

• Use Meta-heuristics (simulated annealing, genetic algorithms, etc.) • Determine a good starting point based on heuristics

• Perform limited post-processing with a meta-heuristic to search for a

better solution


Put a Greater Emphasis on Robustness of the Solution

“Conditions”

Cost

Minimum-Cost Point

But, Narrow Range of Conditions

ILP will yield this solution

Slightly Higher Cost

But, Broader Range of Conditions

Good Heuristics may yield this solution

MNA 36

The Minimum-Cost Solution May Be Impractical

Used under Fair Use for nonprofit educational purposes

One study produced the following „optimal‟ solution:

The major hub of a US backbone network should be located in ….

Rattlesnake Pass (in the Arizona Desert) !

Copyright 2016 Monarch Network Architects LLC

MNA 37

Some Final Words

Continue to push advantageous and strategic innovations • Holistic evaluation

• Accurate modeling methodologies

• Long-term view and transition strategies

• Honest appraisal of alternatives

Many of today‟s technologies were met with initial resistance • Optical Bypass

• Original reaction: Most carriers were dead-set against optical bypass

• Today: Almost all carriers employ optical-bypass technology

• Dynamic Networking • Original reaction: Carriers (and enterprises) did not see a need for it

• Today: Carriers tout their dynamic-service offerings

• Software Defined Networking in the Optical Layer • Original reaction: Lukewarm reception

• Today: The letters „SDN‟ appear in almost every single research paper

and press release regarding the future of optical networks !

• Some circa-2000 proposals initially rejected by carriers are making a comeback

• Support for Asymmetric Traffic

• Drop-and-waste ROADMs


MNA 38

Abbreviations

DP-QPSK - Dual-Polarization Quadrature Phase-Shift Keying

EDFA – Erbium-doped fiber amplifier

EON – Elastic optical network

FMF – Few-mode fiber

GMPLS - Generalized Multi-Protocol Label Switching

ILP - Integer linear programming

LP – Linear programming

MCF – Multi-core fiber

MLR - Mixed-line-rate

ms - millisecond

OOK – On/off keying

PMD – Polarization mode dispersion

ROADM – Reconfigurable optical add/drop multiplexer

RSA - Routing and spectrum assignment

RWA - Routing and wavelength assignment

SDM – Space division multiplexing

SDN - Software Defined Networking

SMF – Single-mode fiber

TxRx – Transmitter/Receiver (Transponder)

XPM – Cross-phase modulation


MNA

From:

BAA 06-29 Proposer Information Pamphlet (PIP) for Defense Advanced Research Projects Agency (DARPA)

Dynamic Multi-Terabit Core Optical Networks:

Architecture, Protocols, Control and Management (CORONET)

A realistic, i.e., non-uniform, traffic should be assumed. As a

guide, if 20% of the nodes are assumed to be “large” nodes,

and the rest “small” nodes, the traffic distribution among the two

types of nodes should be approximately:

Large-Large: 40%, Large-Small: 40%, Small-Small: 20%

Guidelines for National Core Network Traffic Distribution


The Future of Optical Networking: Challenges, Pitfalls...

Documents

Transcript of The Future of Optical Networking: Challenges, Pitfalls...