Central Asia: Internet Structure and Trends

Jim Cowie, Chief Scientist @jimcowie / @DynResearch

Bishkek, Kyrgyz Republic

10 December 2014

/ 2 @jimcowie / @DynResearch

NOTE: Some cities host multiple collectors. Cable Map credit: Telegeography

Active Measurement Infrastructure

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 3 @jimcowie / @DynResearch

Central Asia

Not merely landlocked, but often doubly landlocked

Reliance on neighbors’ terrestrial connectivity is unavoidable

The challenge: increase Internet diversity, reduce latency and cost

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 4 @jimcowie / @DynResearch

Top Sites: Kyrgyz Republic 1.  Google.com 58ms, service from GGC in Voronezh 2.  Youtube.com 65ms, service from Google Moscow

3.  Elcat.kg <5ms, Bishkek

4.  Mail.ru 66ms, Moscow

5.  Facebook.com 204ms, Washington, DC 6.  Odnoklassniki.ru 66ms, Moscow

7.  Yandex.ru 67ms, Moscow 8.  Vk.com 76ms, St Petersburg

9.  Ts.kg <5ms, Bishkek

10.  Ok.ru 66ms, Moscow

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 5 @jimcowie / @DynResearch

Top Sites: Kyrgyz Republic 1.  Google.com 2.  Youtube.com

3.  Elcat.kg

4.  Mail.ru

5.  Facebook.com 6.  Odnoklassniki.ru

7.  Yandex.ru 8.  Vk.com

9.  Ts.kg

10.  Ok.ru

11.  Namba.kg

12.  Wikipedia.org

13.  Google.ru

14.  Blive.kg

15.  Torrent.kg

16.  Google.kg

17.  Amazon.com

18.  Akipress.org

19.  Adme.ru

20.  Aliexpress.com

21.  Vb.kg

22.  Instagram.com

23.  Alibaba.com

24.  Yahoo.com

25.  Bolshoyvopros.ru

26.  Ebay.com

27.  Twitter.com

28.  Super.kg

29.  Rutracker.org

30.  Blogspot.com

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 6 @jimcowie / @DynResearch

Supply and Demand

Central Asian consumers, businesses need to reach content markets in Russia, the United States, and China. How does traffic flow today?

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 7 @jimcowie / @DynResearch

Kazakhstan exports Russian transit to all neighbors

Tajikistan and Turkmenistan each have two choices.

Afghanistan has 3.

China is still not a visible participant

Choice is Important

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 8 @jimcowie / @DynResearch

‘Watersheds’

Central Asian consumers, businesses need to reach content markets in Russia, the United States, and China. How might traffic flow tomorrow?

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 9 @jimcowie / @DynResearch

‘Watersheds’

Russian connectivity: most traffic heads through Moscow en route to Frankfurt, London, Stockholm Southern routes through Ukraine are also available (for the moment)

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 10 @jimcowie / @DynResearch

‘Alternative Watersheds’ 1

Balkan connectivity: Azeri, Armenian, Iranian paths join east-west Turkish pipeline routes for fiber via Ankara, Istanbul Black Sea routes from Baku-Tbilisi-Poti-Varna-Sofia avoid Istanbul

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 11 @jimcowie / @DynResearch

‘Watersheds’

Submarine connectivity: major cables transiting the Suez en route to European markets Iran, Pakistan have excellent connectivity to UAE, Oman (SMW3, SMW4, FLAG, etc.)

‘Alternative Watersheds’ 2

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 12 @jimcowie / @DynResearch

‘Watersheds’

Chinese connectivity: challenging from both a latency, reliability perspective China has its own problems reaching world markets reliably Still, a potential source of diversity

‘Alternative Watersheds’ 3

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 13 @jimcowie / @DynResearch

‘Watersheds’

Combine these 4 watersheds and simplify. What emerges, strategically?

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 14 @jimcowie / @DynResearch

‘Watersheds’

Key transit cities: •  Moscow •  Istanbul •  Dubai •  Beijing

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 15 @jimcowie / @DynResearch

‘Watersheds’

Key transit cities: •  Moscow •  Istanbul •  Dubai •  Beijing

Baku and Tehran are the natural hingepoint

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 16 @jimcowie / @DynResearch

Four Paths to Diversity

We still need to incorporate either •  The Iranian terrestrial network, or •  A route crossing the Caspian Sea,or •  An Afghan-Pakistani alternative

path to the sea

Baku and Tehran are the natural hingepoint

Key transit cities: •  Moscow •  Istanbul •  Dubai •  Beijing

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 17 @jimcowie / @DynResearch

Iran’s Terrestrial Net

Iran already provides considerable Internet support to its neighbors, including:

•  Turkmenistan

•  Iraqi Kurdistan

•  Western Afghanistan

Iran can access all three western watersheds: Russia, Turkey, submarine

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 18 @jimcowie / @DynResearch

Crossing the Caspian Sea

Azerbaijan, Kazakhstan plan to build the transcaspian wet segment.

Only a few formalities remain before construction can begin.

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 19 @jimcowie / @DynResearch

Iran’s Terrestrial Net

Iran already provides considerable Internet support to its neighbors, including:

•  Turkmenistan

•  Iraqi Kurdistan

•  Western Afghanistan

Iran can access all three western watersheds: Russia, Turkey, submarine

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 20 @jimcowie / @DynResearch

Iran’s Terrestrial Net

Iran already provides considerable Internet support to its neighbors, including:

•  Turkmenistan

•  Iraqi Kurdistan

•  Western Afghanistan

Iran can access all three western watersheds: Russia, Turkey, submarine

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 21 @jimcowie / @DynResearch

A Novel Path To The Sea

CASA-1000 project could interconnect electrical grids from Kyrgyz Republic to Pakistan

Includes 24 strands of single-mode fiber, of which 100Mbps is committed for grid operations

Bishkek could be 40ms from Persian Gulf submarine connectivity!

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 22 @jimcowie / @DynResearch

In Summary….

The Internet Improves When Diversity Increases

Diversity drives higher reliability, lower prices

Central Asia is gaining new routes that send traffic through several different “watersheds”

Later today we’ll explore these changes in more detail!

The Importance of Internet Measurement

Jim Cowie, Chief Scientist @jimcowie / @DynResearch

Bishkek, Kyrgyz Republic

10 December 2014

/ 24 @jimcowie / @DynResearch

NOTE: Some cities host multiple collectors. Cable Map credit: Telegeography

Active Measurement Infrastructure

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 25 @jimcowie / @DynResearch

Why We Measure, and What We Measure

1.  The relationships

Who participates in each market: retail, wholesale

Nature of their bilateral agreement: peering or transit

2.  The paths

There are many, many ways traffic can flow between two providers in two cities

3.  Performance and Reliability

Reachability, Round-trip latency, Variance over time

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 26 @jimcowie / @DynResearch

Vocabulary of Interconnection

“Routed Network” or “Routed Prefix”

Range of IP Addresses used for a particular function (here, the World Bank’s office in

Bishkek)

=

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 27 @jimcowie / @DynResearch

Vocabulary of Interconnection

“Routed Network” or “Routed Prefix”

“Autonomous System” Any organization that

requests an AS Number can route its own prefixes

on the global Internet.

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 28 @jimcowie / @DynResearch

Vocabulary of Interconnection

“Routed Network” or “Routed Prefix”

“Autonomous System”

$€

“Transit”

When one ASN purchases transit from another ASN,

the provider agrees to carry the customer’s

traffic anywhere on Earth it needs to go.

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 29 @jimcowie / @DynResearch

Vocabulary of Interconnection

“Routed Network” or “Routed Prefix”

“Autonomous System”

$€

“Peering”

“Transit”

When two ASNs agree to peer together, they are

agreeing to exchange traffic between their

customers.

This saves money on transit and can improve

performance.

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 30 @jimcowie / @DynResearch

Connectivity Example

Kazakhstan’s largest bank buys Internet

transit from two providers

satellite

/ 31 @jimcowie / @DynResearch

Transit Strategies for ISPs

Kyrgyz ISPs have different strategies for interconnection.

We can look over a period of months or years at the percentage of BGP paths inbound through each transit provider.

More providers == more diversity == lower risks of disconnection

/ 32 @jimcowie / @DynResearch

Primary/Secondary

ElCat maintains BGP visibility through a single provider, Vimpelcom, very consistently over months of observation

…But on certain days, Orbita Plus is brought in for satellite transit.

/ 33 @jimcowie / @DynResearch

Diversity Strategy

AKNET splits virtually 50-50 in terms of BGP path selection between IPNET and Kaztranscom.

/ 34 @jimcowie / @DynResearch

Mega-Line

Phased out Kaztranscom transit in June

Became single-homed to IPNET this summer

Brought in Vimpelcom transit briefly in September

/ 35 @jimcowie / @DynResearch

Kyrgyztelecom

Kyrgyztelecom splits transit between Kazakhtelecom and Kvant Telekom

More address space appears each day through Kvant paths than through Kazakhtelecom paths

/ 36 @jimcowie / @DynResearch

Much Can Be Learned from Simply Counting

Let’s take a look at various populations of Autonomous Systems (ASNs) by country, across Central Asia and in nearby markets.

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 37 @jimcowie / @DynResearch

Domestic ASNs

The routed Internet consists of 49,108 ASNs worldwide (29 Nov 2014)

A domestic ASN has at least 80% of its routed customers in a single country

Compare:

•  Ukraine: 1,708

•  Germany: 1,358

•  UK: 1,390

•  Netherlands: 616

3 11

30

284

310 35 718 73

102

4,527

38 33

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 38 @jimcowie / @DynResearch

ASNs per capita

Domestic ASNs per million population, 2014

Compare: •  Ukraine: 39.7 •  Germany: 16.8 •  UK: 21.4 •  Netherlands: 36.2 •  USA: 48.0 Worldwide: 7.10

0.50 1.38

5.0

0.21

3.97 1.35 0.57 0.39

6.0

31.0

1.35 3.3

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 39 @jimcowie / @DynResearch

Frontier ASNs

A frontier ASN has at least one direct interconnection with a foreign ASN

Countries with more frontier ASNs have more diversity of interconnection

Fewer than 5: vulnerable to national disconnection

(Syria, Yemen, N. Korea)

2 of 3 8 of 11

7 of 30

52 of 284

9 of 310 15 of 35 42 of 718 6 of 73

21 of 102

339 of 4,527

1 of 38 3 of 33

/ 40 @jimcowie / @DynResearch

Percent of National Market On-Net Germany 9% EUNetworks AS13237 Czech Republic 12% Telefonica CZ AS5610 Poland 22% Netia AS12741 Romania 29% Romtelecom AS9050 Ukraine 29% Datagroup AS21219 Bulgaria 33% BTC AS8866 Armenia 36% GNC-ALFA AS6682 Georgia 37% Caucasus Online AS16010 Kyrgyz Republic 43% KyrgyzTelecom AS12997 Tajikistan 45% Babilon-T AS24722 Russia 49% Rostelecom AS12389 Belarus 68% Beltelecom AS6697 Turkey 83% Turk Telekom AS9121 Kazakhstan 87% Kazakhtelecom AS9198 Azerbaijan 90% Delta Telecom AS29049 Turkmenistan 97% Turkmen Telecom AS20661

Low largest-provider share suggests strong market competition… Among 112 countries worldwide: •  Median 42% •  Average 45%

/ 41 @jimcowie / @DynResearch

Internationally Connected (Cross-Border) ASNs Germany 9% EUNetworks AS13237 894 of 1,363 Czech Republic 12% Telefonica CZ AS5610 189 of 368 Poland 22% Netia AS12741 583 of 1,700 Romania 29% Romtelecom AS9050 494 of 1,121 Ukraine 29% Datagroup AS21219 208 of 1,654 Bulgaria 33% BTC AS8866 167 of 477 Armenia 36% GNC-ALFA AS6682 11 of 58 Georgia 37% Caucasus Online AS16010 25 of 50 Kyrgyz Republic 43% KyrgyzTelecom AS12997 7 of 30 Tajikistan 45% Babilon-T AS24722 9 of 12 Russia 49% Rostelecom AS12389 356 of 4,514 Belarus 68% Beltelecom AS6697 4 of 87 Turkey 83% Turk Telekom AS9121 27 of 329 Kazakhstan 87% Kazakhtelecom AS9198 23 of 102 Azerbaijan 90% Delta Telecom AS29049 3 of 33 Turkmenistan 97% Turkmen Telecom AS20661 2 of 3

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 42 @jimcowie / @DynResearch

Bishkek latencies are typically 100-150ms for Europe Note slow speeds to China, Middle East

RTTs

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 43 @jimcowie / @DynResearch

Multiple Paths

Amsterdam to Kyrgyztelecom Purple band:100-180ms via Kvant Telecom Red bands: 190ms/220ms via TTK (two physical paths)

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 44 @jimcowie / @DynResearch

Measuring Failover

Kyrgyz example from 9 July 2014 ElCat (AS8449) loses connectivity through Vimpelcom (AS3216) and fails over to TNS-Plus (AS197556)

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 45 @jimcowie / @DynResearch

Watersheds Again…

How do neighbors cooperate to increase the choices available to everyone?

Can we measure that process?

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 46 @jimcowie / @DynResearch

Kyrgyzstan Exports Internet Connectivity

Tajikistan relies critically on Kyrgyz connectivity for Internet and mobile service in Dushanbe

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 47 @jimcowie / @DynResearch

Measuring a Market Shift: First Signs

October 2014: Tajik provider Saturn-Online stops buying transit from Kyrgyz Telecom – why?

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 48 @jimcowie / @DynResearch

Pakistani Transit for Tajikistan via Afghanistan

Just appeared on Friday, 5 December ‘Special’ transit for the small network that hosts vkontakte.tj

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

/ 49 @jimcowie / @DynResearch

Summary: Internet Measurement Matters

Traditionally regulators and policymakers have waited for ISPs to self-report interconnection and performance changes. The most important metrics (latency, reliability, interconnections) can now be tracked daily as the local and regional market evolves. The Central Asian regional market has several plausible strategies for expanding diversity and improving access to international markets. It’s exciting to see regional ISPs cooperate and interconnect!

/ 50 @jimcowie / @DynResearch

The Importance of Internet Measurement

Jim Cowie, Chief Scientist @jimcowie / @DynResearch

Bishkek, Kyrgyz Republic

10 December 2014