Tracking network evolution process with netTransformer

Bulgarian Internet BGP Peering evolution from 2001 till now

Nikolay MilovanovVasil Yordanov

https://github.com/iTransformers

OpenFest 2014

Network evolution in nutshell

• Network evolution could be expressed as:– Topology changes:

• Appearance, disappearance of nodes or links• Changes in network diameter• Changes in node ranks • Changes in clustarization and giant components

– Changes in node/edge properties• Appearance of new metadata• Disappearance of such• Changes in node/edge metadata values

– All the above reflects changes in the law as per which network evolves :• An example of such law is Barbatsi and Albert law – e.g human

networks evolve as per power law

Examples for evolving networks

• Internet• WWW• Social• Bio• Communication• Networks of SA components and connectors

• Knowing the law will allow us to predict how does the network will evolve

So a question emerge: How can we track how does the network evolves?

– Globul OMC used a notepad to write down each and every event that happens in the network

– Traffic/Performance monitoring systems might give us such information but in really it is quite difficult to abstract it (too much details)

– Well we don’t care so much of the tiny networking details we care for evolution on a architecture level!

Views and perspectives• Many stakeholders result in many perspectives and thus many

views of the network are required!

The cardiologist needs a cardiogram

.. But the orthopedist needs an X-ray scan

Stakeholders needs views of the network that will allow them to reason about it!

In more common languadge

• Core network engineers needs a view of the core network, they don’t care much about the access…

• BGP peering needs a view of the BGP they don’t care much neither about the core nor about the access!

• Support guys from Plovdiv field maintenance team needs a view of the network from their region, they don’t care about Sofia network

netTransformer

• Software for tracking and pushing the network evolution process

• Network transformation – a process in which we push the network evolution in direction that suits our stakeholder interest

• It is able to discover networks, to fill in different network models, to diff between any two discovered network states

• It is able also to “talk” with network devices and to executes the steps of the strategies for network from state to state transformation.

netTransformer

• Written in java, xslt, glue with maven, groovy, Spring, JUNG and a lot of swing ;)

• https://github.com/iTransformers/netTransformer• GPL!• Architecture is based on the following key

Interfaces/components– Discoverers (anybody can implement his own, we have a couple raging

from SNMP, Openflow floodlight, BGP peering and some others). – Node or Network Listeners– Viewer, resourceFactories, activationFactories – postDiscoverers (asses networks after the discovery has finished!)

Demo

SHORT SURVEY ON HOW DOES THE INTERNET IN BULGARIA EVOLVED FROM 2001 TILL NOW!

The approach

• We will generate Global IPv4/IPv6 Internet maps from MRT (RFC 6396) files.• The raw data from the repository of the route views project

ftp://archive.routeviews.org/• Each MRT file will contain full Internet routing table. • We will extract BGP AS PATH attributes and will generate a map based on Internet

Autonomous System numbers. • As part of the map generation process each BGP AS has been enriched by whois info

from http://www.potaroo.net/bgp/iana/asn.txt)• The maps itself are generate d with javaMrt2Graphml https://github.com/iTransformers/javaMrt2Graphml• We will generate one map per year (lazy and statistically incorrect from the last day

of December • Each map will contain the “whole” Internet• Each Internet map will be filtered by netTransformer Filter into a smaller graph that

covers only the Bulgarian & Bulgarian interconnect to external Internet

Node MetadataAutonomous system ID

IPv6 advertised prefix count

AS name & description (whois)

Advertised IPv4/IPv6 prefixes (comma separated)

IPv4/IPv6 address count

Country of affiliation

IPv4 advertised prefix count

Survey outcome

• How does the number of BG AS evolve through the years (totals and node births/deaths)

• What is the number of Internet BGP peering's • What is the number of total advertised

addresses • All that for IPv4 & IPv6

Graph Diff

BG INTERNET IPV4

Bulgarian IPv4 autonomous systems (totals per year)

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 20140

100

200

300

400

500

600

International BGP Peering BG to *

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 20140

20

40

60

80

100

120

Bulgarian IPv4 address space

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 20140

1000000

2000000

3000000

4000000

5000000

6000000

7000000

IPv4 AS birth /dead Achieved by diffing per year

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

2010-2011

2011-2012

2012-2013

2013-2014

0

10

20

30

40

50

60

Node birthsNode deads

Track the evolution process as an animated gif

• https://www.youtube.com/watch?v=ZTT3nr91zuw

BG INTERNET IPV6

BG IPv6 ASNs

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 20140

10

20

30

40

50

60

International IPv6 peering

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 20140

5

10

15

20

25

30

35

Advertised IPv6 Address Space

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 M

50000 M

100000 M

150000 M

200000 M

250000 M

300000 M

BG IPv6 AS birth /dead Achieved by diffing per year

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

2010-2011

2011-2012

2012-2013

2013-2014

0

2

4

6

8

10

12

A few words about OpenSource and motivation

• The surprising truth about what motivates ushttps://www.youtube.com/watch?v=u6XAPnuFjJc

QUESTIONS!

Nikolay MilovanovVasil Yordanovhttps://github.com/iTransformers/

Tracking network evolution process with netTransformer Bulgarian Internet BGP Peering evolution from 2001 till now