Network Monitoring and Measurements at University of Napoli

COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 1

NM2 Network Monitoring and

Measurements: some new perspectives (?!?!)

COMICS Research Group Dipartimento di Informatica e Sistemistica

Università degli Studi di Napoli “Federico II”


Agenda Ø COMICS (COMputers for Interaction and

CommunicationS ) presentation Ø  COMICS research topics Ø  Network Monitoring and Measurements

Ø People Involved Ø Approach Ø Contributions

Ø Traffic Monitoring and Analysis Ø Network Measurements

Ø Contacts Ø Publications Ø Large Scale projects


COMICS Ø COMICS (COMputers for Interaction and

CommunicationS ) headed by Prof. Giorgio Ventre Ø Work spans 2 laboratories and Spin-Offs:

ü UoN/DIS •  @ University of Napoli

ü CINI/ITEM •  a research lab of the Italian University Consortium in Computer

Science & Engineering ü Academic Spin-Offs

Ø Funding mainly from EU, Industry, with some money (?) from national and local government


People@COMICS

Ø Today around 20 people in the group ü Seven of them with tenure and permanent positions

•  Giorgio Ventre •  Roberto Canonico •  Simon Pietro Romano •  Stefano Avallone •  Antonio Pescapè •  Maurizio D’Arienzo •  Salvatore D’Antonio

Ø  Collaborations with industries (Telecom Italia, Telefonica O2, Vodafone, H3G, Alcatel, Engineering Ingegneria Informatica, Accenture, Finmeccanica, Selex Sistemi Integrati, Juniper, Ericsson, IBM, Intel, Skylogic, ACCANTO, ALTO, several other SMEs, etc.) and AGCOM, Poste Italiane, and PA.


Ø Former EU Projects: ü Guardians ü Cadenus ü Intermon ü E-NET, E-Next ü Cost 263 ü Cost 290 ü OneLab ü NetQoS ü Content ü OneLab2 ü Intersection

Research Projects@COMICS (1/2)

Ø Former National Projects: ü COSMIC ü ESALAB ü NADIR ü QUASAR ü WEBMINDS ü RECIPE ü LATINO


Ø Current EU Projects: ü Inspire ü COST Action

IC0703 "Data Traffic Monitoring and Analysis (TMA)

Research Projects@COMICS (2/2)

Ø Current National Projects: ü LINCE


Research@Comics Ø Research areas:

ü Traffic Measurements and Analysis ü Network Monitoring and Anomaly Detection ü Perfomance Evaluation of Networked Systems ü Security, Reliability and Resiliency ü QoS and QoE in Heterogeneous Networks ü Analysis and Detection of Network Outages ü Traffic Engineering ü Wireless Mesh Networks

•  P2P overlay networks ü Management and control of network infrastructures

•  SLA, SLS, Policy based management ü Multimedia services engineering (IETF activities) ü Emulation, Virtualization and Cloud ü Green Networking


NM2, Network Monitoring and Measurements


Network Monitoring and Measurements (NM2) ² NM2 is part of the COMICS research group of the Dipartimento

di Informatica e Sistemistica at University of Napoli Federico II ² People Involved

Alessio Botta

Alberto Dainotti Walter de Donato

Pietro Marchetta

Giuseppe Aceto Antonio Pescapè


NM2 approach and vision

Links

Topologies

Applications

/

Services

Traffic

http://www.grid.unina.it/Traffic/


Network/Traffic Monitoring and Analysis (NTMA)

OSS/BSS Integration •  Customer Service Assurance

•  Perfomance Monitoring

•  Service Quality Management

•  CRM

Third Parties NTMA

Probe Probe Probe Probe Probe

Network/IT Wired/Wireless Infrasctructure

NM2 philosophy, where we are

Distributed NOC

NM2

NM2

NM2

Other


Traffic Monitoring and Analysis


NM2: Contributions in Traffic Monitoring and Analysis

Ø Heterogeneous Network Scenarios (Home Networks, 3G/4G, Wireless Metropolitan Mesh Networks, Overlay Networks, gaming consoles, PDAs, household appliances, smartphones, etc)

ü Traffic Capture ü Traffic Characterization

•  Novel applications (IPTV, games, streaming video, social networks, etc.)

• Malware traffic ü Traffic Modeling ü Traffic Generation and Active Probing ü Traffic and Service Classification

•  New techniques for traffic classification ü Security and Anomaly Detection ü Analysis and Detection of Network Outages


NM2: Traffic Capture, Characterization and Modeling (1/2)

Ø Why? ü  Application and Service understanding and fingerprinting ü  Security ü QoS requirements ü  Performance Analysis ü  Emulation ü  etc.

Ø What? ü High-Speed Packet Capture (COTS, DAG, etc.) ü  Statistical characterization and modeling of traffic

properties •  Multi-level but with specific focus on packet-level •  Per-single application


Ø How? Ø Capture and Analysis

Ø Plab http://www.grid.unina.it/software/Plab

Ø Characterization Ø Matlab toolset for statistical

analysis of network traffic http://www.grid.unina.it/Traffic/Tools/statools.php

Ø Modeling Ø Statistical Modeling of traffic

sources

NM2: Traffic Capture, Characterization and Modeling (2/2)

Hidden States

IPT and PS conditional distributions

Hidden Markov Models for different network applications


NM2: Active Probing and Application Traffic Generation (1/4)

Ø What? Generation of realistic traffic replicating as accurately as possible real applications and collection of information on how the single packets have been processed by the SUT (system under test).

Ø Why? Ø  Network Performance Ø  Testing/benchmarking

Ø  Network Infrastructure

Ø  Device capabilities Ø  Quality of Service

(QoS) architectures Ø  Queuing disciplines Ø  Traffic shapers Ø  Etc.

Traffic generation scenario



Ø How? ü D-ITG (Distributed Internet Traffic Generator)

ü http://www.grid.unina.it/software/ITG ü Distributed architecture: traffic senders and receivers can be spread

over the Internet and controlled by a central point ü Generation of traffic according to both statistical models of the

applications and traffic traces of real applications ü High performance, accuracy and flexibility ü Different kinds of hardware and operating systems supported

Trace-based

Analytical model-based

Application-level

Flow-level

Packet-level

Measurement of performance indicators

Open-loop Closed-loop

Automated & Configurable

Repeatabile


Ø Since year 2003 D-ITG has being: §  Used for the Italian WiMax experimentations (FUB) §  Used for Magnets Network (Berlin) design and testing §  Used in more than 20 EU research projects (Demo) §  Used by more the 50 companies and Telcos for testing their

networking solutions §  Used by NASA for the NASA Crew Exploration Vehicle (CEV)

Space communication link sizing §  Used in Labs for CISCO certifications §  Cited in more than 300 papers/theses worldwide §  Included in several Linux distributions: Debian, Slax, OpenWRT,

Linux Microcore, etc.



NM2: Traffic and Service Classification (1/3) Ø Why?

ü  Accounting ü QoS ü  Security ü Network Analysis ü  etc.

Ø What? Traffic and Service Classification/Identification ü  (new) Payload Inspection ü  Statistical Properties &

Machine Learning ü Multi-Classification

Web report of online traffic classification of a network link


NM2: Traffic and Service Classification (2/3) Ø How?

ü  TIE – Traffic Identification Engine http://tie.comics.unina.it

ü High-speed platform written in C ü Runs on Linux/FreeBSD/MacOSX

ü Modular and Plugin-based ü  Large community

COST-TMA

PacketFilter

SessionBuilder

FeatureExtractor

DecisionCombiner Output

ClassificationPlugin #1

ClassificationPlugin #n...


NM2: Traffic and Service Classification (3/3) Ø Novel Classification Technique: PortLoad*

ü Port-based is fast and privacy-friendly because: •  It needs the 1st packet only •  It uses fixed fields (protocol and port) •  It uses few data It can be considered as a special case of packet-classification

techniques developed for routers, flow-monitors, etc.

ü Payload-based is accurate because relies on application-level headers and other information from the payload

•  Payload-based signatures

Ø Port + Payload = PortLoad Ø Some interest from industry: Telecom Italia, Seven One

Solutions, ACCANTO, Huawei * Patent N.: NA2010AOOOO11


NM2: Security and Anomaly Detection (1/2)

Ø Why? ü  Security of network and critical infrastructures ü  Security of users

Ø What? ü  Traffic Analysis for Network/User Security ü  Network Anomaly Detection ü  Study of Malware Traffic ü  Lawful Interception

Spread of the Slammer Worm in year 2001


NM2: Security and Anomaly Detection (2/2) Ø  How?

ü  Anomaly Detection: traffic analysis through the Wavelet Transform

ü  Study of Malware traffic: characterization and detection of computer worms

ü  Lawful Interception (traffic monitoring, protocol decapsulation, covert channel detection, …) Witty Worm: Joint PS-IPT observed

from MAWI WIDE link

Detection of a Denial of Service attack through Analysis with the Wavelet Transform


NM2: Analysis of Network Outages (1/2)

Ø  BGP ü  BGP updates from route collectors of RIPE-NCC RIS

and RouteViews ü  We combined information from both databases ü  Graphical Tools: REX, BGPlay, BGPviz

Ø  Active Traceroute Probing

ü  Archipelago Measurement Infrastructure (ARK)

ü  Manually-initated traceroutes

Ø  Internet Background Radiation ü  Traffic reaching the UCSD Network Telescope ü  Capable of revealing different kinds of blocking


NM2: Analysis of Network Outages (2/2)

Telescopes vs BGP Ø Contrasting telescope traffic with

BGP measurements can reveal a mix of blocking techniques that cannot be discovered by looking only at BGP

Ø E.g. the second Libyan outage involved overlapping of BGP withdrawals and packet filtering

0

1

2

3

4

5

6

7

8

02-18 12:00

02-19 00:00

02-19 12:00

02-20 00:00

02-20 12:00

02-21 00:00pa

cket

s pe

r sec

ond

0

2

4

6

8

10

12

14

02-18 12:00

02-19 00:00

02-19 12:00

02-20 00:00

02-20 12:00

02-21 00:00

num

ber o

f vis

ible

pre

fixes

AS30981 AS6762 AS21003


Network Measurements


NM2: Contributions in Network Measurements Ø  Network Performance Analysis and Improvement

ü Hybrid approaches (both active and passive) ü QoS, QoE, KPI ü  Informed diversity for performance improvement ü Compression and Reduction of network data

Ø  Broadband Benchmarking ü  In terms of both QoS parameters and protocols

Ø  Network Mapping ü Hybrid and Distributed approaches (routers, links, subnets) ü Accuracy, Discovery time, Intrusiveness

Ø  Bandwidth Monitoring ü Wired and Wireless network scenarios ü Distributed and Hybrid approaches ü Accuracy, Discovery time, Intrusiveness


Ø  Innovative measurement techniques and approaches ü Active → purposely forge synthetic traffic ü Passive → exploit user generated traffic

Ø Able to work in emerging network scenarios ü 3/4G cellular networks, satellite networks, wireless mesh

networks, etc.

Ø Monitored parameters ü One-way delay, round trip time, delay variation (aka jitter),

latency, packet loss, shaping rate, packet reordering, TCP performance (e.g., 0-byte connections, reset segments, out-of-order segments, retransmitted segments, 1-Byte segments retransmitted), etc.

ü Specific Application Performance (DNS, Web, VoIP, IPTV, etc.) ü KPIs synthesized from the parameters above

NM2: Network Performance Analysis (1/3)


A novel technique called Multi-layer Root Cause Analysis of TCP connections (MRCA)*

Ø Works analyzing the traffic generated by network users

Ø Allows to infer the performance of the TCP connections and to determine the associated root causes (network, application, OS configuration, etc.)

Ø Improves and integrates different techniques proposed in literature providing an approach integrating different point of view: aggregate, connection, and host Ø Some interest from the industry: Telecom Italia, Skylogic, ACCANTO, Telefonica O2, etc.

* Patent Under Submission



Ø Monitoring and modeling losses ü Characteristics of the loss process on the Internet and on satellite networks

Ø Detecting and analyzing middleboxes


Internet (through PlanetLab) Satellite network

The effect of a shaper in a cellular network The effect of a PEP in a

satellite network


Informed time diversity Ø Allows to reduce loss burstiness, thus improving application performance Ø We developed an application to use the interleaving in real networks

ü Realizes block interleaving ü Has measurement capabilities to automatically configure and adapt to varying network conditions

NM2: Network Performance Improvement (1/2)


Informed space diversity Ø Allows to improve performance and reliability using multiple

paths Ø A new packet scheduling policy measuring network status

ü Working at IP layer with decisions on a packet-by-packet basis

Ø A tool to apply path diversity on real networks

NM2: Network Performance Improvement (2/2)


NM2: Compression and Reduction of network data Ø  Challenges and obstacles due to huge amount of

monitoring data (OSNs, p2p, high speed links, etc) from both active and passive approaches

Ø  Compression ü  Reduced memory footprint for

stored data ü  A set of operations with reduced

time complexity on coded data Ø  Reduction

ü  There is no need to consider the entire data sets in the processing stage

ü  Entropy-based methodology to reduce network traffic data

ü  Off-line approach

more details, to answer a specific query that can be put inthe form Y = PX , one can use P instead of P and solveY = PX = TCX . Remember that ||T ||0 and ||C||0 areminimized by construction. Therefore, computing the productCX requires at most ||C||0 multiplications of coefficients.Similarly, computing Y = T (CX) requires no more than||T ||0 multiplications. Thus, the complexity of answering aquery that can be put in the form Y = PX is equal to||T ||0 + ||C||0 operations.Using this factorized format allows to answer a range of

queries. For instance, one can answer any max-k transactionquery to find the k largest transactions in the log file. Thiscan be solved by finding the k largest value of the N ! 1row of P that corresponds to the load. One can similarlyfind the total usage of a specific srcID, by summing allbytes value P (3, i) for which P (1, i) = srcID. The matrixC points to which patterns in T the user calls upon. Thussimilar users will have similar coefficient in the C matrix, andcan be identified by observing this sparse matrix. Conversely,the underlying matrix of patterns T embeds some overallbehavior of the system and can be used to identify abnormalusage. In particular, if after computing T over some period oftime ! at regular intervals, one sees dramatic changes in thecomposition of T , say min! ||T (t2)"!T (t1)||2 > " where !is a column permutation and " a threshold, then it might pointto some abnormal behavior in the system and call for someinvestigation.

In order to compute T and C, we use the techniqueproposed by Zujovic et al [5] in the context of pattern matchingalgorithms (applied to query-by-example image retrieval).

III. EXPERIMENTAL EVALUATION

To evaluate the performance of our technique, we haveconsidered its efficiency in terms of memory footprint reduc-tion (Compression Ratio) and different consequences of theapproximation (Bytes error, ID error, URL error). For theevaluation of the memory footprint, the compression ratio iscompared with the output of the general purpose compressionutility bzip2, employing the Burrows-Wheeler block sortingtext compression algorithm, and Huffman coding. The reportedresults are obtained by varying the value of #, that controlsthe amount of distortion allowed in the approximated factor-ization algorithm (the higher the value of #, the higher thetolerated approximation, but also the more sparsity induced inthe representation matrix); preprocessing stages are identical,with a fixed URL threshold2, while # is varied in the set{0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1}.

A. Data set

For a proof of concept of the effectiveness of our technique,we use a real traffic trace: the considered data cover atime span of 1 hour, presenting 26965 sessions, with 110different source IDs exchanging 907.024 MB of data with

2We performed an analysis of the effects of the filtering threshold, andfound that 5 is the maximum value for which the accuracy is not significantlyaffected: all presented results refer to this value of URL threshold.

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

0.001 0.01 0.1 1

Com

pre

ssio

n R

atio

lambda

CCS/fullbzip2full/fullbzip2flt/full

Tot/full

Fig. 1. Compression Ratios

1771 different destination URLs. The data set has the formatof a log file, each record of which represents a single HTTPsession, and is constituted by four fields: timestamp (in UNIXepoch time, µs precision), source ID, destination URL, load(in bytes).

B. Results

1) Compression Ratio: The total size of the compressedversion, as well as the size of specific components, is com-pared against the size of the original data. The quantitieswhose ratio is considered are: CCS - size in bytes of Com-pressed Column Sparse representation of C matrix alone; Tot- sum of the size in bytes of CCS, T matrix, bzip2-compressedordered list of URLs, bzip2-compressed ordered list of sourceIDs (this, with a few metadata, is all is needed to rebuild theoriginal data); bzip2flt - size in bytes of the URL-filtered andbzip2-compressed version of the original data; bzip2full - sizein bytes of the bzip2-compressed version of the original data.The CCS component is calculated in bits as:

nnz · (basesize+ #log2(cols)$) + #log2(nnz)$ · (rows + 1)

where nnz is the number of non-zero elements of the matrix,rows and cols are the dimensions of the matrix, and #·$ is theceiling function. This value corresponds to the size occupancyof a sparse matrix, represented as Compressed Column Sparse(or Compressed Sparse Column [6]), where indexes are binarycoded, and each element is represented with basesize bits.In the considered case, basesize is 32, rows and cols arerespectively 3 and 23516. The matrix T is represented asN · K values of length basesize bits each. Fig. 1 shows thecompression ratio as a function of the granularity parameter#. As expected, by increasing the approximation granularity,more sparsity is found in the factor matrix C, and therefore thesize occupancy for CCS representation decreases, causing thesize of total representation to gracefully decrease for growingvalues of #.2) Error on URL decoding: Error on URL decoding is

calculated as the ratio of entries with mistaken URLs versus

the third row of Table II, we obtain a reduction of 59%. In Fig.3(b) and 5(b)) we can see that the approximation is quite goodfor over the 99.9% of the distribution, and mean and standarddeviation are well approximated (see third row of Table II).Figs. 4 and 5(b) show that the two distributions are close inthe main part and in the tail too.

d) PSO: We sketch the Marginal Utility against thenumber of samples in Fig. 3(c). The QQ-plot in Fig. 3(d)shows a good approximation up to about 500 bytes, whichaccounts for 99.2% of the original data set. In the fourth rowof Table II a summary of the conducted analysis is reported.

0 0.005 0.01 0.015 0.02 0.0250

100

200

300

400

500

[s]

Entire setReduced set

0 2 4 6 8 10 12x 10

!3

0

500

1000

1500

[s]


0 20 40 60 80 100 120 1400

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Bytes


0 50 100 150 200 250 3000

0.005

0.01

0.015

0.02

Bytes

Entire SetReduced Set

Fig. 4. Counter-Strike PDFs (clockwise: IAT, IDT, PSO, PSI).

Finally, the outgoing traffic is well approximated by anIDT/PSO series of about 4 millions of samples.

0 0.1 0.2 0.3 0.4 0.5 0.6!8

!6

!4

!2

0

[s]


(a) IAT

0 0.2 0.4 0.6 0.8 1 1.2!8

!6

!4

!2

0

[s]


(b) IDT

Fig. 5. CCDF of Counter-Strike.

2) Wavelet Analysis of CS Reduced Data Sets: The reduc-tion criterion we use here is based on the analysis of themarginal distributions of traffic data samples. But, in the studyof network traffic also temporal structures and dependencies(e.g. long range dependence and scaling behavior) can beof interest. In this section, we briefly show a time-frequencyanalysis based on the Wavelet Transform, revealing similarbehaviors between the entire and reduced data sets. We usethe Logscale Diagram (LD), which shows the trend of theenergy of the wavelet coefficients at each time scale, allowingto estimate the scaling behavior of the considered process andthe corresponding Hurst parameter (see [12]).

From the Counter-Strike IAT and IDT data sets, we calcu-lated the packet rate time series, with a period of 1 ms, oftraffic flowing in both directions (to and from the server).

2 4 6 8 10 12 14 16 18 20 22

!2

0

2

4

6

8

10

12

14

Octave


(a) output packet rate

2 4 6 8 10 12 14 16 18

!4

!2

0

2

4

6

8

10

12

Octave


(b) input packet rate

2 4 6 8 10 12 14 16 18 20 2212

14

16

18

20

22

24

26

28

30

Octave


(c) output byte rate

2 4 6 8 10 12 14 16 18 20 22

8

10

12

14

16

18

20

22

24

Octave


(d) input byte rate

Fig. 6. Logscale Diagram comparison of CS reduced and original data sets.

Let Sj , S1j be the logarithms of the energy of the wavelet

coefficients at scale j of respectively the entire and reduceddata sets. We found Sj =!j S1

j (for j = 1, ..., 17 in the caseof IAT and for j = 1, ..., 22 in the case of IDT) where the=!j operator takes into account their confidence intervals.This can be seen in the LDs in Figs. 6(a) and 6(b), where,at each scale, the confidence intervals of the two diagramsalways intersect. It is worth noting that we found the sameresults for the byte rate time series (Figs. 6(c) and 6(d)), whichwere obtained by combining information from the IAT andPSI series as well as IDT and PSO series. This comparisonis indeed important, since it is highlights properties of realnetwork traffic by combining information on packet arrivaltimes and their size. The analysis in this section shows that,for the considered data sets, the reduction did not heavily affectthe traffic temporal structures.

3) Effects of the Reduction on the Autocorrelation: Besidethe wavelet spectrum of the packet rate and byte rate series,we study the behavior of the Autocorrelation function for bothcomplete and reduced data sets of IDT and PS. This is doneto further assess the impact of the adopted approach on thesamples temporal behavior and their mutual dependencies.

In Fig. 7 the autocorrelation plots, until lag 100, are reportedfor all the data traces. As shown, for all the consideredvariables, the autocorrelation values of the reduced sets arevery close to those of the original sets. In particular, theRoot Mean Square (RMS) value of the error introduced bythe reduction ranges from 0.0128 (for the IAT series) to0.0232 (for the PSO series). To better observe the effect ofthe reduction, a zoomed view of the IAT autocorrelation isreported in Fig. 8. IAT is the variable that presents morecorrelation among the samples, also, its autocorrelation plotreveals an oscillating trend. The view of Fig. 8 allows to verifythat the trend of the reduced-set autocorrelation is very similarto that of the original set. This witnesses that the temporalstructure of the samples is preserved even in the presence ofa such particular behavior.

!"#$%&'((%)*+)%,-,*.%/-$%,**.%.*0#*/*1%-)%)"*%1#.*2)#34%3&%5666%7388'4#2-)#34$%932#*):%$';<*2)%8-))*.%*+,*.)$%&3.%,';(#2-)#34%#4%)"*%577%=>>?%,.32**1#4@$A%


NM2: Broadband mapping (1/4)

Ø  Different approaches Ø  Web-based (Speedtest.net, Netalizr, ...)

Ø  easy to use Ø  one-shot measure Ø  affected by interferences

Ø  Client-based (Grenouille, Isposure, HoBBIT, ...) Ø  repeated/periodical measures Ø  easy large scale deployments Ø  active only when the PC is turned on Ø  unable to account for interferences

Ø  Router-based (SamKnows, BISMark) Ø  continous periodical measures Ø  observes all traffic passing through network Ø  can take into account interfereces Ø  difficult to obtain large scale deployments

ISP

Home

network

Local loop

Modem

Router

Host

Ø  Measuring from the edge → Independent point of view



BISMark (router-based) Ø Linux-based firmware

ü  customized OpenWRT distro ü  Netgear WNDR 3700v2

Ø On-demand access to the router console

Ø Active and passive measurements

Ø Current deployments ü  16 routers in Altanta ü  15 routers in Cape Town

http://projectbismark.net

HoBBIT (client-based) Ø Multi-platform application

ü  based on Qt libraries Ø Extensible measurement

framework Ø Supports any underlying

measurement tool Ø Active measurements Ø Current deployment

ü  ~100 users in Italy

http://hobbit.comics.unina.it



BISMark Ø Network measurements taken from the home gateway Ø Both active and passive measurements Ø Main features

ü On-demand remote router control/update

ü Measurements synchronization Ø Allows to monitor

ü Factors affecting performance (Local loop, ISP policies, Home network) ü Usage profiles


NM2: Broadband mapping (4/4) Ø Network measurements taken from the users' PC

ü large scale deployments

Ø Active measurements using standard tools ü extensible measurement framework ü geolocation and mapping ü fine-grained management

Ø Main features ü multi-platform ü automatic updates ü per-application measurements

Ø Users can ü monitor their Internet connection ü compare results with others in the

same location


NM2: Network Mapping (1/3) Ø Why?

ü Network control and management •  Fault isolation, performance analysis, service locations, etc.

ü Network simulations •  It is difficult to generate realistic topologies

ü Network aware applications •  E.g. to improve the performance

Ø What? ü Automatic discovery of network maps in terms of: routers, links,

subnets, layer-2 devices, etc. ü Achieving

•  Completeness (i.e. discover the entire topology) •  Accuracy (i.e. make no mistakes) •  Low intrusiveness (i.e. reduce both the discovery duration and the

traffic overhead) •  Integration with Network Inventory solutions


NM2: Network Mapping (2/3) Ø How?

ü Combining multiple passive/active methodologies and techniques ü Hybrid approaches ü Novel techniques based on: IGMP, ParisTraceroute, IP Options, ... ü Hynetd (single vantage point)

•  http://www.grid.unina.it/software/TD ü MERLIN (multiple vantage points)

•  http://svnet.u-strasbg.fr/merlin


NM2: Network Mapping (3/3) MERLIN: MEasure the Router Level of the Internet Ø Target a specific Autonomous System network Ø Multiple techniques integrated and optimized

Ø  Improved IGMP probing ü Paris traceroute ü Alias resolution

Ø Several input sources Ø BGP dumps, CAIDA Archipelago

datasets, MaxMind repositories, ... Ø Geo-Location, DNS mapping,

IPtoAS mapping, ...

MERLIN Monitor

MERLIN Coordinator


NM2: Bandwidth Monitoring (1/2) Ø Why?

ü  Network planning ü  QoS ü  Admission Control ü  Support several kinds of applications

(P2P sharing, overlay networks, CDN, streaming, etc.)

Ø What? Estimation of capacity and available bandwidth in modern heterogeneous networks ü  Optimized approaches for each network scenario:

wired, wireless, broadband access, mixed ü  Allowing for different deployments:

single probe / edge probes / instrumented path ü  Tunable in intrusiveness / accuracy / response speed


NM2: Bandwidth Monitoring (2/2) Ø How?

Measurement platform: UANM (Unified Architecture for Network Measurement) http://grid.unina.it/Traffic/uanm.php ü  Distributed ü  Equipped with state-of-art

techniques ü  Plugin-based (easily expandable

with experimental or cutting-edge techniques)

ü  Decentralized synchronization for interference avoidance

ü  API provided for embedding in applications, monitoring systems, appliances


Research Collaborations (not exaustive list)

Ø Cooperative Association for Internet Data Analysis (CAIDA), San Diego,USA

Ø Georgia Tech, Atlanta, USA Ø Eurécom, Sophia Antipolis, France Ø Telefonica O2, (Spain and Germany) Ø TELECOM ParisTech (formerly known as ENST), France Ø Docomo Labs, Palo Alto, Stanford, USA Ø Deutsche Telekom Laboratories, Berlin, Germany Ø UCL, University of Louvain-la-neuve (Belgium) Ø Universitat Politècnica de Catalunya (Barcelona, Spain) Ø etc.


Contacts

Antonio Pescape' Dipartimento di Informatica e Sistemistica University of Napoli ''Federico II'' Via Claudio, 21 - 80125, Napoli (Italy) [Room n. 3.10] tel. +39 081 7683856 fax +39 081 7683816 e-mail : [email protected] (or [email protected]) Personal web-page: http://wpage.unina.it/pescape Teaching web-site (in Italian): http://www.docenti.unina.it/antonio.pescape


Selected Publications (not exaustive list) Ø  Srikanth Sundaresan, Walter de Donato, Nick Feamster, Renata Teixeira, Sam Crawford,

Antonio Pescapè, "Broadband Internet Performance: A View From the Gateway", to appear in ACM SIGCOMM 2011 proceedings, Toronto, ON, Canada, August 15-19, 2011.

Ø  A. Dainotti, A. Pescapé, K. C. Claffy, “Issues and Future Directions in Traffic Classification", IEEE Network, 2011, to appear

Ø  Pietro Marchetta, Pascal Mérindol, Benoit Donnet, Antonio Pescapé and Jean-Jacques Pansiot. "Topology Discovery at the Router Level: A New Hybrid Tool Targeting ISP Networks". IEEE Journal on Selected Areas in Communication (JSAC), Special Issue on Measurement of Internet Topologies, 2011, to appear

Ø  Alessio Botta, Antonio Pescape', Vinh Bui, Weiping Zhu, "A Markovian Approach to Multi-path Data Transfer in Overlay Networks'', IEEE Transactions on Parallel and Distributed Systems, vol.21, no.10, pp.1398-1411, Oct. 2010

Ø  Alessio Botta, Alberto Dainotti, Antonio Pescape', "Do You Trust Your Software-based Traffic Generator?'', IEEE Communications Magazine, vol.48, no.9, pp.158-165, Sept. 2010.

Ø  A. Botta, R. Canonico, G. Di Stasi, A. Pescapè, G. Ventre, S. Fdida., "Integration of 3G connectivity in PlanetLab Europe - A step of an evolutionary path towards heterogeneous large scale network testbeds", ACM Springer Mobile Networks and Applications Journal, Special Issue on "Advances In Wireless Test beds and Research Infrastructures", Volume 15, Issue 3, June 2010, Pages 344-355.

Ø  Alberto Dainotti, Antonio Pescapè, Giorgio Ventre, "A cascade architecture for DoS attacks detection based on the wavelet transform'', Journal of Computer Security, Volume 17, Number 6/2009, Pages 945-968


Selected Publications (not exaustive list) Ø  Marco Mellia, Antonio Pescapè, Luca Salgarelli, Traffic classification and its applications to

modern networks, Computer Networks, Volume 53, Issue 6, 23 April 2009, Pages 759-760. Ø  A. Thomas Silverston, Olivier Fourmaux, Alessio Botta, Alberto Dainotti, Antonio Pescapè,

Giorgio Ventre, Kavè Salamatian, " Traffic Analysis of Peer-to-Peer IPTV Communities ," Computer Networks, Volume 53, Issue 4, 18 March 2009, Pages 470-484.

Ø  Alessio Botta, Antonio Pescapè, Giorgio Ventre, "An approach to the identification of network elements composing heterogeneous end-to-end paths", Computer Networks, Volume 52, Issue 15, 23 October 2008, Pages 2975-2987, Elsevier.

Ø  A. Dainotti, A. Pescapè, P. Salvo Rossi, F. Palmieri, G. Ventre, "Internet Traffic Modeling by means of Hidden Markov Models"; Computer Networks (Elsevier), Volume 52, Issue 14, 9 October 2008, Pages 2645-2662

Ø  A. Botta, A. Pescapè, R. Karrer, “High-speed backhaul networks: myth or reality?”, Computer Communication Journal (Elsevier), Volume 31, Issue 8, 25 May 2008, Pages 1540-1550.

Ø  A. Pescapè, “Entropy-Based Reduction of Traffic Data”, IEEE Communications Letters, pp. 191-193, Vol.11, No.2 - February 2007.

Ø  S. Avallone, D. Emma, A. Pescapè, and G. Ventre, “Performance evaluation of an open distributed platform for realistic traffic generation”, Performance Evaluation (Elsevier), ISSN: 0166-5316 – Vol. 60, Issues 1-4, May 2005, pp 359-392

Ø  Massimo Bernaschi, Filippo Cacace, Giulio Iannello, Antonio Pescapè, and Stefano Za, “Seamless Internetworking of WLANs and Cellular Networks: architecture and performance issues in a Mobile IPv6 scenario”, IEEE Wireless Communication Magazine (WCM) Journal, pp. 73-80, June 2005


Selected Publications (not exaustive list) Ø  A. Dainotti, A. Pescapè, C. Sansone, "Early Classification of Network Traffic through Multi-

Classification", Third International Workshop on Traffic Monitoring and Analysis (TMA'11) - April 2011, Vienna (Austria).

Ø  A. Botta, A. Pescapè, "Monitoring and measuring wireless network performance in the presence of middleboxes", The 8th International Conference on Wireless On-demand Network Systems and Services (WONS), Bardonecchia (TO), Italy, January 2011. (Download the poster).

Ø  A. Pescape', D.Rossi, D. Tammaro, S. Valenti, "On the Impact of Sampling on Traffic Monitoring and Analysis", 22nd International Teletraffic Congress, September 7 - 9, 2010 in Amsterdam, The Netherlands.

Ø  A. Botta, A. Pescape', G.Ventre, E. Biersack, S. Rugel, "Performance footprints of heavy users in 3G networks via empirical measurement", The 6th International workshop on Wireless Network Measurements, May 31st, 2010, Avignon, France.

Ø  A. Botta, A. Pescapè, G. Aceto, M. D'Arienzo, "UANM: a platform for experimenting with available bandwidth estimation tools", 15th IEEE Symposium on Computer and Communications, June 2010 Riccione (ITALY)

Ø  A. Dainotti, F. Gargiulo, L. Kuncheva, A. Pescapè, C. Sansone, "Identification of traffic flows hiding behind TCP port 80", IEEE ICC 2010 - May 2010, Capetown (South Africa)

Ø  G. Aceto, A. Dainotti, W. de Donato, A. Pescapè, "PortLoad: taking the best of two worlds in traffic classification", IEEE INFOCOM 2010 - WIP Track - March 2010, San Diego (CA, USA)

Ø  V. Carela-Español, P. Barlet-Ros, M. Solè-Simò, A. Dainotti, W. de Donato, A. Pescapè, "K-dimensional trees for continuous traffic classification", 2nd International Workshop on Traffic Monitoring and Analysis (TMA'10), Zurich, Switzerland, April 7, 2010.


Selected Publications (not exaustive list) Ø  A. Dainotti, W. De Donato, A. Pescapè “TIE: a Community-Oriented Traffic Classification

Platform", International Workshop on Traffic Monitoring and Analysis (TMA'09) @ IFIP Networking 2009 - May 2009, Aachen (Germany)

Ø  A. Dainotti, W. De Donato, A. Pescapè, P. Salvo Rossi, "Classification of Network Traffic via Packet-Level Hidden Markov Models", IEEE GLOBECOM 2008 - Dec 2008, New Orleans (LA, USA)

Ø  Alessio Botta, Walter de Donato, Antonio Pescapè, Giorgio Ventre, "Networked Embedded Systems: a Quantitative Performance Comparison", IEEE Globecom 2008, New Orleans (LA), USA, 30 November - 4 December, 2008.

Ø  Alessio Botta, Roberto Canonico, Giovanni Di Stasi, Antonio Pescapè, Giorgio Ventre, "Providing UMTS connectivity to PlanetLab nodes", 3rd International Workshop on Real Overlays & Distributed Systems, collocated with ACM CoNEXT 2008, Madrid, Spain, 9 - 12 December, 2008.

Ø  Alessio Botta, Antonio Pescapè, Vinh Q Bui, Weiping Zhu, "An MDP-based Approach for Multipath Data Transmission over Wireless Networks", 2008 IEEE International Conference on Communications (ICC 2008), page(s): 268 - 274

Ø  M.K. Afzal, Aman-Ullah-Khan, A. Pescape', Y. Bin Zikria, S. Loreto, "SCTP vs. TCP Delay and Packet Loss," Multitopic Conference, 2007. INMIC 2007. IEEE International , vol., no., pp.1-5, 28-30 Dec. 2007

Ø  Roger Karrer and Antonio Pescape', "2nd generation wireless mesh networks: technical, economical and social challenges". In Proceedings of the 2007 IEEE International Conference on Future Generation Communication and Networking, Jeju Island, Korea, December 2007.


Selected Publications (not exaustive list) Ø  A. Botta, W. de Donato, A. Pescapé and G. Ventre, “Discovering Topologies at Router Level:

Part II”, Globecom 2007, Washington, D.C., 26-30 November, 2007 Ø  Alessio Botta, Antonio Pescapè, Giorgio Ventre, Roger P. Karrer, "High-speed wireless

backbones: measurements from MagNets” in proceedings of the Fourth IEEE International Conference on Broadband Communications, Networks, and Systems (Broadnets), September 2007, Raileigh, North Carolina (USA).

Ø  Vinh Q Bui, Weiping Zhu, Antonio Pescape', Alessio Botta, "Long Horizon End-to-End Delay Forecasts: A Multi-Step-Ahead Hybrid Approach", 12th IEEE Symposium on Computers and Communications, 2007

Ø  Roger P. Karrer, Istvan Matyasovszki, Alessio Botta, Antonio Pescapè, "MagNets - experiences from deploying a joint research-operational next-generation wireless access network testbed”, TRIDENTCOM 2007, May 2007, Orlando, Florida (USA).

Ø  Alberto Dainotti, Antonio Pescapè, Giorgio Ventre, "Worm Traffic Analysis and Characterization", 2007 IEEE International Conference on Communications (ICC 2007)

Ø  A. Dainotti, A. Pescapè, P. Salvo Rossi, G. Iannello, G. Ventre, F. Palmieri “An HMM Approach to Internet Traffic Modeling", 2006 IEEE Globecom Conference, Quality, Reliability and Performance Modeling for Emerging Network Services Symposium

Ø  A. Dainotti, A. Pescapè, G. Ventre, “Wavelet-based Detection of DoS Attacks", 2006 IEEE Globecom Conference, Network Security Systems Symposium

Ø  Giulio Iannello, Francesco Palmieri, Antonio Pescapè, and Pierluigi Salvo Rossi,“End-to-End Packet-Channel Bayesian Model applied to Heterogeneous Wireless Networks”, IEEE Globecom 2005 General Conference - ISBN 0-7803-9415-1 - December 2005, St. Louis (MO, USA)


Large Scale Projects


NM2: Large Scale Projects Ø  BISMark

ü  Router-based platform for performing measurements of ISP performance, as well as traffic inside the home

ü  http://projectbismark.net ü  http://www.bufferbloat.net

Ø  HobbIT ü  User-based platform for performing measurements of ISP performance ü  http://hobbit.comics.unina.it

Ø  MERLIN ü  Distributed platform to MEasure the Router Level of the Internet ü  http://svnet.u-strasbg.fr/merlin

Ø  MagNets ü  Berlin Wireless MAN design and analysis ü  http://www.net.t-labs.tu-berlin.de/~roger/magnets.html

Ø  Distributed Monitoring and Measurements Architectures for ü  Operational 3G Networks ü  Operational Satellite Networks


NM2: BISMark (1/3) Ø Network measurements taken from the home gateway

ü A vantage point into the home network

Ø Both active and passive measurements ü Customized to user profile ü Data anonymization

Ø Main features ü On-demand remote router

control/update ü Measurements synchronization

Ø Allows to monitor ü Factors affecting performance

•  Local loop •  ISP policies •  Home network

ü Usage profiles

NOX Box 500Mhz Geode CPU 256 MB RAM 2GB Flash

Custom Debian OS

Netgear WNDR3700 680Mhz MIPS CPU 64 MB RAM 8MB Flash

Custom OpenWrt OS

Currently Supported devices


NM2: BISMark (2/3)


NM2: BISMark (3/3) Current worldwide deployment status

  2 management servers

  more than 50 routers

  more than 50 measurement servers (Universities, MLab)


NM2: HobbIT (1/2) Ø Network measurements taken from the users' PC

ü large scale deployments

Ø Active measurements using standard tools ü extensible measurement framework ü data geolocation and mapping ü fine-grained resource management

Ø Main features ü multi-platform client ü automatic updates ü per-application measurements

Ø Users can ü monitor their Internet connection ü compare results with others in the

same location

About 90 clients in Italy


NM2: HobbIT (2/2)


NM2: MERLIN MEasure the Router Level of the Internet Ø Target: a specific Autonomous System network Ø Efficient joint among the state-of-art techniques in the router

level topology discovery field: ü  Improved IGMP probing ü Traceroute (paris-variant) ü Alias resolution technique

Ø Optimizations: ü Overcome technique's limitation while preserving benefits ü Limit the intrusiveness with a central smart coordination

Ø Several input sources: BGP dumps, CAIDA Archipelago datasets, MaxMind repositories, ...

Ø Geo-Location, DNS mapping, IPtoAS mapping, ...


NM2: MERLIN

Sprint Network

MERLIN Monitor

Internet

I want... Sprint?!

MERLIN Coordinator


MagNets: Berlin Wireless MAN

Specific active measurement techniques designed to infer Ø  Throughput, latency, and loss of the links Ø  Impact of enhanced transmission modes Ø  Impact of the environment

http://www.net.t-labs.tu-berlin.de/~roger/magnets.html

Comprises a wireless backbone and different wireless mesh networks

Joint research with Deutsche Telekom Laboratories, Berlin


Operational 3G networks

Ø Different kinds of (passive ) analyses on the user traffic Ø  Traffic classification and application identification Ø  TCP performance Ø Root cause analysis Ø  Impact of middleboxes

Ø On different operational networks from different European telecom operators


Operational satellite networks

Ø Distributed architecture for passive and active monitoring and measurements Ø Different kinds of passive and active analyses Ø  End-to-end TCP and UDP

performance Ø  TCP performance through

passive analysis Ø  Influence of traffic shaping

mechanisms and middleboxes Ø  Impact of meteorological

conditions on performance

http://broadband-satellite.atrexx.com/

Network Monitoring and Measurements at University of Napoli

Documents

Transcript of Network Monitoring and Measurements at University of Napoli