Implementing and Deploying Network MonitoringService Oriented Architectures

Brazilian National Education and Research Network Measurement Experiments

Leobino Sampaio, Ivo Koga, Rafael Costa, Herbert Fausto Vetter, Guilherme Fernandes, Murilo VetterMonteiro, Jose A. Suruagy Monteiro Nucleo de Processamento de Dados - NPD

Computing and Networking Research Group - NUPERC Universidade Federal de Santa Catarina - UFSCUniversidade Salvador (UNIFACS) Campus Universitario Trindade

Rua Ponciano de Oliveira, 126 - Rio Vermelho 88040-900 - Florian6polis - SC - Brazil41950-275 - Salvador - BA - Brazil. {fvetter, fernandes, murilo}@npd.ufsc.br

{leobino, ivo.koga, rafael.costa, herbert.monteiro,suruagy } @unifacs.br

Abstract-Recently there is a growing interest in accessing In order to face the problems related to networknetwork measurement and status data across multiple domains, heterogeneity, some initiatives have adopted Web Services as ain order to detect eventual network problems which can hinder solution for tools integration, measurement data correlation andthe use of advanced applications. Diversity on network visualization [1]. The implemented services expose thetechnologies, technical solutions and network managers add to monitoring tools functionalities in a standard format, also beingthe complexity of providing such information. In order to platform independent and generally taking advantage of thefacilitate the performance data access and visualization, some web ubiquity. In these solutions, a large number of servicesinitiatives have been directed to the development and deployment have been developed by different administrative domains withof network monitoring service oriented architectures (NMSOA). different purposes. A service can have one or more providersThis paper aims at presenting RNP's End-to-end PerformanceWorking Group activities on implementing and deploying two of and the clent can select them based on ssues related to servicethese architectures: piPEs-BR and perfSONAR. Initially it is q a cpresented an overview of related work, followed by a detailed The use of Web Services is, in fact, one of the best optionspresentation of these architectures. perfSONAR is being jointly available for services integration, since they rely on the use ofdeveloped by Internet2, Geant2, RNP, and the University of broadly accepted protocols like HTTP. Therefore, under thisDelaware. We focus on the components developed by our WG perception, some efforts have been dedicated in the direction ofand our experience on deploying this measurement environment the development of network monitoring service orientedat Ipe and Clara networks. The paper concludes with a architectures (NMSOA).presentation of the major learned lessons with these experiencesand which can be useful to deploy such environment in other A flow monitoring model based on a SOA extension whereNRENs, followed by our concluding remarks and pointers for the flow collectors are managed through the use of Webfuture directions. Services is proposed in [2]. Another effort in this area is the

development of the NetCollector Java based tool' that aims atKeywords - multi-domain monitoriring; network measurement; providing a monitoring environment to manage NetraMet [3]

collectors through Web Services communication. Anothereffort that provides a dynamic and scalable system adopting

I. INTRODUCTION Web Services is MonALISA (Monitoring Agents in a LargeThe increased interest in performance data that comes from Integrated Services Architecture) [4]. MonALISA is a scalable

networks located in different administrative domains reflects in dynamic distributed services architecture that provides to eachthe development of monitoring environments that cope with a service the ability to register itself and then to be discoveredvariety of network technologies, technical solutions and and used by any other services. With this capability, the usernetwork management systems. In general, in each can get data from a measurement data access service andadministrative domain there are various measurement tools visualize them using, for example, MonALISA's GUI.with different purposes in use due to network managers' Following the interest in the availability of networkconvenience in adopting tools based on their needs. As a monitoring data, Internet2's End-to-End Performance Initiativeconsequence of this diversity, new challenges arise towards the (E2Epi) initially proposed a scalable and distributed system forintegration of such tools which could help in identifying end- monitoring, testing, and reporting end to end performance,to-end performance problems as well as who iS responsible fortheir solution.

1 NetCollector - http://www.nuperc.unifacs.br/netcollector

1-4244-1 182-3/07/$25.00 ©)2007 IEEE

called E2E piPEs (End-to-End Performance Initative Geant2, ESnet, and the University of Delaware to develop thePerformance Environment System)2. As a step forward, current releases ofperfSONAR.Internet2 and Geant2 joined forces in the GN2 Joint ResearchActivityl (JRA1)3 to develop a document that specifies a All these efforts intend to provide important networkservice oriented measurement infrastructure called General monitoring data to the users, detect and prevent problems, andFramework Design (GFD) [5]. The GFD points out the give information that is, in general, unavailable or muchconceptskneeded to be 1Fo.iTneNMSA d opnt t and difficult to access such as the performance informationconcepts needed to be followed in NMSOA development and between different administrative domains. The infrastructure ofdeployment, under the perspective of measurement information the NMSOA environments gives a layer of services that makebeing exchanged through standardized services, each oneexecuting actions that offer necessary functionalities to provide possble to abstract the internal differences of each component.beer accessibility to the network monitoring information. As In other words it abstracts the interaction and availability ofdepicted in Figure 1. GFD services act as a set of middleware provided dat and open aitservices that facilitate the visualization tools access to applications that interact with itmeasurement tools and data. The remainder of this paper is organized as follows:

Section II presents an overview of networking monitoringNO2C: visualisation

Interface ProjectDatastorage service-oriented architectures; Section III focuses on the caseUser intertaces studies; Section IV presents the lessons learned from the

_______________________________ lmonitoring activities; and Section V presents the paper'sGeneral / \ ~ Iconcluding remarks and future directions.

Desinig AbsactionLayerXv | II. NETWORK MONITORING SERVICE ORIENTED

Scope (red) ARCHITECTURES

Numerous efforts have been devoted to integrate themeasurements tools through the use of Web Services. The

Measurernent (t Lj () Lh<t > < HC lI problem is that most of these initiatives only try to expose thet measurement tools functionalities to provide performance data

Domain A Domain B Domain Cand perform tests. However, tools integration is an importantissue that is necessary to deal with aspects related to test

Available Bandwidth E]One-waydelay One-way delay scheduling, authentication and authorization, as well as data_Measurement Points Measurement Points pe Measurement Points tpe2 storage and visualization. piPEs-BR and PerfSONAR are two

Figure1. General Framework Design[5]. of the architectures that cope with these issues and aredescribed below.

The services defined in the GFD architecture are:Measurement Point (MP) service, used to collect measurement A. piPEs-BRdata; Transformation service (TS), used to pipeline data The piPEs-BR was a system designed following the ideasbetween the other services within the framework; Measurement of the piPEs project from Internet2, which had the purpose toArchive (MA) which stores measurement data collected by the standardize network measurement data recovery andMPs or transformed by the TS; Lookup Service (LS) used to information exchange. This Brazilian version had somediscover and publish services enabling the insertion and query specific features for managing measurements at RNP's Ipeof the environment services; Authentication Service (AS) network, such as the Scheduling and the Test Managementwhich allows the authentication and authorization in the modules. Its architecture can be seen in Figure 2. Threeenvironment and provides decision attributes for what can be environment user's types were envisaged: end-users, advancedshown for a given resource or service; and Topology Service users and administrators/managers.(TopS) that provides information about the network topology. End-users are those who have low level knowledge of

Based on GFD services definitions, the development of a network measurements and just want to know, for example,prototype called perfSONAR (Performance focused Service what are the causes of a problem for running his/her networkOriented Network monitoring ARchitecture) [6] has started application. The interaction between this kind of user and thefollowing the defined services. The perfSONAR initiated environment is achieved through a friendly interface.developing and deploying the basic services MP and MA.

Advanced users are researchers and other users who haveIn the mean time, the Brazilian National Education and an intermediary knowledge of network measurements and want

Research Network (RNP) End-to-End Performance Working to get raw measurement data through an interface with moreGroup (E2E-WG)4 developed and deployed an environment resources than the regular end-users.based on piPEs' principles, named piPEs-BR. When the GFDdocument was made available, the E2E-WG proposed to adjust Finally, the administrators are the ones with full knowledgepiPEs-BR with the development of an environment called of the environment and who schedule tests and manage thepiPEs-BR/GFD aiming at being compatible with perfSONAR network measurement environment, interacting directly withservices. Currently, the E2E-WG partnered with Internet2, the environment through a full resources administration

' interface. Within this interface, the administrators can___________________________configure the MPs through the Measurement Point Dynamic

2 ....Configurator (MPDC) and schedule tests that have to pass aE2E piEs -ttp:/e2epiinteret2.eu/e2eipes/policy check. The scheduled tests must be stored on an agenda

3Joint Research Activity 1: Performance measurement and which triggers the corresponding tests at the appropriate times.monitoring - http://www.geant2.net/server/show/nav.754. An MP specified in the agenda has to do performance tests and

4 hftp://wiki.nuperc.unifacs.br/

29

report the results to a centralizer module which stores the Message: message/storemeasurement data from the environment.

MetadataSometimes it is also possible for end-users or advanced |D|

users to schedule some tests through the tests managementmodule. This module is requested to catch performance data Metadatafrom the environment and do some kind of transformation, butin the case where the environment doesn't have that data, itmakes an automatic test schedule. Data

metadataIdRef IH ~~~~~~~~~~~~~~~~~~~~~~~~~~~IEnvironment DataAdministrators Dynamic

Configuratrl metadata I d Ref 2Network (MPDC)

Visualization administratorEnd-Users Friendly and detection Inteace Figure 3. perfSONAR metadata and data and their reference ids.

interface

Chek Py Furthermore, metadata elements are divided intoCbC ki c.y constituent elements, being the three major groups: subject,

Raw daa End-user A parameters and type as depicted inPAGEREF Figure 4. TheR>da.>linteCatel/ Mjsubject element defines the entity related to the data. TheUsers S 1 Cen llzer parameters element has one or more parameter elements that

define the different conditions to be used, or which were used,Trilggeragenda X for the measurements. The type element, which is defined as

,/ S11 edler _ 11eventType, identifies the type of the requested data. The dataITests | Dot$ already ma ,...*1TVsnarh6ts ~stored element is also subdivided into more specific elements. Themain group of this division is the datum class. Datum elementscan be singleton measurements, events derived from

Through the adoption of Web Services, the piPEs-BR transformations of sets of data, or individual events internal to

environment tried to be scalable and able to provide a way to the system.REFintegrate new network measurement tools as necessary, without Message Storehaving to modify the infrastructure. This environment also dSri string

mretaSdat:aidRef * Sitringtried to have an easier way to find the services provided by the Type: String - lmodules. The E2E-WG investigated the use of a Publish and Metadata DataDiscovery service to search the available Web Services in the MeItadataidRefS tring metadataidngf Stringnetwork monitoring infrastructure. In this sense, the UDDI

S tMetadat(Universal Description, Discovery and Integration) standard [7] Subjectring|was adopted, which is the Web Services standard used for IKd Sting orKeydiscovery and publish services. Parameters Id String

ParametersId -.String

orB. perJSONAR EventTyp CommonTime

String type: StringAs stated before, perfSONAR is a prototype Timerange or

implementation of GFD. It defines the protocols for Datumcommunication, the base services architecture and different orimplementation examples. The perfSONAR protocol is based Datum

time: Timeon SOAP (Simple Object Access Protocol) XML messages and Resultsfollows the GGF NMWG XML schema [8] for networkmeasurements. Hence, perfSONAR uses NMWG messages for Figure 4. perfSONAR message structure.communication between any of the framework components.

Besides the protocol, perfSONAR developers also definedThe key goals of the NMWG XML schema are basic architecture services. Several components were defined

extensibility, normalization, and readability. It breaks to make a generic service which could be used as the basis forrepresentation of performance measurements down into basic any other service in the framework. This effort aimed atelements. These elements can be combined to create derived reducing conception time for the services, while keeping themdata for further analysis. The messages in this schema have at with a known similar structure. Afterwards, perfSONARleast one metadata and one data element: metadata elements developers started the implementation of the services definedcarry information that identifies a set of data related to a by the GFD, to be able to test and validate its concepts. Somespecific event from a particular time span and the data elements Of these services have been developed by RNP developers andcarry the data specified by the related metadata. In the will be presented on the next section.messages, data elements relate to metadata through a metadatareference id, as depicted in Figure 3. 1) RRD MA Service

The first implemented service was the RRD (Round RobinDatabase) Measurement Archive (MA). This service is

30

basically a wrapper around the binary files of the RRDTools5. CNM (Figure 5) started to be developed in 1997 by DFN7It stores and publishes data, which is kept in RRD databases, to provide the German universities and research institutionsthrough a standard interface. Currently, this MA supports information about their national research backbone [14]. Sincedifferent types of measurement data, like utilization, L2 status, the start of JRA1 in September 2004 it started to be adapted toand one-way delay data. The functionalities provided by the provide perfSONAR functionalities. Currently it shows theservice are accessed through Web Services using specific topology of the network using a set of hierarchical topologymessage types. The three main message types are: maps that has network nodes and links including statusMetadataKeyRequest, SetupDataRequest and information and metrics allowing also the visualization ofMeasurementArchiveStoreRequest. historical information.

The MetadataKeyRequest message is used to request whichdata is being stored at the RRD MA database, through thedefinition of their related metadata. The SetupDataRequestmessage is the one used to request a specific set of data, basedon the metadata, which describes the data wanted, and on aspecific time range. The MeasurementArchiveStoreRequestmessages are used to send data to be stored at the RRD MA.

Since this service operates directly with rrd files, it ispossible to generate these files directly without having to use ---I.the store request message, using the RRD MA only to make thedata available.

2) SQL-MA Service

An alternative type of measurement archive service was Figure 5. CNM Screenshot showing the Geant2 connections.developed which uses SQL type databases. The SQL-MA hasthe same functionalities as the RRD-MA and currently supports Nemo (Figure 6) is a system used to provide visualizationsthe same type of metrics. It can be configured to use any of the to the customers showing how they are connected to themainstream SQL database implementations. network and to provide them statistics about their QoS [14]. It

3) MP Services is composed of a set of components that provide the user thevisualization of the network in a geographic map with its

After the implementation of the MAs, the MP services have network metrics and entities. The system imports statisticalbeen developed. One group of services deals with command data from various network measurement systems and storesline tools (CL-MP) and the other one deals with SNMP them with resolutions from a few minutes to a week.(SNMP-MP). CL-MP can handle several measurement tools bytranslating the standard perfSONAR XML communicatingmessages to the specific tools parameters. Currently, it can .. Xhandle OWAMP [9], Ping, Traceroute and BWCTL6. TheSNMP-MP aims at providing standard perfSONAR messagesto access SNMP equipments in a multi-domain environment. Ithas implemented the SNMP Get functionality and a Key basedaccess mechanism where keys are configured into the service.

4) LSService

The Lookup Service (LS) [18] was implemented to handleall the registration and search for the services inside theenvironment. The Lookup Service allows the services toregister themselves and be visible to the other services andusers that want to search for them. It acts like a directoryservice, registering the services information in a database. Inthe specific case of perfSONAR, it was chosen an XML Figure 6. Nemo Screenshot.database to store that data, in order to facilitate the insertion ofdata that is already in XML format. This database is composed The VisualperfSONAR (Figure 7) is a web applicationof a collection of data and metadata's that describe each developed by CARNet8 that provides network utilizationregistered service to the clients. statistics of all the interfaces found through the parsing of a

given traceroute command output. The data are retrieved by5) Visualization tools querying the measurement archives deployed for each detected

Beyond ~ .thaviaiiyo.hs nrsrcuesrie interface and it shows the results using a geographic map usingthere are also many initiatives for the development Of network Gol assriemonitoring visualization and detection tools like the Customer PerfSONARUI (Figure 8) provides information of networkNetwork Management (CNM), Nemo, VisualperfSONAR and utilization and capacity as well as details of each interface andperfSONARUI [14]. it is also possible to query historical data. It started to be

5http://oss.oetiker.ch/rrdtool/ 7Deutsche Forschungsnetz. hftp://www.dfn.de/6 http://e2epi.internet2.edu/bwctl/ 8 Croatian Academic and Research Network. http://www.carnet.hr/

31

developed at the end of 2005 by the need of an open source, The goals of this project were the study of Ipe's backboneeasy to use and powerful stand alone client, capable of network traffic flows in order to identify traffic problems,querying the available perfSONAR services [14]. anomalies and find bottlenecks.

Ipe's backbone at the time of the project had 27 Points ofPresence (PoPs) which connects client institutions in every

Canada Brazilian state. The E2E Performance WG task was to deploy a-.,.... lDanada ....................... .... passive environment to collected Netflow information

measured on the backbone routers interfaces. Flows wereUnited E~~~~~~~p~~ ~~ exportedl to a central collector located in Rio de Janeiro state

A_l.ti _ |4 _ irAqtl (PoP-RJ). fheflow-tools package" was used in order toAextjo.Ea LiW wEfJy l~process the flow data and generate the reports desired by

_ n0s, NWr bi .T./.= RNP's Network Operation Center (NOC). Also, the workingE,, hv group developed automated solutions to respond to some

specific demands and studied the integration of visualizationFigure 7. VisualperfSONAR Screenshot. tools to be used by the network administrators. Some examples

of the demands were:REFREF

. Traffic matrices PoP x PoP;

perfS9NAR * Traffic matrices Institution x Institution;* Analysis per port and protocol type

* Denial of Service (DoS) attacks prevention;* Analysis of traffic exchange between RNP and

_ 7 ~~290,3S.a43;3 sqQ9GRSF26s; MH^^99y Nf ,61 NIb t5qA, stpg

20 13 46 NesOalm 205 Mt MRedClara networks;RNP 13 J942iM HU3s MIS3 W,',EOIPPs-iRXlit9 ---jM 3,i4 rj .i.e. ,N3F

HQstn3e 0, 1§2 0 -I --

lS6tp}r:--HOe R70A134Ms!00 S E#ff#,zYS .8gX.X/* Identification of the most active clients, autonomousp3a-clonhSo3B000 E l= .tI9tsystems and PoPs.

To perform the collection, the routers were configured toexport the collected flow data every 5 minutes to the mainj\ (<1 D 11 collector. Packet sampling is used at a 1/1000 ratio in order toreduce router and monitoring traffic loads. With this approach,

_9g 1gg1=.--. ..X.IL2 RNP was able to identify problems and find solutions related tothe traffic flowing through its backbone.

Figure 8. PerfsonarUl Main Window Screenshot. B. RNP's E2E Performance WG NMSOA Developments

Based on the NMSOA initiatives presented before, RNP'sIII. CASE STUDIES E2E Performance WG started the development of important

Over the recent years, the RNP's E2E performance WG components to cope with services publication and discoveryhas been interested in networkmasurem(LS), measurement data storage (MA), measurement execution

has been interested in network measurements to identify end- (CL-MP) and performance data visualization. The componentsto-end problems and analyze, in particular, the Ipe backbone'. interoperation occurs as depicted in Figure 9.The efforts involved activities to evaluate and test differentnetwork measurement solutions like Cisco Netflowl0, 1) Command Line Measurement Point (CL-MP)BWCTL, OWAMP, Cacti", amongst others. The CL-MP is a service developed in a joint effort between

This section contains information about some experiences Internet2 and RNP to make a measurement point that abstractsgathered by RNP's E2E performance WG concerning different established command-line network measurement toolsdevelopment deployment and tests of network measurement under a standard interface. This service is based on the sameenvironments in the contexts of ELA Project Task 2.4 12 and principles developed for the test module on piPEs-BR, but in

RNP's Ipe Network. This section also presents the developed this case it interacts with other services using NMWG webvisualization environments that helped on the network services messages.performance analysis. Its main objective is to enable users to run tests across

domains with a common language to reach the tools and theirA. RNP 's E2EPerformance WGpast measurement activies results. It basically wraps different network command-line

The xpermentofpassve masurmens cocernthe tools using a common schema language, having a well defined*~~ ~ ~ ~ ~~o sevie on RN' iu 1e structure, a fact that makes it easy to add new tools. The CL-imlmntto an deploymet T^TMP is able to do on-demand tests and to schedule tests in aNetwrk Bckboe. Tey ae reatedto "pe Fows rojet". cron-like manner, storing the corresponding results in the SQL-

MA.hftp://www.rnp.br/enlbackbone/ ~~CL-MP iS based on the basic principle that any tool running

10http://www.cisco.com/ on a text prompt command may be represented by two sets: a

"hftp://cacti.net/ __________________12 http://www.eu-eela.org/eela_wp2_network.php 13 hftp://www.splintered.net/sw/flow-tools/

32

set of parameters and a set of results. Parameters are input presented in the XML query messages. For example, ininformation that describe each specific test and are normally deregister action for a service with 200 data elements the LS-represented on metadata blocks of a perfSONAR message. XML spent 108 seconds of delay on the other hand the LS-Results are answers to tests and are represented in data UDDI spent only 2 seconds.elements which are grouped into a data element. It is alsoimportant to remember that every single test sample should beassociated to a timestamp; otherwise, results would not allow L S Ucorrelation and interpretation over time based on larger data register_service Registersets. Querydate servite

UDDI LS Inter ace Remove Client orfind_service hange serviceremnove_seriice Keepalive-

Lookup Service - LS * Internet Computer _ - choLS - Publi h and Discovery of Web

9 UDD type network Eye-ICEServices Module. It is uAsed to locate UD yentokEe-C

aGcess information of the Web * UDDI access UDDIServices of the measurements ibrary * OSGI

infrastructure. * Web services * Web Services

MA - Data storage Module,Collected data and other data

available by piPEs-BRiGFD are Figure 10. LS-UDDI Structure and Funcionalities.stored in this.

CLMP Measurement Point3) Measurement Archive (MA)

C(LMP - Mieasuremenet Point. lXlExecutes the measurement tools o u e ewith the desired parameters. Gets In order to understand and explain different events andthe results on demand and stores behaviorsperiodic measurements in the along tme, the data obtained from network

Measuirement ArchiveX performance measurements must be stored. In this way, dataLegenda collected from devices and MPs can be correlated for a better

Plugin OSGI explanation about specific events and behaviors on networks.| Query and Register ofservices through web services * Measurement * CommandLirie At RNP, the initial experiments have been done by the use4- Text execution command Archive - MA Measurement Point -CLMP

Query and Storage of data * MySQL * Measurementtools ~ of a MySQL database to store the data. The second experienceRRD e OWAMP was achieved through the perfSONAR SQL-MA service as anWeb services * BWCTL (IPERF)* Traceroute interface to the database. The measurement data has being* Ping

o Web services stored in very high detail, being used mainly to allow tests andFigure 9. RNP's NMSOA components interoperation. analysis on the experimental measurement environments. In

this way, all the single existent events on each sample wereThe service allows measurements for the supported tools to stored.

be requested via the perfSONAR Web Services interface, In the case of the storage of the data obtained from the CL-making it easy to start measurements at several endpoints even MPs, it has been necessary to expand the metrics andfrom a foreign administrative domain, as long as they functionalities supported by the MAs. Metrics for OWAMP,implement this same set of services. BWCTL and ping are now supported in RNP's SQL-MA

2) Lookup service infrastructure version. Also, a new functionality has been implemented toallow the dynamic update of the keys used by the SQL-MA:

Since the initial efforts of the RNP's E2E Performance WG MeasurementArchiveStoreKeyRequest.for Publish and Discovery Services were using the UDDItechnology, it was proposed an adaptation of the service to 4) Visualization Environmentsfollow perfSONAR's definitions, thus creating a new type of In order to integrate the access to the measurement toolsLookup service: the LS-UDDI. Hence, both services (whether deployed at RNP and validate the use of the networkLS-XML or LS-UDDI) can interoperate to make possible that measurement services, it has been developed a networkpiPEs-BR/GFD also interoperates with perfSONAR. Soon the monitoring access environment called ICE [11]. ThisLS-UDDI will implement all LS functionalities according to environment allows the users to query in an integrated way theperfSONAR's definition and use the standardized XML supported NMSOAs modules and provides flexible ways to domessages exchanges. Figure 10 shows the service structure and the development of new features. This flexibility is achievedfunctionalities. through the use of a framework based on OSGi technology

With the use of UDDI, it is possible to aggregate in the [12], called FLAVOR [13], that provides ways to do theLookup Service some functionalities and advantages that the development and deployment of standard software componentstechnology offers. UDDI has been proposed for Web in the ICE environment as depicted in Figure 11. It shows theService's publication and discovery. Therefore, it has integrated functionalities and the bundles to access themechanisms such as: the creation of indexes for the services, NMSOAs modules (AMP Access, CL-MP, Ping and Lookupservice~~caeoiain reaiosi bewe evcs t. 'Service), a local monitoring data source and also the FLAVOR

serv1ce categor1zahon, relahonsnlp netweenserv1ces,etc.

Another important aspect iS that the Lookiup service basedl on OSGi bundles functionalitiesandrelationships.UDDI has a better performance than its XML counterpart Figure 12 exhibits ICE's main window. Every ICE'sduring the searches and updates, since it relies on the use of a functionality starts in a different window. In this example, the

relational database.access to the CL-MP measurement data is provided throughSome analysis tests has been done with the LS-UDDI and some tabs that provide interfaces to do the on-demand tests

LS-XML. It was verified a huge performance loss in the LS- provided by this kind of MP. This interface shows all the MPsXML queries due to the amount of metadata information available at RNP's Ipe network that it is able to query for

33

specific network measurement metric. Within this, the user can ufreely choose one of them and query for data. After the request, 'the user will get the data inside a table and a chart view. P

Available Net orkMonitoring SrVices 0

V;suahzatonOat ~~~Network Monitering> tw:i Service

Vi~~~NV aN ~~~~~~Other Services

. ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 0 58 XOD mm 30 cope l260i1\010;0

Local DataAs~~~~~~~~~~X Source

W integratedFunctionalities

7 Flaler Figure 14. ICE built-in Ping Interface.

Figure 11. ICE Architecture. The use of OSGi components inside the ICE environmentprovided easier development of new services through the reuseof the capabilities already implemented and makes the tools

.1[

customizable in the way that users can choose whatcomponents they want to install and use. The developed

iwrnper fX = stbundles can be assembled and composed into their applicationsusing the benefits of the software components to dynamicallydeploy components and, consequently, dynamically

..n fD n i iconfiguring their access environments._____ _ ____ In order to get more flexibility in the development of new

functionalities, some FLAVOR bundles were developed andmade available to the research community interested in RNP's

__ I I~~~~~~~~~~~~penetwork measurement data. One of the research groups wasRNP's Video Management Working Group (VMWG) . In

_= their case, two of the developed bundles were the RRD-MAaccess bundle and the RRD-MA visualization bundle, thatmakes the access and provides visualization for the data storedon the RRD-MA perfSONAR service, respectively. Thesebundles can be used by other applications that use Java and

Figure 12. ICE Main Window Screenshot. have the support for OSGi components and use FLAVORwithout any extra development effort.

The screenshot presented on Figure 13 shows the result of aquery to the traceroute data from MP-SC (located in Santa C. Deployments on Ipe NetworkCatarina State) to the RNP's website (located in Rio de Janeiro RNP's Ipe Network is composed of 27 points of presencestate) producing the results showed on the window's left side in (PoPs) generally located in Brazilian state capitals, connecteda table and a chart with 3 lines indicating that it was made through 4 Mpbs up to 10 Gbps links. RNP's network has alsobased on three distinct traceroute results. three international connections, a 400 Mbps and a 1 Gbps links

......for Internet commodity traffic and a 155 Mbps connection toRedClara15, the Advanced Latin American Network. Through

....................................................... .................................. }11-gEadpoirtWse 1 0S7 this connection with RedClara, RNP is connected to otherea w u 18014..8 ,s advanced networks such as Geant2 and Internet2.

|Packlet Size(btgiohal)i 2um804~ tO96lFirt,tTTL(bpt_ona____ Ipe Network measurement points were deployed on theglMnxTTLQuptionaI) 1 10st J'tEnetwork PoPs which are presented in Figure 15. These||Wait Time (ojptibhnal):H80 -,i/t

aen 70l f 1Xf I measurement points are deployed with perfSONAR's CL-MP|lPbrtfoVt(0&16): | E i3geD2/$XIservices. Figure 15 also shows that in this network the CL-MPToS (optibr1aI). 1 > 4EI: /A i Iservicesare not fully deployed at every PoP, and some which

Number zof Queries (olptiona: 20 Xare part of the main rings, are still desired. At the majority ofthe Ipe network PoPs were actually installed two measurement

823qunc4unnr 67 | points machines in order to improve its performance1 ~~~~measurements, since it iS recommended to run delay tests and

bandwidth tests on distinct devices.Figure 13. CLMP Traceroute User Interface.

Figure 16 presents some results achieved on the IpeAnother ICE' functionality is a built-in ping that can be network. These sample graphs present information about delay

used to do local tests to any remote point in the network. Figure on its backbone links. Information was gathered using the14 shows the screen of the ping results from the local machineto RNP's website. It gives the result in a chart and in a text areabelow that. 14 http://gtgv.larc.USp.br!

15 hftp://www.redclara.net/

34

CactiSONAR plugin, which integrates perf'SONAR services to deployed a monitoring framework based on the perf'SONARthe Cacti network management environment, project.

CL-MP services were deployed along Clara PoPs and it~will be used to inform both users and network administratorsabout delays and achievable bandwidths of each PoP.CactiSONAR was also chosen to enable some visualization

IR ~~~~~from this environment, once it was integrated to anevrnent that Clara network administrator were used to.

Also, once the plugin is focused on enabling analysis of long.........................t.erm historical information, it is possible to observe tendencies,

even tough daa are sored on iRfies taby naturesmaize data. Some examples of measured metrics on

redClara backbone may be seen in REFFigure 17 that showsthe Bandwidth measurements results in RedClara betweenBrazil and Mexico showing te maximum, average and

A. ~~~~~~~~~~~~~~~minimumresults of each sent and received test packages.0Acthiv With GPS

*ActiewihiotitCGPS4Ated'abift~ Dsirbl

Figure 15. Ip~Network Measurement Points Deployment.

[. ..A

MUMtUad 4A0. co. 6i9 m$ M" 44.$& P1 Agq. LO44 * Mai. &2 i44 1Ka smEa i h CU a( H tM4i0i A44 7i98 M "4a 25tM M

5 ~~~~~~~~~~~3O1 A ~~~~~~~~~~~~~~~~~~~~~~~ay 5mAvr"

A~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~i

~~AW~~~D&La~~~~:~~~ Figure 17. Achievable Bandwidth Results onredClaraNetwork.~~~~~~~~~~~~~~~~~~~~~~~~~~.........

Q0 a 19 miosal A00 Ohe CLMPserics wredepoyd i odertosupor

~~LMt~: instittions: NAM in exico,wURJDin Bazil, adC[BMAFigure 16. One-WayDelay Results on the Ip~~~ Network. in Spain. Fgure1A ev18ashwstheuMPsdeploedC fora Nthisoroet

Prelminry round tervipcela resutred proeseintored in Fiurep19.D. EEL ' prject masuremnt iniiativeLA' prjcFTs .4i4dead1arsXRdRS

The B-Ifrastruture shred beween EuopedandLatinandApM ewrsinovn eio Braz1il, MandrAmerica (BELA) project was launchedths intialphastoEEprovideet isineestedi

interoperation among Latin Americag thirgrdandplEnablingnvlGridshe forowe-

EELA has access to networkingtutins:servicesxicavailablerazlinnLatinA

NRENs). The infrastructure ofSpin.EELAe18 shwistecomposedyedofr multiplectautonomous sysemsWith seveal hesuterognteousNtwrPelmnr resuceundFiurr1.iEL' Proecty Measult rempentPontsdiDeploy ent.

D.EELA 's network opeasrationtisitobtviosl depedenWon ts LSSOSMLEdNEnTwor wnfastavryuimprtsantdissetowee cuonsierdInti meseetand stausiataacrssMultpledomins inada

wAy,erNPcasanE EELA andec persNA prjcects membier wolwd ai.Atog mtrouinoigEdMAininteroperation among Latexst,naAprblemaariss whenldalingrwitsmultipl

Science, Europe (EGEE). CLARASP~~~~~~~~~~~~~~~3

administrative domains. Today, it is still difficult to solve of changing the database structure. For example, aproblems across them. Efforts to solve these problems involve service that would like to be registered with anmany academic researchers from different networks. additional characteristic, only need to add it to the dataStandardization is a way when it is evolving to a more complex set to be stored. The choice service to be used dependsenvironment of cooperative work to manage end-to-end paths. on the context where it will be applied.Some lessons related to multi-domain measurement activitiesare: * MA Deployment: During the experiments on deploying

and using SQL-MA, it was noticed that retrieving fullraw data from tests is unfeasible. It may cause serviceunavailability and too long delays. All requests should

F#P&GNAP C P ii C MAT ~Fi~dIRIS~ return to the user summarized information for a largerange of time from this data. Possible solutions thatmight be considered are: to limit the amount of datathat can be retrieved on raw perspective; to limit to amaximum time frame that could be retrieved on a

12-01 soIJ

in0 4 o0 0 a011aaI single request; or to create specific summary views on11f11 Lt&p M:o uR.r 'A n 2M A'" 'W Wso this service. Another solution would be to store thisam Ml* uti 4;1UI9 ifti' Z 3W test information on different technologies, like

RRDTool files, that are intended to store time seriesdata. The only problem is that for some kinds of data,Sr C [FlOdIF1S RRDTool is not the most recommended technologyonce it by nature summarizes data. The function that itnormally uses or summarization is Average, what isnot the most recommended for all types of data. Oneexample of data that is not well represented on

@. averages is delay, once outliers really change the0;" OdA.y WC cut- 2g .i2.09 MiZ1 Dw&Mt 44~laperception of delay to the end-user. Anyway, when0 M tkl,3 ZOO "s WD !2tm WZ Xg gAm -X; Hr i,it '. deploying MA services, these considerations should be

Figure 19. Round Trip Delay Results on EELA Network. taken into account and, depending on the main uses forthis stored information, an appropriate method shouldbe selected. Auto-summarizable storage mechanisms,

* Visualization Tools: Even using precise measuring like RRDTool, are more advisable on historical timetools, an environment without easy-to-use visualization series and for tendency analysis. Other technologies,tools which permit user to interact, may have no value. such as SQL databases, could be selected as a solutionUsers need flexibility to define exactly their needs. when it is needed to have detailed perspective on testsCharacteristics that attend this demand will enable and not for really long historical time series.deployment of such an environment on a productionscenario.

V. CONCLUDING REMARKS AND FUTURE DIRECTIONS* Clock Precision: The usability of data, mainly when This paper presented the motivation for network monitoring

measuring one-way delay, is highly dependent of service oriented architectures (NMSOA), and the activities ofprecise clocks between source and destination. A high RNP's End to end performance WG, on developing anddifference may give a wrong perspective of real delay deploying such environments.on network. It also may invalidate measurement,mainly when measuring networks with low one-way The E2Ep WG started developing a "Brazilian" version ofdelays, since we may end up with negative values. To the piPEs architecture proposed by Internet2 and which wassolve this sort of problems the recommended solution dubbed, piPEs-BR. With the definition of the Generalis to use clock synchronization mechanisms based on Framework Design (GFD) by Internet2 and Geant2 JRA1, itGPS or CDMA (indirect GPS) systems. It is was decided to adapt piPEs-BR to be compatible with GFD,recommended to use GPS receivers whenever possible. and the new architecture was called piPEs-BR/GFD. In this

Md is, process the WG partnered with Internet2, Geant2, and the* MP deployment: Another recommendation i, University of Delaware for developing perfSONAR.

whenever possible, to have separate MPs to measuredelay and achievable bandwidth. Bandwidth The WG has been interested in network measurements tomeasurements normally require a high use of CPU identify end-to-end problems and analyze, in particular, the Ipeclocks. This factor may interfere on the precision of backbone. The efforts involved activities to evaluate and testone-delay measurements. different network measurement solutions like Cisco NetfIow,

BWCTL, OWAMP, Cacti, amongst others.* Lookup Services: For the publishing and discovery

services, it was verified that using the XML based LS Some experiences gathered by RNP's E2E performanceimplies in a loss of performance. This happens mainly Working Groups concerning development, deployment andwhen the amount of data to be stored or to be queried tests of network measurement environments in the contexts ofis big, as the complexity of data manipulation increases EELA Project Task 2.4 nd RNP's Ipe Network are presented.in the same way. The XML database is not as good as On the development side, the WG was responsible fora relational database since there are no indexes or other developing the Command Line Measurement Point (CLMP),mechanisms to improve the queries. On the other side, developed an alternative lookup service based on UDDI,the XMAL database is more flexible, as it is possible tomodify the data structure to be stored without the need

36

extended the information stored on the MAs, and developed The authors would like to thank The Brazilian Nationaltwo visualization environments. Education and Research Network (RNP), The Bahia's State

As far as deployment is concerned, we presented the Research Foundation (FAPESB), and The Brazilian Nationalexperience on installing MP and the other basic services on Council for Scientific and Technological DevelopmentRNP's Ipe network and on redCLARA in support of EELA's (CNPq) for all their support during the development of thisProject Task 2.4. work.

The current deployment phase is almost completed, and we REFERENCESare working with the users to test the environment and, in [1] J.W. Boote, E. L. Boyd, J. Durand, A. Hanemann, et al., "Towardsparticular, its usefulness to them. multi-domain monitoring for the european research networks,"

TERENA Networking Conference, 2005.Managing end-to-end is becoming more complex everyday.. [2] L. Sampaio, and J. A. S. Monteiro, "SFM3: A Service-based Flow

Just measuring will not be enough to explain network behavior Traffic Measurement Management Model for IP Networks." In Proc. ofanymore. Managers, in future, will need to associate events on The 2nd Workshop on End-to-End Monitoring Techniques and Servicesnetwork with measurement results to be able to explain why (E2EMON), San Diego, CA, October 2004.such a metric has a strange behavior in such period of time. [3] N. Brownlee. "Traffic Flow Measurement: Experiments with

NeTraMet", RFC 2123, March 1997.Future developments include a Resource Protector (RP) [4] I. C. Legrand, H. B. Newman, R. Voicu, C. Cirstoiu, C. Grigoras, M.

service, a Topology service (TopS) and the improvement of Toarta, and C. Dobre, "Monalisa: An agent based, dynamic servicevisualization and possibly advisory tools. system to monitor, control and optimize grid based applications,"

Computing in High Energy and Nuclear Physics (CHEP) Conference,A Resource Protector would be queried every time a test is 2004.

scheduled in order to check whether or not there are enough [5] J. W. Boote, E. Boyd, M. Campanella et al., "GEANT2 Generalnetwork resources to accommodate such tests. This decision Monitoring Framework Design," JRAI's Deliverable DJ. 1.2.1, 2005.must be based on appropriated policies so that the total amount [6] A. Hanemann, J. W. Boote, E. Boyd, J. Durand, L. Kudarimoti, R.of test traffic would never go over a given threshold (e.g., Lapacz, M. Swany, S. Trocha, and J. Zurawski. (2005). Perfsonar: Afraction of links bandwidths). service oriented architecture for multi-domain network monitoring. InProceedings of the Third International Conference on Service Oriented

It is expected that the Topology Service will dynamically Computing (ICSOC 2005). ACM Sigsoft and Sigweb.handle topology information to other components of the [7] F. Curbera, M. Duftler, R. Khalaf, W. Nagy, N. Mukhi, and S.framework. It has to keep a topology database constantly Weerawarana. "Unraveling the Web Services: an Introduction to SOAP,updated and anasbe eenpin many pointsof views.A posstantle WSDL and UDDI", IEEE Internet Computing, vol 6, pp. 86-93, 2002.

updated a ci [8] B. Lowekamp, B. Tierney, L. Cottrel, R. Hughesjones, T. Kielemann,solution is to base it on OSI's three lower layers: physical, link and M. Swany. "A hierarchy of network performance characteristics forand network layers. Another approach for this problem is to grid applications and services". Global Grid Forum, 2004.work on organization information's level and see how they are [9] B. Teitelbaum, A. Karp, J. W. Boote, and M. Zekauskas, RFC 4656: Aconnected. One-way Active Measurement Protocol (OWAMP), 2006.

[10] L. Sampaio, I. Koga, H. Monteiro, I. Koga, G. Rhoden, F. Vetter, G.This topology service needs to work on intra-domain and Nunes, E. Melo, and J. A. S. Monteiro. "piPEs-br: Uma arquitetura para

inter-domain network structures. Intra-domain has to keep a medi,co de desempenho em redes IP". In XXIV Simp6sio Brasileirotrack of all information related to a specific domain. Dealing de Redes de Computadores, 2006.with inter-domain, it needs to exchange summarized [11] I. Koga, H. Monteiro, L. Sampaio, and J. A. S. Monteiro. "ICE: Ainformation among domains, flexible network monitoring access environment". In Proc of The 25th

Brazilian Symposium on Computer Networks and Distributed Systems,Internet216 is trying to solve this issue based on layer three Belem, PA, 2007.

using their local databases. Geant defined a more complex [12] D. Marples, and P. Kriens, "The Open Services Gateway Initiative: Ansolution which is known as Common Network Information Introductory Overview". IEEE Communications Magazine, vol 39, pp.Service (cNIS) [16] and it is going to involve three lower 110-114,2001.layers. RNP's WG intends to have a more abstract view of this [13] I. Koga, L. Sampalo, and J. A. S. Monteiro. "FLAVOR: A dynamic and

based on organization hierarchy structure. open framework for the development of network measurement access

problem, oase on organzahon llerarcly structure.and visualization tools". In Proc of The 25th Brazilian Symposium on

Another goal is to provide information to a number of Computer Networks and Distributed Systems, Belem, PA, 2007.visualization tools, like the ones presented on this paper. Tools [14] A. Hanemann, V. Jeliazkov, 0. Kvittem, L. Marta, J. Metzger, and I.

will query it to display geographical maps, including nodes, Velimirovic. "Complementary visualization of perfsonar networkWill query it to di1splay geographical maps, includilng nodles, performance measurements." In Proceedings of the Internationallinks and interfaces information. Not just geographical maps Conference on Internet Surveillance and Protection (ICISP).can be done, but they can get network link maps, usage maps, IARIA/IEEE, 2006.weather maps, etc. It is important to know that, although all the [15] H. Zimmermann. "OSI Reference Model - The ISO Model ofgroups have different points of view to solve the problem, it is Architecture for Open Systems Interconnection". Communications,going to be defined a unique well-known standard. IEEE Transactions on, 1980.

[16] Deliverable DS3. 13.1: Common Network Information Service SchemaFinally, it is desirable to develop more intelligent advisory Specification. http://www.geant2.net/upload/pdf/GN2-07-045v4-DS3-

tools which can access measurement data, and infer network 131_common_Network_Information_Service_Schemastatus in order to advise non expert users with low level Specification.pdf.knowledge of network measurements, who just want to know [17] N. Jeliazkova, I. Lliev, and V. Jeliazkov. "UPerfsonarUI - a Standalone

.,.-............. ,. ' .. . ~Graphical User Interface for Querying perfSONAR Services,".whether he/she can safely run his/her applications. International Symposium on Modern Computing (JVA), 2006.

[18] J. Zurawski, J. Boote, E. Boyd, M. Glowiak, A. Hanemann, M. D.ACKNOWLEDGMENTS Swany, S. Trocha, Hierachically Federated Registration and Lookup

within the perfSONAR Framework, In Moving from Bits to BusinessValue: Proceedings of the 2007 Integrated Management Symposium,2007, IFIP/IEEE, Munchen, Germany, May, 2007.

16http:/llwww.internet2.edu

37