GRNOGWorkshop#1,Athens–May2017

Introduc?ontopmacct

PaoloLucentepmacct

whoami

PaoloLucenteGitHub:paololucenteLinkedIn:plucenteDiggingdataoutofnetworksworldwideforfunandprofitformorethan10years

libpcap

pmacctisopen-source,free,GPL’edsoOware

sFlow

BGP

maps

IGP

MySQL PgSQL SQLite

MongoDB BerkeleyDB

flat-files

RabbitMQ Kafka

memory tables

sFlow

tee

NetFlow IPFIX

NetFlow IPFIX

hPp://www.pmacct.net/

Streaming Telemetry

BMP GeoIP

pmacct:afewsimpleuse-cases

BMP

flat-files

tee

NetFlow IPFIX

sFlow

Kafka

IPFIX libpcap

pmacct:aslightlymorecomplexuse-case

BGP

flat-files

tee

NetFlow IPFIX

Kafka

MySQL

aggregation method #1

aggregation method #2

nfacctd

Theuse-caseformessagebrokers

pmacct-to-elas?csearch0.3.0

Creditsto:PierCarloChiodi,hPps://github.com/pierky/pmacct-to-elas?csearch

Usecasesbyindustry

Keypmacctnon-technicalfacts

§  10+yearsoldproject§  Can’tspellthenameaOertheseconddrink§  Free,open-source,independent§  Underac?vedevelopment§  Innova?onbeingintroduced§ Welldeployedaround,alsolargeSPs§  Aimstobethetrafficaccoun?ngtoolclosertotheSPcommunityneeds

Sometechnicalfacts(1/2)

§  Pluggablearchitecture:•  Caneasilyaddsupportfornewdatasourcesandbackends

§  Correla?onofdatasources:•  Na?velysupporteddatasources(ie.BGP,BMP,IGP,StreamingTelemetry)

•  Externaldatasourcesviatagsandlabels§  Pervasivedata-reduc?ontechniques,ie.:•  Dataaggrega?on•  Filtering•  Sampling

Sometechnicalfacts(2/2)

§  Buildmul?pleviewsoutoftheverysamecollectednetworktrafficdataset,ie.:•  Unaggregatedtoflat-filesforsecurityandforensics;ortomessagebrokers(RabbitMQ,Kaga)forBigData

•  Aggregatedas[<ingressrouter>,<ingressinterface>,<BGPnext-hop>,<peerdes?na?onASN>]andsenttoaSQLDBtobuildaninternaltrafficmatrixforcapacityplanningpurposes

§  Enableanaly?csagainstthecollecteddatasources(ie.BGP,BMP,StreamingTelemetry):•  Streamreal-?me•  Dumpatregular?meintervals(possiblestatecompression)

Furtherinforma?onaboutpmacct

§  hPps://github.com/pmacct/pmacct•  OfficialGitHubrepository,wherestarandwatchusJ

§  hPp://www.pmacct.net/lucente_pmacct_uknof14.pdf•  MoreaboutcouplingtelemetryandBGP

§  hPp://ripe61.ripe.net/presenta?ons/156-ripe61-bcp-planning-and-te.pdf•  Moreabouttrafficmatrices,capacityplanning&TE

§  hPps://github.com/pmacct/pmacct/wiki/•  Wiki:docs,implementa?onnotes,ecosystem,etc.

ItjustseemedGreecewastherightplacewheretoshareoctopusstories..

GRNOGWorkshop#1,Athens–May2017

Thanks!Ques?ons?

PaoloLucente<paolo@pmacct.net>

hPp://www.pmacct.net/|hPps://github.com/pmacct/pmacct

Introduc?ontopmacct

GRNOGWorkshop#1,Athens–May2017

GRNOGWorkshop#1,Athens–May2017

LatestsonBGPmonitoring

PaoloLucenteNTTCommunica?ons|pmacct

BGP

§  Protocoltoadver?seReachabilityInforma?on:•  TheNetworkLayerpartofthestory,whiles?lldominant,is“old”:BGPisusedastransportforavarietyofdifferentinfo

§  Goodatpolicycontrol:•  Althoughqualityfactors,ie.latency,jiPerandpacketloss,increasinglypopularforcontentdeliveryinplaceofthetradi?onalBGPselec?onalgorithm

§  Goodatinforma?onhiding:•  But,thenagain,thisistherecipeforscalingtothecurrentInternetsizeandbeyond

EarlyaPemptsatgainingvisibility

Creditsto:E.Jasinska(Nezlix),P.Lucente(pmacct)@NANOG61

§  Circa2013§  Goal:seeallpathsinaBGPmul?-pathscenario,avoidingscreenscraping

BMP§  BGPMonitoringProtocol§  RFC7854:•  firstdraOin2008,sparseworkun?l2012;•  stallbetween2012and2015;•  realtrac?onkicksin:10draOsbetween2015and2016;•  RFCawardinJun2016

§  Uncomplicatedprotocoldesign§  Greateffortbut..•  ..industryevolvedalltheseyears•  increasedhungerfordata

ADevOpsguyduringlunchbreak

Tradi?onalBGPmonitoring

Peers

VantagePoint(VP)

Best

Path

BGPistheworld’sbestinforma?onhidingprotocol<tm>.Itonlygivesme

therouter’sbestpath

PeeringRouter

P0

P1

P2

P3

P4

P3

AndVPdoesnotevenknowwhyP3waschosen

Creditsto:R.Bush(IIJ)@BMPBoF,RIPE74

BGPmonitoringwithBMP(1/2)

Creditsto:R.Bush(IIJ)@BMPBoF,RIPE74

All

Paths

VantagePoint

WithBMP,Ilearnallthepathsthepeeringrouterheard

PeeringRouter

P0-4

Peers

P0

P1

P2

P3

P4

BGPmonitoringwithBMP(2/2)

BMP:problemstatement

§  TheBGPprotocolisoneoftheveryfewprotocolsrunningontheInternetthathasastandardized,cleanandseparatemonitoringplane,BMP(think,forexample,toDNS..)

§  S?llBMP,initscurrentshape,doescoveronlypre-andpost-policiesAdj-RIB-In;anoperatorwouldprobablyneed:•  ActualBGPpeering(s)forloc-RIB•  Worse-case,screenscrapingforAdj-RIB-Out

BGPPeer-BAdj-Rib-In(Pre)

Adj-Rib-In(Post)

Filters/Policy Filters/Policy

Adj-Rib-In(Post)

BGPPeer-AAdj-Rib-In(Pre)Wecanseethis

Andthis

Problemstatementvisualized

Creditsto:T.Evens(Cisco),S.Bayraktar(Cisco),P.Lucente(NTT)@GROWWG,IETF98

Sta?cBGPPeer-BAdj-Rib-In(Pre)

Adj-Rib-In(Post)

Filters/Policy Filters/Policy

Accepted Accepted

Adj-Rib-In(Post)

Adj-Rib-Out(Pre)

Adj-Rib-Out(Post)

Filters/Policy

BGPPeer-AAdj-Rib-In(Pre) ISIS

Wealsowanttoseethis

Andthis

Andthis

Loc-Rib

Creditsto:T.Evens(Cisco),S.Bayraktar(Cisco),P.Lucente(NTT)@GROWWG,IETF98

Proposal:extendBMPtoloc-RIBandAdj-RIB-Out(1/3)

Proposal:extendBMPtoloc-RIBandAdj-RIB-Out(2/3)

Proposal:extendBMPtoloc-RIBandAdj-RIB-Out(3/3)

Creditsto:T.Evens(Cisco),S.Bayraktar(Cisco),P.Lucente(NTT)@GROWWG,IETF98

§  Loc-RIB:§  Monitorroutesselectedanduserbytherouter:

o ECMPo Correla?onwithNetFlow/IPFIXo Next-hoppreserva?on

§  MonitorlocallyoriginatedandBGProuteswithoutrequiringpeering

§  Policyverifica?on§  Adj-RIB-Out:

§  Monitorroutesadver?sedtopeers§  Policyverifica?on

draO-evens-grow-bmp-{local-rib,adj-rib-out}use-cases

Thanks!Ques?ons?

PaoloLucenteNTTCommunica?ons|pmacct

LatestsonBGPmonitoring

GRNOGWorkshop#1,Athens–May2017