Flow-tools Tutorial

Mark Fullmer

maf@oar.net

Agenda

• Deployment motivation.• Network flows.• Cisco / Juniper implementation –

NetFlow.• Cisco / Juniper Configuration.• flow-tools programs overview and

examples from Abilene and Ohio-Gigapop.

Motivations

• Where your campus exchanges traffic with by IP address, IP Prefix, or ASN.

• What type and how much traffic (SMTP, WEB, File Sharing, etc).

• What services running on campus.• Department level traffic summaries.• Track network based viruses back to

hosts.

Motivations

• Track DoS attacks to the source(s), ie the 100 Solaris servers flooding whitehouse.gov that have been 0wn3d.

• Find busy hosts on campus.• How many destinations each campus

host exchanges traffic with.• Campus host counts by service, ie how

many active web servers.

Network Flows

• Packets or frames that have a common attribute.

• Creation and expiration policy – what conditions start and stop a flow.

• Counters – packets,bytes,time.

• Routing information – AS, network mask, interfaces.

Network Flows

• Unidirectional or bidirectional.

• Bidirectional flows can contain other information such as round trip time, TCP behavior.

• Application flows look past the headers to classify packets by their contents.

• Aggregated flows – flows of flows.

ssh sessionfaith.splintered:~% ssh eng4.oar.net w10:12PM up 476 days, 21:06, 7 users, load averages: 0.00, 0.00, 0.00USER TTY FROM LOGIN@ IDLE WHATmaf p0 dhcp9578217.colu 30Sep02 2days sshmaf p1 dhcp9578217.colu Wed12AM 12 -zsh (zsh)maf p2 dhcp9578217.colu Wed07AM 3:03 -zsh (zsh)maf p7 login.enss.net 22Mar02 199days -maf p8 login.enss.net 12Apr02 47days -maf p9 login.enss.net 24Jul02 75days -maf pa login.enss.net 25Jul02 47days -

ssh sessionfaith.splintered:/usr/home/maf# tcpdump -q -i sis0 -n ip and host eng4.oar.net tcpdump: listening on sis022:27:46.565664 24.95.78.217.1065 > 192.148.244.24.22: tcp 0 (DF)22:27:46.603058 192.148.244.24.22 > 24.95.78.217.1065: tcp 0 (DF)22:27:46.603523 24.95.78.217.1065 > 192.148.244.24.22: tcp 0 (DF)22:27:46.717333 192.148.244.24.22 > 24.95.78.217.1065: tcp 15 (DF) [tos 0x10] 22:27:46.717706 24.95.78.217.1065 > 192.148.244.24.22: tcp 20 (DF)22:27:46.830214 192.148.244.24.22 > 24.95.78.217.1065: tcp 276 (DF) [tos 0x10] 22:27:46.846743 24.95.78.217.1065 > 192.148.244.24.22: tcp 156 (DF)22:27:46.967105 192.148.244.24.22 > 24.95.78.217.1065: tcp 12 (DF) [tos 0x10] 22:27:46.967292 24.95.78.217.1065 > 192.148.244.24.22: tcp 20 (DF)22:27:47.062173 192.148.244.24.22 > 24.95.78.217.1065: tcp 60 (DF) [tos 0x10] 22:27:47.062239 192.148.244.24.22 > 24.95.78.217.1065: tcp 12 (DF) [tos 0x10] 22:27:47.062433 24.95.78.217.1065 > 192.148.244.24.22: tcp 0 (DF)22:27:47.062636 24.95.78.217.1065 > 192.148.244.24.22: tcp 140 (DF)22:27:47.196829 192.148.244.24.22 > 24.95.78.217.1065: tcp 140 (DF) [tos 0x10] 22:27:47.204546 24.95.78.217.1065 > 192.148.244.24.22: tcp 28 (DF)

28 packets, 728 byte + IP/TCP overhead.

Unidirectional Flow with Source/Destination IP Key

10.0.0.1 10.0.0.2

% ssh 10.0.0.2 w

reply

Active FlowsFlow Source IP Destination IP

1 10.0.0.1 10.0.0.22 10.0.0.2 10.0.0.1

Unidirectional Flow with Source/Destination IP Key

10.0.0.1 10.0.0.2

% telnet 10.0.0.2

login:

Active FlowsFlow Source IP Destination IP

1 10.0.0.1 10.0.0.22 10.0.0.2 10.0.0.1

% ping 10.0.0.2

ICMP echo reply

Unidirectional Flow with IP, Port,Protocol Key

10.0.0.1 10.0.0.2

% telnet 10.0.0.2

login:

Active FlowsFlow Source IP Destination IP prot srcPort dstPort

1 10.0.0.1 10.0.0.2 TCP 32000 23

2 10.0.0.2 10.0.0.1 TCP 23 32000

3 10.0.0.1 10.0.0.2 ICMP 0 04 10.0.0.2 10.0.0.1 ICMP 0 0

% ping 10.0.0.2

ICMP echo reply

Bidirectional Flow with IP, Port,Protocol Key

10.0.0.1 10.0.0.2

% telnet 10.0.0.2

login:

Active FlowsFlow Source IP Destination IP prot srcPort dstPort

1 10.0.0.1 10.0.0.2 TCP 32000 23

2 10.0.0.1 10.0.0.2 ICMP 0 0

% ping 10.0.0.2

ICMP echo reply

Application Flow

10.0.0.1 10.0.0.2

% netscape http://10.0.0.2/9090

Content-type:

Active FlowsFlow Source IP Destination IP Application

1 10.0.0.1 10.0.0.2 HTTP

Web server onPort 9090

Aggregated Flow

Flow Source IP Destination IP prot srcPort dstPort

1 10.0.0.1 10.0.0.2 TCP 32000 23

2 10.0.0.2 10.0.0.1 TCP 23 32000

3 10.0.0.1 10.0.0.2 ICMP 0 04 10.0.0.2 10.0.0.1 ICMP 0 0 Source/Destination IP Aggregate

Flow Source IP Destination IP

1 10.0.0.1 10.0.0.22 10.0.0.2 10.0.0.1

Main Active flow table

NetFlow data reduction

Look at 1 day of flows exports from krc4:

Flows 111182160Octets 2450050798277Packets 4057574675Flow size 48 bytesFlows/packet 30 (typical)Packet overhead 24 bytesOctets in a flow 1464 bytes

NetFlow data reduction

Look at 1 day of flows exports from krc4:

Packets/Flow 37 Octets/Flow 22036Octets/Packet 603Packets/Flow export 1095Octets/Flow export 661092Octets/Octets ina flow export 452

Packets/Flow distribution

Octets/Flow distribution

Flow Descriptors

• A Key with more elements will generate more flows.

• Greater number of flows leads to more post processing time to generate reports, more memory and CPU requirements for device generating flows.

• Depends on application. Traffic engineering vs. intrusion detection.

Flow Accounting

• Accounting information accumulated with flows.

• Packets, Bytes, Start Time, End Time.

• Network routing information – masks and autonomous system number.

Flow Collection

• Passive monitor.

• Router.

• Other existing network device.

Passive Monitor Collection

Workstation A Workstation B

Campus Flow probe connectedto switch port in“ traffic mirror” mode

Router Collection

Flow collectorstores exported flows from router.

LAN

LAN

LAN

LAN

Internet

Passive Monitor• Directly connected to a LAN segment

via a switch/router port in “mirror” mode, optical splitter, or repeated segment.

• Generate flows for all local LAN traffic.

• Must have an interface or monitor deployed on each LAN segment.

• Support for more detailed flows – bidirectional and application (external probe has more resources).

Router Collection

• Router will generate flows for traffic that traverses the router.

• Flows are not generated for local LAN traffic (not a sniffer).

• Limited to “simple” flow criteria (packet headers).

• Generally easier to deploy – no new equipment.

Cisco NetFlow• Unidirectional flows.• IPv4 unicast and multicast.• Aggregated (v8) and unaggregated

(v1,5,6,7).• Flows exported via UDP.• Supported on IOS and CatIOS

platforms.• Catalyst NetFlow is different

implementation.

Cisco NetFlow Versions

• 4 Unaggregated types (1,5,6,7).• 14 Aggregated types (8.x).• Each version has its own packet format.• Version 1 does not have sequence

numbers – no way to detect lost flows.• The “version” defines what type of data

is in the flow.• Some versions specific to Catalyst

platform.

Cisco NetFlow Versions

• v9 in development. More flexible packet format so new fields can be added without creating yet another version while still maintaining compact encoding.

NetFlow v1

• Key fields: Source/Destination IP, Source/Destination Port, IP Protocol, ToS, Input interface.

• Accounting: Packets, Octets, Start/End time, Output interface.

• Other: Bitwise OR of TCP flags.

• Historical – don’t use. No sequence #’s.

NetFlow v5

• Key fields: Source/Destination IP, Source/Destination Port, IP Protocol, ToS, Input interface.

• Accounting: Packets, Octets, Start/End time, Output interface.

• Other: Bitwise OR of TCP flags, Source/Destination AS and IP Mask.

• Packet format adds sequence numbers for detecting lost exports.

NetFlow v8

• Aggregated v5 flows.

• 3 Catalyst 65xx specific that correspond to the configurable flow mask.

• Much less data to post process, but lose fine granularity of v5 – no IP addresses.

NetFlow v8

• AS• Protocol/Port• Source Prefix• Destination Prefix• Prefix• Destination (Catalyst 65xx)• Source/Destination (Catalyst 65xx)• Full Flow (Catalyst 65xx)

NetFlow v8

• ToS/AS

• ToS/Protocol/Port

• ToS/Source Prefix

• ToS/Destination Prefix

• Tos/Source/Destination Prefix

• ToS/Prefix/Port

NetFlow Packet Format

• Common header among export versions.

• All but v1 have a sequence number.• Version specific data field where N

records of data type are exported.• N is determined by the size of the flow

definition. Packet size is kept under ~1480 bytes. No fragmentation on Ethernet. No PMTU detection.

NetFlow v5 Packet Example

NetFlow v5 header

v5 record

IP/UDP packet

v5 record

…

…

NetFlow v5 Packet (Header)

struct ftpdu_v5 { /* 24 byte header */ u_int16 version; /* 5 */ u_int16 count; /* The number of records in the PDU */ u_int32 sysUpTime; /* Current time in millisecs since router booted */ u_int32 unix_secs; /* Current seconds since 0000 UTC 1970 */ u_int32 unix_nsecs; /* Residual nanoseconds since 0000 UTC 1970 */ u_int32 flow_sequence;/* Seq counter of total flows seen */ u_int8 engine_type; /* Type of flow switching engine (RP,VIP,etc.) */ u_int8 engine_id; /* Slot number of the flow switching engine */ u_int16 reserved;

NetFlow v5 Packet (Records) /* 48 byte payload */ struct ftrec_v5 { u_int32 srcaddr; /* Source IP Address */ u_int32 dstaddr; /* Destination IP Address */ u_int32 nexthop; /* Next hop router's IP Address */ u_int16 input; /* Input interface index */ u_int16 output; /* Output interface index */ u_int32 dPkts; /* Packets sent in Duration */ u_int32 dOctets; /* Octets sent in Duration. */ u_int32 First; /* SysUptime at start of flow */ u_int32 Last; /* and of last packet of flow */ u_int16 srcport; /* TCP/UDP source port number or equivalent */ u_int16 dstport; /* TCP/UDP destination port number or equiv */ u_int8 pad; u_int8 tcp_flags; /* Cumulative OR of tcp flags */ u_int8 prot; /* IP protocol, e.g., 6=TCP, 17=UDP, ... */ u_int8 tos; /* IP Type-of-Service */ u_int16 src_as; /* originating AS of source address */ u_int16 dst_as; /* originating AS of destination address */ u_int8 src_mask; /* source address prefix mask bits */ u_int8 dst_mask; /* destination address prefix mask bits */ u_int16 drops; } records[FT_PDU_V5_MAXFLOWS];};

NetFlow v8 Packet Example(AS Aggregation)

NetFlow v8 header

v8 record

IP/UDP packet

v8 record

…

…

NetFlow v8 AS agg. Packetstruct ftpdu_v8_1 { /* 28 byte header */ u_int16 version; /* 8 */ u_int16 count; /* The number of records in the PDU */ u_int32 sysUpTime; /* Current time in millisecs since router booted */ u_int32 unix_secs; /* Current seconds since 0000 UTC 1970 */ u_int32 unix_nsecs; /* Residual nanoseconds since 0000 UTC 1970 */ u_int32 flow_sequence; /* Seq counter of total flows seen */ u_int8 engine_type; /* Type of flow switching engine (RP,VIP,etc.) */ u_int8 engine_id; /* Slot number of the flow switching engine */ u_int8 aggregation; /* Aggregation method being used */ u_int8 agg_version; /* Version of the aggregation export */ u_int32 reserved; /* 28 byte payload */ struct ftrec_v8_1 { u_int32 dFlows; /* Number of flows */ u_int32 dPkts; /* Packets sent in duration */ u_int32 dOctets; /* Octets sent in duration */ u_int32 First; /* SysUpTime at start of flow */ u_int32 Last; /* and of last packet of flow */ u_int16 src_as; /* originating AS of source address */ u_int16 dst_as; /* originating AS of destination address */ u_int16 input; /* input interface index */ u_int16 output; /* output interface index */ } records[FT_PDU_V8_1_MAXFLOWS];};

Cisco IOS Configuration

• Configured on each input interface.• Define the version.• Define the IP address of the collector

(where to send the flows).• Optionally enable aggregation tables.• Optionally configure flow timeout and

main (v5) flow table size.• Optionally configure sample rate.

Cisco IOS Configurationinterface FastEthernet0/0/0 ip address 10.0.0.1 255.255.255.0 no ip directed-broadcast ip route-cache flow

interface ATM1/0/0 no ip address no ip directed-broadcast ip route-cache flow

interface Loopback0 ip address 10.10.10.10 255.255.255.255 no ip directed-broadcast

ip flow-export version 5 origin-asip flow-export destination 10.0.0.10 5004ip flow-export source loopback 0

ip flow-aggregation cache prefix export destination 10.0.0.10 5555 enabled

Cisco IOS Configurationkrc4#sh ip flow export Flow export is enabled Exporting flows to 10.0.0.10 (5004) Exporting using source IP address 10.10.10.10 Version 5 flow records, origin-as Cache for prefix aggregation: Exporting flows to 10.0.0.10 (5555) Exporting using source IP address 10.10.10.10 3176848179 flows exported in 105898459 udp datagrams 0 flows failed due to lack of export packet 45 export packets were sent up to process level 0 export packets were punted to the RP 5 export packets were dropped due to no fib 31 export packets were dropped due to adjacency issues 0 export packets were dropped due to fragmentation failures 0 export packets were dropped due to encapsulation fixup failures 0 export packets were dropped enqueuing for the RP 0 export packets were dropped due to IPC rate limiting 0 export packets were dropped due to output drops

Cisco IOS Configurationkrc4#sho ip ca flIP packet size distribution (106519M total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .002 .405 .076 .017 .011 .010 .007 .005 .004 .005 .004 .004 .003 .002 .002

512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .002 .006 .024 .032 .368 .000 .000 .000 .000 .000 .000

IP Flow Switching Cache, 4456704 bytes 36418 active, 29118 inactive, 3141073565 added 3132256745 ager polls, 0 flow alloc failures Active flows timeout in 30 minutes Inactive flows timeout in 15 seconds last clearing of statistics neverProtocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)-------- Flows /Sec /Flow /Pkt /Sec /Flow /FlowTCP-Telnet 2951815 0.6 61 216 42.2 26.6 21.4TCP-FTP 24128311 5.6 71 748 402.3 15.0 26.3TCP-FTPD 2865416 0.6 916 843 611.6 34.7 19.8TCP-WWW 467748914 108.9 15 566 1675.8 4.9 21.6TCP-SMTP 46697428 10.8 14 370 159.6 4.0 20.1TCP-X 521071 0.1 203 608 24.7 24.5 24.2TCP-BGP 2835505 0.6 5 94 3.3 16.2 20.7

Cisco IOS Configurationkrc4#sho ip ca fl

TCP-other 1620253066 377.2 47 631 18001.6 27.3 23.4UDP-DNS 125622144 29.2 2 78 82.5 4.6 24.7UDP-NTP 67332976 15.6 1 76 22.0 2.7 23.4UDP-TFTP 37173 0.0 2 76 0.0 4.1 24.6UDP-Frag 68421 0.0 474 900 7.5 111.7 21.6UDP-other 493337764 114.8 17 479 1990.3 3.8 20.2ICMP 243659509 56.7 3 166 179.7 3.3 23.3IGMP 18601 0.0 96 35 0.4 941.4 8.1IPINIP 12246 0.0 69 52 0.1 548.4 15.2GRE 125763 0.0 235 156 6.9 50.3 21.1IP-other 75976755 17.6 2 78 45.4 3.9 22.8Total: 3176854246 739.6 33 619 24797.4 16.2 22.6 SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP PktsAT5/0/0.4 206.21.162.150 AT1/0/0.1 141.219.73.45 06 0E4B A029 507 AT4/0/0.10 132.235.174.9 AT1/0/0.1 137.99.166.126 06 04BE 074C 3 AT4/0/0.12 131.123.59.33 AT1/0/0.1 137.229.58.168 06 04BE 09BB 646 AT1/0/0.1 137.99.166.126 AT4/0/0.10 132.235.174.9 06 074C 04BE 3

Cisco CatIOS Configuration

set mls flow fullset mls nde version 7set mls nde 10.0.0.10 9110set mls agingtime 32

Cisco CatIOS Configurationswlap1> sh mlsTotal packets switched = 5116997156Total bytes switched = 2289120109999Total routes = 2283

IP statistics flows aging time = 32 secondsIP statistics flows fast aging time = 0 seconds, packet threshold = 0IP Current flow mask is Full flowNetflow Data Export version: 8Netflow Data Export disabledNetflow Data Export configured for port 9110 on host 10.0.0.10Total packets exported = 6545

IPX statistics flows aging time = 256 secondsIPX flow mask is Destination flowIPX max hop is 255

Module 15: Physical MAC-Address 00-04-9b-78-bb-fc

Cisco CatIOS Configurationswlap1> sh mls stat entry Last UsedDestination IP Source IP Prot DstPrt SrcPrt Stat-Pkts Stat-Bytes---------------- --------------- ----- ------ ------ ---------- --------- 64.219.177.137 206.21.217.6 TCP 2206 WWW 4 1113 207.254.193.44 198.30.37.194 TCP 56403 WWW 9 10567 198.30.37.19 66.183.100.164 TCP WWW 1604 4 731 192.88.193.144 199.218.4.130 UDP DNS 61449 1 71 192.88.193.144 199.218.5.131 TCP DNS 2157 5 235 199.218.4.3 208.249.92.145 TCP WWW 37519 6 506 12.42.50.51 198.30.37.19 TCP 11200 WWW 5 494 199.218.5.130 142.22.16.54 TCP WWW 54658 6 745 207.254.193.44 198.30.37.194 TCP 56391 WWW 40 50762 206.21.0.139 24.178.12.64 TCP 110 1085 8 484 217.10.162.50 199.218.5.130 TCP 46983 WWW 53 73775 65.224.146.242 198.30.37.19 TCP 2282 WWW 4 390 199.218.4.135 65.31.16.245 TCP WWW 1196 6 650 24.150.19.19 205.133.123.10 TCP 58043 WWW 4 176

Cisco Catalyst Native IOS Configuration

mls flow ip destination-sourcemls nde flow includemls nde src_address 10.0.0.9 version 7ip flow-export source Loopback0ip flow-export version 5 peer-asip flow-export destination 10.0.0.5 5555

Cisco Issues• Cat 65K Sup2/MSFC2 NetFlow

implementation does not fill in important fields like input/output interface. Resolved?

Bug Id : CSCdt21216

Headline Netflow records source / dst interface index missing Product cat6000 Model x6k-sup2 Component earl Duplicate of Severity 3 Status C Version Found 6.1(1) Fixed-in Version 6.2(1) 6.1(3) 6.3(1)PAN Release Notes On Sup2, the destination and source Interface indices would be reported as zero in netflow exports. This is a hardware limitation.

Juniper Configration

• Sample packets with firewall filter and forward to routing engine.

• Sampling rate is limited to 7000pps (addressed with future PIC).

• Fine for traffic engineering, but restrictive for DoS and intrusion detection.

• Juniper calls NetFlow cflowd.

Juniper Configration

firewall { filter all { term all { then { sample; accept; } } }}

forwarding-options { sampling { input { family inet { rate 100; } } output { cflowd 10.0.0.16{ port 2055; version 5; } } }}

Firewall filter Enable sampling / flows

Juniper Configration

interfaces { ge-0/3/0 { unit 0 { family inet { filter { input all; output all; } address 192.148.244.1/24; } } }

Apply firewall filter to each interface.

Flow-tools• Collection of programs to collect and

post process Cisco NetFlow compatible flows.

• Written in C, designed to be fast (scales to large installations).

• Includes library (ftlib) for custom applications.

• Installation with configure;make;make install on most platforms.

• Distributed design for large installations.

flow-capture

• Collect NetFlow export packets and store to disk.

• Built in compression.

• Manages disk space by expiring older flow files at configurable limits (dataset size or number of files).

• Pre-filtering and Pre-tagging.

flow-capture

• Instrumentation for flows/second, packets/second, and dropped packets.

• Server for TCP based flow-client.

• Privacy mask option for removing host bits from flows.

flow-capture – flows/second

flow-capture – packets/second

flow-capture – flow drops/second

flow-fanout

• Replicate NetFlow UDP streams from one source to many destinations.

• Destination may be a multicast address.

• Same instrumentation as flow-capture.

• Can translate NetFlow packet format on output.

• Privacy mask support.

flow-expire

• Expire (remove) old flow files based on dataset size or number of files.

• Same functionality built in to flow-capture.

• Used when managing disk space in a distributed environment.

Abilene Configuration

• Collect and process flows for Abilene routers.

• Use sampled NetFlow on Cisco GSR’s.

• Distribute flows to Asta and Arbor Networks.

• Nightly usage reports.

• Archive of raw anonymized flows.

Abilene Configuration

• Dsitribute post processed data to internet2.edu.

• Moving to sampled data from Juniper T640’ and new architecture.

Current Abilene Configuration

12 Abilene core routersConfigured with sampled NetFlow

NetFlow exports (UDP) Server runningflow-fanout @ IU

AstaArbor

Internet2.eduPost Processed NetFlow (ssh)

Ohio ITEC

Current Abilene Configuration

NetFlow exports

Ohio ITEC

4 servers runningflow-capture(data anonymized at collection) Server with 2 Terabyte

RAID5 Array & web serverfor nightlyreports.

flow-expire tomanage disk space

Pull compressed flow fileswith rsync from collectors.

New Abilene Configuration

POP router with local directlyconnected server runningflow-capture and flow-fanout.

NetFlow exports (UDP)

Server runningflow-fanout @ IU

AstaArbor

Internet2.eduPost Processed NetFlow (ssh)

Compressed files(rsync via TCP)

Ohio ITEC

New Abilene Configuration

11 AbilenePOPS

Server with 2 TerabyteRAID5 Array & web serverfor nightlyreports.

flow-expire tomanage disk spaceflow-report, flow-nfilter,flow-tag for reports.

Pull compressed flow fileswith rsync from collectors.

Ohio ITEC

Collector Placement and configuration

• NetFlow is UDP so the collector should ideally be directly connected to the router to minimize packet loss and IP spoofing risks.

• No flow control. Undersized collector will drop flows. Monitor netstat –s | grep buf and configure syslog so dropped flows will be logged.

flow-print

• Formatted output of flow files.eng1:% flow-print < ft-v05.2002-01-21.093345-0500 | head -15srcIP dstIP prot srcPort dstPort octets packets131.238.205.199 194.210.13.1 6 6346 40355 221 5 192.5.110.20 128.195.186.5 17 57040 33468 40 1 128.146.1.7 194.85.127.69 17 53 53 64 1 193.170.62.114 132.235.156.242 6 1453 1214 192 4 134.243.5.160 192.129.25.10 6 80 3360 654 7 132.235.156.242 193.170.62.114 6 1214 1453 160 4 130.206.43.51 130.101.99.107 6 3226 80 96 2 206.244.141.3 128.163.62.17 6 35593 80 739 10 206.244.141.3 128.163.62.17 6 35594 80 577 6 212.33.84.160 132.235.152.47 6 1447 1214 192 4 132.235.157.187 164.58.150.166 6 1214 56938 81 2 129.1.246.97 152.94.20.214 6 4541 6346 912 10 132.235.152.47 212.33.84.160 6 1214 1447 160 4 130.237.131.52 130.101.9.20 6 1246 80 902 15

flow-cat

• Concat many flow files or directories of files.

eng1:% lsft-v05.2002-01-21.160001-0500 ft-v05.2002-01-21.170001-0500ft-v05.2002-01-21.161501-0500 ft-v05.2002-01-21.171501-0500ft-v05.2002-01-21.163001-0500 ft-v05.2002-01-21.173001-0500ft-v05.2002-01-21.164501-0500 tmp-v05.2002-01-21.174501-0500

eng1:% flow-cat . | flow-print

srcIP dstIP prot srcPort dstPort octets packets138.26.220.46 192.5.110.20 17 62242 33456 40 1 143.105.55.23 18.123.66.15 17 41794 41794 40 1 129.15.134.66 164.107.69.33 6 1214 2222 4500 3 132.235.170.19 152.30.96.188 6 6346 1475 128 3

flow-merge

• Flow-merge is similar to flow-cat except it maintains relative ordering of flows when combining the files.

• Typically used when combining flows from multiple collectors.

flow-filter

• Filter flows based on port, protocol, ASN, IP address, ToS bits, TCP bits, and tags. (Historical, use flow-nfilter).

eng1% flow-cat . | flow-filter -P119 | flow-print | head -10

srcIP dstIP prot srcPort dstPort octets packets155.52.46.50 164.107.115.4 6 33225 119 114 2 128.223.220.29 129.137.4.135 6 52745 119 1438382 1022 155.52.46.50 164.107.115.4 6 33225 119 374 6 164.107.115.4 192.58.107.160 6 60141 119 5147961 8876 128.223.220.29 129.137.4.135 6 52745 119 1356325 965 128.223.220.29 129.137.4.135 6 52714 119 561016 398 130.207.244.18 129.22.8.64 6 36033 119 30194 121 155.52.46.50 164.107.115.4 6 33225 119 130 2 198.108.1.146 129.137.4.135 6 17800 119 210720652 216072

flow-nfilter• Filter flows based any defined fields

including derived operations such as pps, bbs, and duration.

• Configuration file based.• Supports AND and OR operations.• Filters and primitives are named for

ease of use.• Use patricia trees, hash tables, and

bucket lookups where possible for fast processing.

flow-nfilterfilter-primitive abilene-interface type ifindex permit 25

filter-definition to-abilene match dst-ifindex abilene-interface

filter-definition from-abilene match src-ifindex abilene-interface

filter-primitive UDPTCP type ip-protocol permit tcp permit udp

filter-definition udptcp match ip-protocol UDPTCP

flow-nfilterfilter-primitive OSU type ip-address-prefix permit 128.146/16 permit 140.254/16 permit 164.107/16

filter-primitive DNS type ip-port match 53

filter-primitive WEB type ip-port match 80,8080,443

filter-definition OSUDNS match ip-address OSU match ip-protocol UDP match ip-destination-port DNS

flow-nfilter

• IP address lookup is patricia trie, worst case performance O(W) (W is length of address).

• IP Protocol lookup is bucket, always O(1).

• IP Port lookup is bucket, always O(1).• Performance remains relatively constant

even if loading up prefix list with full internet routing table (100,000+ entries).

flow-nfilter

• Some other lookups use hash tables, for example IP address list or tag list. Usually O(1).

• A few filters do require linear lookups but by nature list is short, ie pps filter or start time filter.

• Short circuit evaluation for AND and OR operations.

flow-split

• Split flow files into smaller files.

• Split based on time, tags, or number of flows.

• Typically used with flow-stat/flow-report for graphing. For example flow-split can produce 5 minute intervals from a day long dataset for time-series graphing.

flow-tag

• Adds a tag field to flows based on IP exporter, IP prefix, Autonomous System, or next hop.

• Used to manage groups of prefixes or ASN’s.

• Example, group IP prefixes by customer ID for billing.

flow-tag## tag format## 0 7 15 23 31# 0000 0000 0000 0000 0000 0000 0000 0000 (32 bits)# RRRRRRRRRRRRRR TTTT NNNNNNNNNNNNNNNNNNN# | | | Site name# | | Site type# | Reserved## BGP community 65000:X is site name (X -> N)# BGP community 65001:Y is site type (Y -> T)# # SITE_NAME_MASK = 0x0000FFFF # SITE_TYPE_MASK = 0x00FF0000## ID Name#---------------------------------# 0x0001 OSU# 0x0002 CWRU# 0x0003 BGSU# 0x0004 UC# 0x0005 UAKRON# 0x0006 WRIGHT# 0x0007 KENT# 0x0008 DAYTON# 0x0009 OBERLIN

# ID Type#------------------------# 0x01 Participant# 0x02 SEGP# 0x03 Sponsored-Participant# 0x04 Gigapop# 0x05 MULTICAST

flow-tagtag-action OHIO-GIGAPOP_DST type dst-prefix# OSU match 128.146/16 set-dst 0x010001 match 164.107/16 set-dst 0x010001 match 140.254/16 set-dst 0x010001 match 192.153.26/24 set-dst 0x010001# CWRU match 129.22/16 set-dst 0x010002 match 192.5.110/24 set-dst 0x010002 # BGSU match 129.1/16 set-dst 0x010003# UC match 129.137/16 set-dst 0x010004# UAKRON match 130.101/16 set-dst 0x010005# WRIGHT match 130.108/16 set-dst 0x010006# KENT match 131.123/16 set-dst 0x010007# DAYTON match 131.238/16 set-dst 0x020008# OBERLIN match 132.162/16 set-dst 0x020009

tag-action OTHER_DST type src-prefix match 0/0 set-dst 0x0

tag-action OTHER_SRC type src-prefix match 0/0 set-src 0x0

tag-definition OHIO-GIGAPOP term input-filter 25 action OTHER_DST action OHIO-GIGAPOP_DST term output-filter 25 action OTHER_SRC action OHIO-GIGAPOP_SRC

flow-header• Display meta information in flow file.

eng1:% flow-header < ft-v05.2002-01-21.093345-0500## mode: normal# capture hostname: eng1.oar.net# exporter IP address: 0.0.0.0# capture start: Mon Jan 21 09:33:45 2002# capture end: Mon Jan 21 09:45:01 2002# capture period: 676 seconds# compress: on# byte order: little# stream version: 3# export version: 5# lost flows: 0# corrupt packets: 0# sequencer resets: 0# capture flows: 341370#

flow-stat

• Generates reports from flow files.• Output is readable and easily imported

into graphing programs (gnuplot, etc).• IP Address, IP address pairs, ports,

packets, bytes, interfaces, next hop, Autonomous System, ToS bits, exporter, and tags.

• Historical – use flow-report.

flow-stat# --- ---- ---- Report Information --- --- ---## Fields: Total# Symbols: Disabled# Sorting: None# Name: Overall Summary## Args: flow-stat -f0 #Total Flows : 111182160Total Octets : 2450050798277Total Packets : 4057574675Total Time (1/1000 secs) (flows): 2414764456464Duration of data (realtime) : 86409Duration of data (1/1000 secs) : 88281720Average flow time (1/1000 secs) : 21718.0000Average packet size (octets) : 603.0000Average flow size (octets) : 22036.0000Average packets per flow : 36.0000

flow-report

• Replacement for flow-stat.• Configuration file based.• Multiple reports per data pass.• Concurrent output per report (file,

program, sorting options, fields).• Integrated tagging and filtering for

performance gain and readability.• ~ 70 reports currently defined.

flow-reportinclude-filter ./filterinclude-tag ./gigapop-tags

stat-report to-abilene-by-customer type source-tag tag-mask 0xFF 0xFF filter to-abilene output path out/to-abilene-by-customer options +header,+names fields -pps,-bps,-duration sort +octets

stat-report from-abilene-by-customer type destination-tag tag-mask 0xFF 0xFF filter from-abilene output path out/from-abilene-by-customer options +header,+names fields -pps,-bps,-duration sort +octets

stat-report from-abilene-by-protocol type ip-protocol filter from-abilene output path out/from-abilene-by-protocol options +header,+names fields -pps,-bps,-duration sort +octets

stat-report to-abilene-tcp type ip-destination-port filter tcp-to-abilene output path out/to-abilene-tcp options +header,+names fields -pps,-bps,-duration sort +octets output path out/to-abilene-tcp.p options +header,+names,+percent-total fields -pps,-bps,-duration sort +octets

flow-reportstat-definition all tag OHIO-GIGAPOP report to-abilene-by-customer report from-abilene-by-customer report to-abilene-by-group report from-abilene-by-group report to-abilene-by-protocol report from-abilene-by-protocol report to-abilene-tcp report from-abilene-tcp report to-abilene-udp report from-abilene-udp

stat-report to-abilene-host-count type ip-source-address-destination-count filter to-abilene output path out/to-abilene-host-count options +header,+names fields -pps,-bps,-duration sort +count

stat-report to-abilene-multicast type ip-source/destination-address filter mcast-to-abilene output path | flow-rptfmt %Y-%d-%m options +header fields -pps,-bps,-duration sort +octets

Flow-report

• The following examples are all for flows received on Oct 2, 2002.

• The output has been formatted with flow-rptfmt.

flow-reportTo abilene by customer

source-tag flows octets packets OHIOU 12468213 346780284114 474584209 OSU 10083752 216222538921 456970704 MIAMI 11555447 85209430257 211139319 UC 3408919 69337541299 150235362 CWRU 2146398 53967968301 71557236 KENT 3548222 35856830763 98904064 UAKRON 680448 30197772896 76634728 WRIGHT 695999 29646284289 57189398 DAYTON 2439731 18139794910 59435826 BGSU 2572766 16310992145 40206277 OBERLIN 802277 10831055817 21659836 CSU 542017 9751230620 17011505 0 578005 8311955805 17920369 XAVIER 509438 8303822859 19977123 DENISON 1335965 4930300208 25778299 CENTRAL_STATE 12113 3587783969 2946397 OHIO-GIGAPOP 145075 3409588817 4352398 OTTERBEIN 393828 3337857516 6427175 (27 records total)

flow-reportFrom abilene by customer

destination-tag flows octets packets OSU 8206984 404410296660 493518138 OHIOU 10632410 378662580090 553882360 UC 3039419 146987191906 166540812 MIAMI 10203261 71462550082 182178469 WRIGHT 625040 64552082817 63606132 KENT 3571049 55469667804 100115387 CWRU 1809929 47298827158 66748219 UAKRON 1203008 42302272842 76792866 BGSU 2556832 27454028852 38833117 DAYTON 4039081 27083462962 64213411 OHIO-ITEC 24500 23197891069 15634326 DENISON 1771432 22412975083 28021003 OSC 74247 21575406055 15617253 XAVIER 539444 17389557985 21496360 OBERLIN 976343 16896684646 21949892 0 495554 14545105602 18824353 BWC 1132527 9667335610 15154831 MULTICAST 37725 9408996691 102650979 CSU 569675 9202402106 16078735 (28 records total)

flow-reportTo/From abilene by group

source-tag flows octets packets PART 36146734 808071443348 1443293483 SEGP 19074659 140069521118 376283762 0 578005 8311955805 17920369 GIGAPOP 253853 7210265828 16180995 SPART 154001 685371077 4022173

destination-tag flows octets packets PART 32214346 1176339350235 1576115766 SEGP 19779445 182946067604 357284495 GIGAPOP 129062 48388568451 35179510 0 495554 14545105602 18824353 MULTICAST 37725 9408996691 102650979 SPART 144483 5741808731 5270298

flow-reportTo/From Abilene by Protocol

ip-protocol flows octets packets tcp 38903962 949344938829 1780524499 udp 14733318 14280174521 70130455 icmp 2054787 578664366 5964561 gre 1104 81503144 230466 169 510943 58796208 816614 esp 671 3200640 20259 ipv6 2467 1279468 13928

ip-protocol flows octets packets tcp 36737595 1389381631832 1905052404 udp 13734354 47439394244 184078339 icmp 1862340 379697738 5177470 gre 906 110143341 225355 169 462826 54259200 753600 esp 646 1672800 12308 ipv6 1731 1577881 11379 ax.25 93 882162 3288 ipencap 92 590868 11124 igmp 12 44296 86 narp 18 2576 40 pup 2 376 8

flow-reportTo Abilene TCP Dest Port

ip-destination-port flows octets packets FastTrack 24.835828 6.603882 11.541223 Gnutella 7.692790 2.308477 4.933651 Napster 0.159431 1.214922 1.358084 7999 0.003208 0.607430 0.240277 http 4.813186 0.491013 2.402686 59 0.000496 0.398512 0.193378 innosys 0.033706 0.396740 0.303003 ssh 0.031968 0.386647 0.218494 smtp 0.360454 0.371606 0.239597 eDonkey-2000 0.064587 0.353453 0.253181 nntp 0.014585 0.336791 0.247589 2234 0.070181 0.335752 0.190099 2702 0.015261 0.314940 0.135489 10021 0.000673 0.302725 0.114831 ftp-data 0.012528 0.293089 1.122457 telnet 0.030663 0.291930 0.143617 1107 0.011328 0.259371 0.107312 7668 0.003095 0.253869 0.100611 Gnutella 0.972104 0.246086 0.593843 Neomodus-Direct-Connect 0.003958 0.204204 0.204218 …

flow-reportFrom Abilene TCP Dest Port

ip-destination-port flows octets packets FastTrack 13.538197 5.509694 9.777353 msg-icp 0.000133 0.934877 0.456198 Neomodus-Direct-Connect 0.007053 0.526346 0.383283 Gnutella 6.074029 0.466143 1.583139 ftp 0.063839 0.394695 0.224846 nntp 0.027901 0.365369 0.303404 4422 0.013477 0.339689 0.179834 http 4.857253 0.319698 1.131411 aol 0.012902 0.257276 0.351811 3819 0.012325 0.221330 0.159259 47399 0.000180 0.217029 0.123149 Napster 0.031839 0.215064 0.369720 smtp 0.357062 0.201978 0.202899 innosys 0.021909 0.181785 0.135780 1976 0.013640 0.173067 0.113857 1097 0.020655 0.166452 0.088196 2925 0.018736 0.165947 0.088380 7776 0.000452 0.158655 0.082350 ftp-data 0.009565 0.154542 0.395363 1677 0.021615 0.148936 0.258940…

flow-reportTo Abilene UDP Dest Port

ip-destination-port flows octets packets 2055 0.006312 17.053153 2.329062 commplex-link 0.000387 12.342609 1.677807 41170 25.035270 10.970910 32.354782 49156 0.000217 5.389666 1.138699 10000 0.002640 3.168155 0.496024 49606 0.000339 2.922246 1.292035 FastTrack 13.620897 2.795093 3.861790 27015 0.351659 2.651897 7.886515 5012 0.000068 2.514314 0.582493 49176 0.000081 2.157357 0.650475 27005 0.001317 2.108271 2.622176 5016 0.000041 2.087801 0.468828 6970 0.000543 2.067873 0.665665 radius 2.717311 1.561363 3.717907 domain 7.108589 1.528589 4.462461 12203 0.000842 1.458771 1.551721 49198 0.000034 1.287165 0.283817 8888 0.000964 1.228599 0.167672 49154 0.001140 1.196136 0.468568 1235 0.002213 0.996790 0.385473 …

flow-reportFrom Abilene UDP Dest Port

ip-destination-port flows octets packets 56464 0.049555 17.677823 51.282764 4252 0.000692 11.352055 1.960886 4247 0.000612 7.053652 1.220310 4244 0.000655 6.386842 1.103243 4250 0.000619 5.128119 0.885821 6970 0.002621 4.618467 1.651840 4245 0.000604 4.574414 0.790175 4242 0.000619 4.147176 0.716384 afs3-prserver 0.026030 3.593472 0.627750 4254 0.000714 2.819961 0.487140 41170 30.282116 2.635842 9.693955 4243 0.000728 2.179707 0.376556 4248 0.001318 1.769443 0.307173 59818 0.001427 1.730026 4.210950 4246 0.000670 1.627574 0.283242 27005 0.003007 1.580597 2.507883 49606 0.000218 1.134879 0.370680 radius 3.873011 1.073899 3.489514 4249 0.000699 1.040643 0.181706 afs3-callback 0.032007 1.015535 0.179987 …

flow-reportTo Abilene Multicast S,G

ip-source-address ip-destination-address flows octets packets 128.146.112.93 224.2.0.1 52 3644792 22848 164.107.73.49 224.2.133.133 49 2104830 8190 164.107.73.50 224.2.133.133 48 2042136 8136 128.146.111.36 224.2.0.1 57 1732352 22988 164.107.73.50 224.2.133.134 48 1502592 8256 164.107.73.49 224.2.133.134 48 1490216 8188 193.166.0.41 224.2.127.254 286 237666 286 171.68.122.14 224.2.127.254 275 224675 275 128.107.150.34 224.2.127.254 199 197010 199 129.217.131.30 224.2.127.254 280 180040 280 192.148.244.23 224.5.5.5 3 149304 102 64.65.127.133 224.2.127.254 287 110495 287 130.83.47.123 224.2.127.254 183 91317 183 171.69.248.71 224.2.127.254 163 61832 163 130.83.126.22 224.2.127.254 168 61824 168 212.219.151.116 224.2.127.254 67 29390 67 205.189.33.76 224.2.127.254 22 27214 22 139.133.204.110 224.2.127.254 50 27000 50 207.75.164.44 224.2.127.254 41 23042 41 …

flow-reportFrom Abilene Multicast S,G

ip-source-address ip-destination-address flows octets packets 155.101.21.38 224.2.127.254 64 14115449 49187 131.252.80.100 224.2.127.254 580 11091500 38113 150.29.224.28 224.2.177.155 54 8569074 9722 131.252.176.50 224.2.127.254 284 6283704 18984 171.69.248.71 224.2.145.19 18 5535907 6313 128.3.10.50 224.2.177.155 60 5488196 7028 131.193.77.102 224.2.177.155 50 4987121 6273 134.174.178.254 224.2.127.254 49 4559324 17402 134.174.178.253 224.2.127.254 49 4557228 17394 128.223.83.33 224.2.127.254 252 4197456 3839 129.105.153.48 224.2.127.254 121 4122602 13059 129.105.153.49 224.2.127.254 116 3628240 11935 129.105.12.35 224.2.177.155 23 3188598 3723 141.225.215.61 224.2.127.254 66 2843808 9416 63.105.122.14 224.2.127.254 206 2719485 4495 131.247.105.10 224.2.133.133 55 2169700 8345 128.227.8.136 224.2.133.133 49 2158428 8366 140.221.8.53 224.2.177.155 23 2154591 2722 …

flow-reportTo Abilene by source Address

ip-source-address flows octets packets 128.146.11.117 0.001658 1.198353 0.436777 64.247.109.217 0.001745 0.827703 0.323890 164.107.77.104 0.001007 0.683243 0.350049 131.187.254.2 0.553731 0.622047 0.615308 164.107.77.100 0.000203 0.490449 0.412104 64.247.105.200 0.010792 0.472982 0.172323 129.22.39.145 0.002991 0.472295 0.164211 132.235.196.5 0.000059 0.443100 0.177759 64.247.65.121 0.011723 0.398491 0.150519 164.107.35.58 0.000751 0.394327 0.154084 128.146.135.184 0.000151 0.393943 0.145393 129.137.150.144 0.006720 0.388005 0.178204 129.22.164.158 0.000557 0.371489 0.137157 64.247.94.142 0.012429 0.371459 0.132373 64.247.110.33 0.002179 0.363492 0.205407 164.107.210.142 0.001585 0.363146 0.197966 137.148.203.177 0.012794 0.351391 0.139850 206.21.71.10 0.687776 0.340878 0.340017 …

flow-reportFrom Abilene by dst Address

ip-destination-address flows octets packets 192.148.248.24 0.000335 0.931192 0.426093 164.107.77.100 0.000206 0.908344 0.494996 64.247.108.177 0.088410 0.694466 0.339623 131.187.254.2 0.526903 0.601483 0.536259 233.2.171.1 0.012888 0.583444 4.505289 129.137.164.187 0.021733 0.529286 0.263854 164.107.242.71 0.019257 0.523264 0.286042 129.137.150.89 0.012727 0.493833 0.252883 64.247.85.174 0.001750 0.486535 0.226529 164.107.3.40 0.019604 0.475143 0.279502 129.137.155.167 0.003318 0.418701 0.248909 206.21.71.10 0.703418 0.417402 0.315859 129.22.130.35 0.023909 0.406043 0.188727 129.22.34.18 0.005377 0.379816 0.204850 192.148.251.54 0.000104 0.374664 0.172263 129.137.240.228 0.002119 0.370352 0.180073 128.146.135.106 0.000263 0.361298 0.197680 164.107.220.83 0.013447 0.361096 0.180690 64.247.65.205 0.001045 0.360547 0.175456 140.254.232.142 0.000386 0.349813 0.160018 …

flow-reportTo Abilene SMTP / Customer

source-tag flows octets packets OSU 25.239783 20.947484 21.210771 MIAMI 13.119068 10.971892 17.199629 OHIOU 3.949198 9.196418 7.206689 DENISON 0.432857 9.156816 5.579017 CWRU 7.459834 8.121457 6.653185 UC 5.421768 8.035289 8.601174 DAYTON 1.192318 5.946226 3.953595 KENT 15.641335 4.911743 6.972634 OBERLIN 1.494677 3.988554 2.602640 WRIGHT 1.337793 3.655101 3.306141 0 0.456390 2.449019 1.528096 BGSU 4.155287 2.430585 3.172927 UAKRON 2.397473 2.240855 1.994214 BWC 0.472078 1.331747 1.045406 OSC 12.000200 1.294894 4.188048 CEDARVILLE 0.479922 1.276318 0.854342 CHMCC 0.704552 1.221819 0.934251 CSU 1.298572 1.101268 1.252083 MCO 0.250301 0.854932 0.566654 FINDLAY 0.084860 0.285458 0.242798 (26 records total)

flow-reportTo Abilene SMTP / Address

ip-source-address flows octets packets 140.141.31.28 0.054196 7.621240 4.219975 131.238.75.33 0.482775 4.601938 2.969837 132.162.1.220 0.733789 3.700020 2.225292 130.108.128.60 0.759461 3.048794 2.735314 134.53.253.21 2.367522 3.025292 4.628053 129.137.2.198 0.800822 2.759806 1.988307 64.247.72.226 0.024246 2.716476 1.904014 134.53.253.22 2.437407 2.517202 3.913066 192.153.34.91 0.046352 2.402720 1.415066 131.123.72.253 13.613252 2.224239 4.916420 134.53.7.26 7.292967 2.135118 5.008120 128.146.216.45 0.526275 2.114172 1.957881 128.146.216.43 3.745249 2.021460 2.449103 129.22.8.4 0.755183 2.019472 1.396735 140.254.120.28 0.517004 1.849312 1.211226 140.141.2.2 0.280965 1.496925 1.315912 131.123.250.221 0.187548 1.478498 0.869953 132.235.8.45 1.506799 1.474040 1.207475 129.137.3.131 0.362973 1.469402 1.203561 …

flow-report

flow-report

flow-report

flow-dscan• DoS detection / network scanning tool.

• Flag hosts which have flows to many other hosts.

• Flag hosts which are using a large number of TCP/UDP ports.

• Works better on smaller networks or with flow-filter to limit traffic. For example filter TCP port 25 to detect hosts infected with e-mail worm.

flow-gen

• Debugging tool to generate flows.eng1:% flow-gen -V8.1 | flow-print | head -10 srcAS dstAS in out flows octets packets duration0 65280 0 65280 2 1 1 4294901760 1 65281 1 65281 4 2 2 4294901760 2 65282 2 65282 6 3 3 4294901760 3 65283 3 65283 8 4 4 4294901760 4 65284 4 65284 10 5 5 4294901760 5 65285 5 65285 12 6 6 4294901760 6 65286 6 65286 14 7 7 4294901760 7 65287 7 65287 16 8 8 4294901760 8 65288 8 65288 18 9 9 4294901760

flow-send

• Transmit flow files with NetFlow protocol to another collector.

• Can be used to take flow-tools files and send them to other NetFlow compatible collector.

flow-receive• Like flow-capture but does not manage

disk space. Output is to standard out and can be used directly with other flow-tools programs.

• Typically used for debugging.

eng1:% flow-receive 0/0/5555 | flow-print flow-receive: New exporter: time=1011652474 src_ip=199.18.112.114 dst_ip=199.18.97.102 d_version=8srcPrefix srcAS dstPrefix dstAS input output flows 143.105/16 600 128.9/16 4 48 25 1 140.141/16 600 150.216/16 81 48 25 4 132.235/16 17135 130.49/17 4130 38 25 25 131.123/16 11050 129.59/16 7212 42 25 1 206.21/16 600 128.239/16 11975 48 25 2 199.218/16 600 128.255/16 3676 48 25 1

flow-import

• Import flows from other formats into flow-tools.

• Currently supports ASCII, cflowd and Cisco NFC formats.

flow-export

• Export flows from flow-tools files to other formats.

• Currently supports ASCII, cflowd and MySQL formats.

• ASCII output can be used with perl or other scripting languages (with a performance penalty).

flow-xlate

• Translate flows among NetFlow versions.

• Originally intended for use with Catalyst switches since they export some flows in version 7 and others in version 5 format.

• Can also mask tag values.

Future work

• Flow-report – more reports, time series data generation, data formatting.

• Clean interface to rrdtool and possibly other graphing utilities.

• Flow-report -> SQL for billing or long term trend analysis.

• IPFX support.• Cleaner interface to BGP information.• Probe support.

Contributed software• Dave Plonka’s Cflow module. Perl

interface to flow-tools. Allows flow-tools to be the back-end to flowscan.

• Robin Sommer’s Python interface.• E. Larry Lidz @ U Chicago Network

forensics tools.• Miguel Paraz & William Emmanuel

inter.net billing and other tools.• Other stuff – look in contrib area.

References

• flow-tools: http://www.splintered.net/sw/flow-tools

• Abilene NetFlow page http://www.itec.oar.net/abilene-netflow

• Simon Leinen’s FloMA Pointers & Software page: http://www.switch.cf/tf-tant/floma/software.html

• IETF standards effort: http://ipfix.doit.wisc.edu