The Evolution of Network Configuration:A Tale of Two Campuses

Hyojoon Kim†, Theophilus Benson‡

Aditya Akella‡, Nick Feamster†

†Georgia Tech

‡University of Wisconsin, Madison

1

What is Network Configuration?

• Collection of configuration files

• Express network policy

• Determines the overallnetwork behavior

2

The Network State Changes

• Topology change

• Policy change

Configuration change

3

How does network configuration change over time?

Growth of firewalls in Georgia Tech

Configuration Changes

4

Georgia TechNetwork Devices

Number of line changes

Routers 326,458Firewalls 539,171Switches 353,420

Total 1,219,049

Line changes in the past 5 years

What are causing the changes?

Where are the changes happening?

Is there a noticeable pattern?

Our Contribution

• Examine change patterns over time

• Look at many different types of devices

• Provide better understanding– Help develop better configuration tools• e.g., Change recommendations, feedbacks

– Reduce misconfigurations

5

Our Data

• Configuration data from two campus networks–5 years of accumulated configuration files

• Tools– CVS– RANCID (Really Awesome New Cisco confIg Differ)

6

Collecting Configuration Files

7

Pull configuration

CVS Server

CVS commit

…

RANCID

Remote login(telnet, ssh)

Revision Control on Configuration Files

• When is the change?

• What changed?

• Regenerate eachrevision

8

... 1.51log@Fri Feb 5 15:04:28 EST 2010@text@a141 1 port-object range bootps bootpca160 4object-group service 12-123-12-13-any-udp udp port-object range bootps bootpcobject-group service 12-123-12-14-any-udp udp port-object range bootps bootpcd173 16a188 9object-group service 13-14-15-16-any-udp udp port-object range bootps bootpcobject-group service 14-15-16-17-any-udp udp...

RCS Format

Our Approach

9

Data(RCS)

Revisions

SnapshotAnalysis

Change Analysis

LongitudinalAnalysis

CorrelationAnalysis

Group simultaneouschanges

Take latest snapshot

Compare revisions

Sort revisions by time

Classifying Configuration lines

10

logging buffered 1024000enable secret [deleted]username [deleted]aaa new-model…Interface Port-channel1 description WiSM-A virtual channel switchport trunk encapsulation dot1q switchport trunk allowed vlan 316,805,807-809,816,1296,1312 switchport mode trunk…router ospf xxxx router-id x.x.x.x…ip access-list extended access-vty-in permit tcp x.x.0.0 0.0.255.255 any range 22 telnet log-input permit tcp x.x.0.0 0.0.255.255 any range 22 telnet log-input…

Management

Layer 1

Layer 2

VLAN

Layer 3

ACL

Security

Control Filter

QoS

Overview of Results

• Routers are multi-functional – Univ. of Wisc: Layer 3 changes are 30% of total changes– Georgia Tech: Layer 3 changes are 5% of the total changes

• Firewall changes are concentrated on ACL– Around 87% of the total changes– Steep increase in the access control list lines

• Switches are about providing connectivity– Port-centric changes

11

Change Analysis on Routers

12

Number of line changes in all routers over 5 years - GT

Static ARP

78%

Change Analysis on Firewalls

13

Number of changes in all Georgia Tech firewalls over 5 years

Access Control

87%

Longitudinal Analysis on Firewalls

14

Change in number of Lines in all Georgia Tech firewalls

Change in number of firewallsin Georgia Tech

Change Analysis on Switches

15

Number of line changes in all switches in Univ. of Wisconsin

snmp trap

Correlation Analysis on Switches

16

Univ. of Wisconsin SwitchesCorrelated changes %

ACL, L1 24%L1, VLAN 11%

L1, L2, MGT 11%MGT, L1 10%

VLAN, MGT 9%

Conclusion

• Study on how network configuration changes over time

• Reveal interesting characteristics about network changes– Magnitude and frequency of changes– Causes of changes

17

Conclusion

• Provide better understanding

• Improve current methods of configuring and managing network devices– Change recommendations– Reduce misconfigurations– More automation

Questions?joonk@gatech.edu

18

Georgia Tech Network

19

Routers Firewalls Switches Total

16 365 716 1,097