CONSIDERATIONS WHEN DEPLOYING LAYER 2 ETHERNET SWITCHES

AUGUST 2010

Abstract: This paper reflects the work done to increase our understanding of protocol vulnerability, and makes a number of recommendations to network specialists for the design and implementation of secure Ethernet configurations. The guidance also includes 10 questions for a potential customer to ask their switch vendor before committing to a purchase. Disclaimer: CPNI has taken every care in preparing this protective security advice, which is informed by intelligence on the threat. However, CPNI cannot accept any liability to any person or company for any financial loss or damage arising from the use of this advice or from any failure to give advice.

Executive summary

VLAN technology should not be used as a security mechanism to separate data classifications. However, there are numerous other valid uses for Layer 2 technology. It is important that Layer 2 technology is used safely to avoid leaving networks and data exposed to well-publicised vulnerabilities. CPNI coordinated a structured set of tests to highlight the potential issues. Nine vendors participated in a set of tests, involving 18 different products, to establish where useful guidance could be developed to secure common implementations of Layer 2 and in particular carrier-grade or Metro-Ethernet protocols. It is important to note that while no significant vulnerabilities were found, a number of generic issues, which can be easily fixed but are also commonly overlooked, were identified. Recommendations have been developed to guide implementation planning and network implementation. Additionally a series of 10 questions have been developed for potential customers to aid their understanding of the steps taken to secure the products by the equipment manufacturer or vendor.

2

Introduction

UK Government Policy on data communications specifies that, for systems supporting protectively marked information, Virtual Local Area Network (VLAN) technology is not considered to be a suitable security mechanism to be used to separate data classifications. However, there are numerous other valid uses of VLAN which are fundamental to Next Generation Network (NGN) architecture. OSI Layer 2 Ethernet architecture is flexible and implementations of the standard can vary, although generally based on available good practices provided by the product vendors. There are however vulnerabilities inherent in the Layer 2 protocols and it is important to understand the Layer 2 issues better, and so CPNI and industry partners determined to coordinate a series of tests on Layer 2 Ethernet implementations. Tests were carried out in two phases; Phase 1 took place in 2008/9, involving 6 vendors, and Phase 2 was carried out during 2009/10. Phase 2 expanded the list of vendors and equipment, and included testing at higher protocol layers where appropriate.

Equipment testing

The aim of the testing was to identify whether common vulnerabilities existed across a representative sample of carrier grade Ethernet switches. Codenomicon was asked to develop specific tools to test Ethernet and Metro Ethernet protocols. Equipment was tested in Codenomicon, vendor and operator labs depending upon equipment availability. 9 vendors provided appropriate equipment which had a Layer 2 protocol stack and relevant Ethernet interfaces. 18 different products were tested. It is important to note that equipment vendors were fully engaged with the protocol testing project with Codenomicon and provided expertise as well as equipment. A vendor representative was invited to each set of tests involving their equipment to ensure testing took place within correct parameters and to rectify any issues discovered. Vendors and participating operators received full test results for their specific equipment under test. No specific vendor details or product vulnerability details will be disclosed publicly. In every case, minor protocol weaknesses were discovered. Equipment manufacturers and vendors were given full test results and were encouraged to fix any issues. It was a vendor responsibility to communicate the relevant fixes to their customers.

3

The tests were developed and executed against the following protocol interfaces. Note that not all of the protocols were present in every tested device. In fact, none of the tested devices supported all of these protocols:

Basic IEEE 802.3 Ethernet BFD CFM E-LMI GARP OAM/LFM (802.3ah)

LLDP PBT/PBB-TE L2TPv3 LACP STP/RSTP/MSTP PPPoE

4

Summary of findings

The capabilities in the switches under test varied quite widely. Some switches were traditional Layer 2 switches with only minimal Layer 3 management plane functionality, while some other switches implemented many of the more complex Metro-Ethernet protocols such as Connectivity Fault Management (CFM) and Bidirectional Forwarding Detection (BFD), as well as full Layer 3 routing capabilities. This presented a challenge and also resulted in suboptimal testing coverage of some vendors' equipment in Phase 1 and led directly to the extended testing of Phase 2. As a very crude rule of thumb, the more Layer 2/3 protocol interfaces the switch presented, the more likely it was to fall in the face of an attack. Testing did not detect the presence of any fundamental Layer 2 protocol problems such as VLAN breakouts, thought to be due to the fact that such attacks are widely documented and recognized and understood by the manufacturers design and implementation teams. Testing of basic Ethernet frames, defined by the core IEEE 802.3 Ethernet specification, did not produce dramatic or unexpected results as the basic frame structure is not very complex and devices do not implement much functionality in analyzing basic frames when forwarding traffic. However, Metro-Ethernet extensions to the core specification provide enough complexity to require further processing logic and therefore can expose some loose implementations to vulnerabilities. It is important to note that while no significant vulnerabilities were found, a number of generic issues, which can be easily fixed but are also commonly overlooked, were identified. In every case, minor issues were discovered. In general these were not considered serious issues. Where more serious issues were discovered, the vendor invariably developed a fix during the testing phase. It was seen as a vendor responsibility to inform their customer base if existing configurations were affected. Metro-Ethernet protocols are quickly reaching a level of equal complexity with the IP protocol family. As a general result, the testing showed that Metro Ethernet protocol implementations are subject to exactly the same implementation-level flaws that have plagued routers and other OSI Layer 3 equipment over the past 10-20 years. CPNI, Codenomicon, industry specialists and vendors have examined the results and developed a number of recommendations. This document focuses on recommendations and good practices that network designers and architects should consider during the planning stages, and that implementation teams should take into account when configuring switches and routers. Equipment manufacturers design and build their products to meet commercial demand, and Service Providers are strongly urged to consider asking for design features that can be shown to meet such recommendations when specifying equipment. Appendix 1 contains a glossary, useful reading and pointers to previous research in this area.

5

Recommendations

Recommendations are divided into three areas: 10 questions to ask a vendor at the procurement stage; 6 recommendations for more secure design at the planning stage; and 6 recommendations for safer configurations at the network implementation stage.

Procurement: 10 Questions to ask your switch vendor

Basic equipment design:

1. From a complexity standpoint, integrated Layer 2 switches / Layer 3 routers present a much larger attack surface than simple Layer 2 devices. More modular devices empower customers to choose the complexity level of the device. What are the advantages for me of purchasing a more complex device?

2. The same testing practices for testing software and protocols at Layer 3 and above should be used for verifying robustness of Layer 2 interfaces. How much testing do you perform at the Layer 2 interface level?

3. All code should be rigorously tested and reviewed for security bugs, especially where development has been outsourced. Can you show me what steps you have taken, in the form of a due diligence approach, to test software code?

4. A security flag in the basic configuration to auto-configure the device to its most secure state would be a major security improvement. Does your product offer a secure configuration, and if not why not?

5. System architecture design should allow unnecessary protocol logic to be turned off. When specific protocol functionality is turned off, no part of the system should analyse or attempt to make sense of protocol structures belonging to that particular protocol. What guarantees can you offer me that, once turned off, unnecessary protocol logic will not respond or otherwise try to analyse any structures associated with that protocol?

Default conditions for out of the box use:

6. Customers should be encouraged to enable protocols only when they require them, limiting the default attack surface of the device. Can you demonstrate how you build security-optimised devices and turn off all unnecessary protocol interfaces by default?

7. Intelligent customers need the FULL configuration to be made available; often a protocol interface may be exposed to the network even though the device configuration file contains no reference to it. Can you guarantee that we will be given a FULL configuration for the device?

6

8. Protocols with known flaws such as STP and LACP should be disabled by default unless a customer has specifically asked for them. If a product contains some kind of mitigation features for known attacks against weak protocols such as STP, then these mitigation functions should be turned on by default where these protocols must be used. Can you demonstrate where you have disabled or otherwise addressed protocols with known flaws?

9. Where possible a customer should be able to use RSTP or MSTP in place of basic STP,

or consider employing a proprietary protocol to achieve similar functionality. Do you have a process to identify weak protocols and suggest to your customers a more secure or robust alternative?

10. Customers need a secure deployment guide for your products that is shipped as part of the product. This guide could usefully contain details on designing networks and topologies that are as resilient as possible towards both currently known and as yet unknown fundamental Layer 2 attacks. Do you have a process to identify customer-specific security requirements and the ability to deliver a secure deployment guide?

Implementation planning and design stage:

1. Be aware of the existing configuration guidance which exists for the chosen products. The advice given in this document is supplemental to much of what is considered to be business as usual configuration guidance such as: use secure management protocols, and create a strong password etc. The information in Appendix 1 provides some good starting points, and it is recommended that the latest secure configuration guides are obtained from your vendors.

2. Document why each feature is required and how it should be configured in the planned environment.

3. Typically a switch will have a default configuration where the majority of features are enabled. This scenario will expose input paths to the protocols running on the switch from other devices and from people connected to the switch. Do not allow the default configuration to run in the live environment.

4. Once unused functionality and protocols are designed out, recognize that all remaining protocols will be accessible from connected networks and devices. Enumerate the vulnerabilities associated with each of these remaining protocols and consider the consequences if those vulnerabilities are exploited from each connected network (or device). Document these findings in a Risk Register, and draw up a plan for how the risk from the discovered vulnerabilities can be mitigated.

7

5. Avoid using protocols that have known design flaws and/or that can be abused easily to create load for switch processors, especially on interfaces through which external parties or customers may be able to generate Layer 2 traffic. This includes LLDP and other discovery protocols, STP and other Layer 2 topology convergence protocols, and LACP/GVRP and other trunking/aggregation protocols. Some alternative proprietary or public protocols may also be considered to camouflage the standard attack surface. Examples are too numerous to list here, but it should be remembered that any given alternative may also come with its own problems, so should be rigorously tested before deployment.

6. Avoid topologies and configurations that allow attackers to disrupt network operations with

known protocol design flaws (e.g. STP, LACP, ARP).

Network implementation stage:

1. Turn off or disable protocols on interfaces where they are not required. For example, spanning tree (STP) can be disabled on specific logical or physical interfaces where it is not required.

2. Permanently disable the process that provides a protocol where that protocol is not used

at all.

3. Only enable OAM and any other "optional" protocols in those customer interfaces where they are absolutely required - every complex protocol increases your vulnerable attack surface.

4. Always recheck device configurations after software upgrades, as new software may introduce new vulnerabilities.

5. Actively audit vendors' default configurations for the presence of unwanted protocols. (Remember that Layer 2 is fundamentally a broadcast domain).

6. Be aware that in some switches a protocol may be turned on by default, even if it is not explicitly shown to be enabled in a configuration file. The exact procedure for doing this varies, but the basic means for detecting the presence of unwanted Layer 2 services is to review the switch configuration, issue commands to check the status of a particular protocol service, and to look at the process table in the switch (if available) and see if separate protocol services can be observed as separate processes. For Layer 2, discovering unwanted services can be very difficult. Service Providers should consider consulting the switch vendor for more information. For protocols at Layer 3 and above, this type of service discovery is more trivial, since it can also be done using port scans (e.g. nmap) in addition to system-intrinsic investigations.

8

Acknowledgements

CPNI gratefully acknowledge the help and support of the following during the preparation of this report:

Alcatel-Lucent AT&T BSkyB British Telecom Cable & Wireless CESG Cisco Systems Codenomicon COLT Extreme Networks Foundry Fujitsu Global Crossing

Hutchison 3G Huawei Juniper Networks LINX (London Internet Exchange) Nortel Networks O2 Orange Talk Talk Tellabs T-Mobile Verizon Virgin Media Vodafone

9

Appendix 1: Glossary, useful reading and previous research

Layer 2 Standards:

Bridging IEEE 802.1D Ethernet IEEE 802.3 VLAN IEEE 802.1Q LACP IEEE 803.ad PBT IEEE 802.1ay

STP IEEE 802.1D VPLS RFC4761/2 LLDP IEEE 802.1AB PPP/PPPoE RFC1661/2516 L2TPv3 RFC3931

Acronyms:

ARP Address Resolution protocol BFD Bi-Direction Forwarding Detection BPDU Bridge protocol Data Unit CFM Connectivity Fault Management CPU Central Processor Unit E-LMI Ethernet Local Management Interface GARP Generic Attribute Registration Protocol GVRP Generic VLAN Registration Protocol OAM Operations Administration and Management LACP Link Aggregation Control Protocol LFM Link fault Management LLDP Link Layer Discovery Protocol L2TP Layer 2 Tunnelling Protocol MAC Medium Access Control MPLS Multiprotocol Label Switching MSTP Multiple Spanning Tree NNI Network-Network Interface PBB Provider Backbone Bridge QoS Quality of Service RSTP Rapid Spanning Tree Protocol PBT Provider Backbone Trunking PPP Point to Point Protocol STP Spanning Tree Protocol SUT Switch under test S-VLAN Stacked VLAN UNI User-Network Interface VLAN Virtual Local Area Network

10

Useful reading

A seminal presentation specifically on Metro Ethernet / Carrier Ethernet protocol extensions and environments is "Security Best Practices for Carrier Ethernet Networks and Services" (Santitoro 2008). This presentation is available at: http://metroethernetforum.org/PPT_Documents/CEWC2008/Carrier-Ethernet-Security-Santitoro-MEF-CEWC-2008.ppt Cisco has also published a very good white paper on Layer 2 vulnerabilities as an addendum for their earlier SAFE Enterprise white paper called "SAFE Layer 2 Security In-Depth - Version 2" (Dubrawsky 2004). This white paper can be found at http://www.cisco.com/warp/public/cc/so/cuso/epso/sqfr/sfblu_wp.pdf NSA document (Cisco IOS switch configuration) http://www.nsa.gov/snac/os/switch-guide-version1_01.pdf Foundry Networks (Enhancing Internal Network Security) http://www.foundrynet.com/pdf/wp-enhancing-lan-security.pdf Title: Building Resilient IP Networks Author: By Kok-Keong Lee, Fung Lim, Beng-Hui Ong Publisher: Cisco Press Chapter 6: Design resilient Layer 2 networks by understanding the access module Title: LAN Switch Security Author: By Eric Vyncke and Christopher Paggen Publisher: Cisco Press

Previous research

The state-of-the-art in tools for Layer 2 testing has been the Yersinia framework (http://www.yersinia.net/). The Yersinia authors refer to a master's thesis by Guillermo Marro (2003), which provided a very comprehensive look at Spanning Tree (STP) vulnerabilities. Marro's thesis is available at http://seclab.cs.ucdavis.edu/papers/Marro_masters_thesis.pdf Marro in turn acknowledges earlier work done by Sean Convery (2002), which documented a lot of the Layer 2 issues that are now considered as the starting point for recent research. As a result of research work done by Convery et al, several papers on Layer 2 issues have originated from Cisco in recent years. A seminal and highly recommended presentation on Layer 2 vulnerabilities and attack mitigation techniques was published by Yusuf Bhaiji (2005). This presentation is available at http://www.sanog.org/resources/sanog7/yusuf-L2-attack-mitigation.pdf

11

12

Some of the material in the above presentation seems to come from another earlier Cisco presentation (2003), that also contains some additional materials on attacks against STP and 802.1X/EAP. This presentation is available at http://www.seanconvery.com/SEC-2002.pdf The material above has also been covered in very similar style by others, for example Figueroa, Figueroa & Williams 2007, whos presentation is available at http://www.defcon.org/images/defcon-16/dc16-presentations/defcon-16-figueroa-williams.pdf Tools for testing at Layer 2 Nmap http://nmap.org/Hping http://www.hping.org/Yersinia http://www.yersinia.net/Scapy http://www.secdev.org/projects/scapy/Wireshark http://www.wireshark.org/TCP Dump http://www.tcpdump.org/Ettercap http://ettercap.sourceforge.net/Mausezahn http://www.perihel.at/sec/mz/index.html Nemesis http://nemesis.sourceforge.net/ packETH http://packeth.sourceforge.net/

Executive summary IntroductionEquipment testing

Summary of findings RecommendationsProcurement: 10 Questions to ask your switch vendorBasic equipment design:Default conditions for out of the box use:

Implementation planning and design stage:Network implementation stage:

Acknowledgements Appendix 1: Glossary, useful reading and previous researchLayer 2 Standards:Acronyms:Useful readingPrevious research