CLARKSON UNIVERSITY

A CRITICAL ANALYSIS OF LAYER 2 NETWORK SECURITY IN

VIRTUALIZED ENVIRONMENTS

A Doctoral Dissertation

By

Ronny L. Bull

Department of Computer Science

Submitted in partial fulfillment of the requirements

for the degree of

Doctor of Philosophy, Computer Science

Fall 2016

Accepted by the Graduate School

,

Date, Dean

A CRITICAL ANALYSIS OF LAYER 2 NETWORK SECURITY IN

VIRTUALIZED ENVIRONMENTS

The undersigned have examined the dissertation entitled A Critical Analysis of Layer 2Network Security in Virtualized Environments presented by Ronny L. Bull, a candidatefor the degree of Doctor of Philosophy in Computer Science, and hereby certify that it is

worthy of acceptance.

Date

Dr. Jeanna Matthews Dr. Christopher Lynch

Dr. Yaoqing Liu Dr. Burak Kantarci

Dr. John Marsh

ABSTRACT

Cloud service providers offer their customers the ability to deploy virtual machines in

a multi-tenant environment. These virtual machines are typically connected to the physi-

cal network via a virtualized network configuration. This could be as simple as a bridged

interface to each virtual machine or as complicated as a virtual switch providing more ro-

bust networking features such as VLANs, QoS, and monitoring. In this research, I explore

whether Layer 2 network attacks that work on physical switches apply to their virtualized

counterparts by performing a systematic study across seven different virtual networking

configurations using enterprise grade hypervisor environments. First, I used a malicious

virtual machine to run a MAC flooding attack and evaluate the impact on co-resident vir-

tual machines. Then I investigated the effects of using a malicious virtual machine to run

a rogue DHCP server under multiple DHCP attack scenarios. After that, I evaluated each

of the seven virtual network configurations against a series of VLAN hopping experiments,

and finally looked at how effective ARP poisoning attacks are in virtualized networks by

performing scripted Man-in-the-Middle attacks across every major hypervisor platform. In

summary, it was found that some of the evaluated hypervisor platforms utilize virtualized

networks that are more resilient to certain classes of Layer 2 attacks than others. However,

no single environment proved to have the capability of mitigating every attack.

iii

ACKNOWLEDGMENTS

Special thanks go out to the following individuals, without their support this research

would not have been possible:

Dr. Jeanna Matthews (Clarkson University - Research advisor)

Provost John Johnson (Utica College - Research funding and institutional support)

Dr. John Marsh (SUNYIT - Research funding and institutional support)

Christopher Urban (SUNYIT - Research funding and institutional support)

Nick Merante (SUNYIT - Research equipment acquisition, and for letting me run

malicious experiments on his network!)

Micheal Moore (Harris Critical Networks - A good friend and mentor who diligently

proof read every publication)

Prof. John Cook (Herkimer College - For helping me find this path so many years

ago!)

Many thanks go out to all of the people that have helped make this project happen,

especially to all of my student research assistants that helped with some of the grunt work!

To my family, thank you for all of your support and patience throughout this process.

iv

This work is dedicated to my late Grandmother Susan M. Dawidowicz who passed

away January 13th, 2016 at the age of 92. She was an inspiration and a reminder that hard

work and dedication are the keys to getting past any obstacle in life.

v

TABLE OF CONTENTS

LIST OF TABLES xii

LIST OF FIGURES xiv

1 BACKGROUND AND RELATED WORK 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 LAYER 2 NETWORKING 5

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Collision Domain vs. Broadcast Domain . . . . . . . . . . . . . . . . . . . 8

2.3 Common Layer 2 Networking Protocols . . . . . . . . . . . . . . . . . . . 9

2.3.1 IEEE 802.3 Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.3.2 Address Resolution Protocol (ARP) . . . . . . . . . . . . . . . . . 11

2.3.3 Spanning Tree Protocol (STP) . . . . . . . . . . . . . . . . . . . . 12

2.3.4 Virtual Local Area Networks (VLAN) . . . . . . . . . . . . . . . . 13

2.3.5 Dynamic Host Configuration Protocol (DHCP) . . . . . . . . . . . 15

vi

2.4 Virtual Layer 2 Networking Options . . . . . . . . . . . . . . . . . . . . . 16

2.4.1 Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.4.2 Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.5 Layer 2 Network Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.5.1 Media Access Control (MAC) Attacks . . . . . . . . . . . . . . . . 20

2.5.2 Dynamic Host Configuration Protocol (DHCP) Attacks . . . . . . . 21

2.5.3 Virtual Local Area Network (VLAN) Hopping . . . . . . . . . . . 21

2.5.4 Address Resolution Protocol (ARP) Spoofing . . . . . . . . . . . . 22

2.5.5 Spanning Tree Protocol (STP) Attacks . . . . . . . . . . . . . . . . 22

3 RESEARCH ENVIRONMENT 25

3.1 Original Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2 Upgraded Research Environment . . . . . . . . . . . . . . . . . . . . . . . 28

4 MAC FLOODING ATTACK 31

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.2 Attack Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.3.1 Linux 802.1d Bridged Interface . . . . . . . . . . . . . . . . . . . 33

4.3.2 Open vSwitch 1.11.0 Interface . . . . . . . . . . . . . . . . . . . . 35

4.3.3 Open vSwitch 2.0.0 Interface . . . . . . . . . . . . . . . . . . . . . 35

vii

4.3.4 Citrix XenServer 6.2 . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.3.5 Microsoft Hyper-V Server 2008 R2 . . . . . . . . . . . . . . . . . 37

4.3.6 VMware vSphere (ESXi) 5.5 - free edition . . . . . . . . . . . . . 37

4.3.7 Summary of MAC Flooding Results . . . . . . . . . . . . . . . . . 38

4.4 Mitigation Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5 DHCP ATTACKS 42

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.2 Attack Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.2.1 Seeding Clients With a Poisoned DNS Server . . . . . . . . . . . . 43

5.2.2 Providing Clients With an Invalid or Malicious Default Gateway . . 45

5.2.3 Remote Execution of Code . . . . . . . . . . . . . . . . . . . . . . 45

5.3 Malicious Extension of the ShellShock Proof of Concept Attack . . . . . . 47

5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.5 Mitigation Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6 SWITCH SPOOFING VLAN HOPPING ATTACK 52

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

6.2 Attack Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

6.4 Mitigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

viii

7 DOUBLE-TAGGING VLAN HOPPING ATTACK 61

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

7.2 Attack Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

7.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.4 Mitigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

8 ARP SPOOFING - MAN-IN-THE-MIDDLE ATTACK 70

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

8.2 Attack Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

8.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

8.4 Mitigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

9 CONCLUSION 75

APPENDICES 88

A RESEARCH DEMO VIDEOS 89

B DHCP AND DNS SERVER CONFIGURATIONS 91

B.1 Legitimate DHCP Server /etc/hosts File . . . . . . . . . . . . . . . . . . . 91

B.2 Legitimate DHCP Server Configuration File . . . . . . . . . . . . . . . . . 91

B.3 Rogue DHCP Server /etc/hosts File . . . . . . . . . . . . . . . . . . . . . 92

B.4 Rogue DHCP Server Configuration File . . . . . . . . . . . . . . . . . . . 92

ix

C INITIAL SWITCH CONFIGURATIONS FOR VLAN HOPPING EXPERIMENTS 93

C.1 Cisco 2950 48-Port Switch . . . . . . . . . . . . . . . . . . . . . . . . . . 93

C.2 Cisco 2950 24-Port Switch . . . . . . . . . . . . . . . . . . . . . . . . . . 95

D SCRIPTS 96

D.1 MAC Flooding Attack Scripts . . . . . . . . . . . . . . . . . . . . . . . . 96

D.1.1 TCPDump HTTP Traffic Sniffer . . . . . . . . . . . . . . . . . . . 96

D.1.2 GNUPlot Scripts for Latency Graphs . . . . . . . . . . . . . . . . 96

D.2 Rogue DHCP Server Monitoring Scripts . . . . . . . . . . . . . . . . . . . 97

D.2.1 TCPDump DHCP Monitoring Script . . . . . . . . . . . . . . . . . 97

D.2.2 Python/Scapy Rogue DHCP Server Identification Script . . . . . . 99

D.3 VLAN Hopping Attack and Monitoring Scripts . . . . . . . . . . . . . . . 100

D.3.1 TCPDump VLAN Traffic Monitoring Script . . . . . . . . . . . . . 100

D.3.2 VLAN Hopping Python Script . . . . . . . . . . . . . . . . . . . . 100

D.4 ARP Poisoning Man-in-the-Middle Attack Scripts . . . . . . . . . . . . . . 101

D.4.1 ARP Poisoning Python Script . . . . . . . . . . . . . . . . . . . . 101

D.4.2 TCPDump PCAP Parser . . . . . . . . . . . . . . . . . . . . . . . 103

E MAC FLOODING PERFORMANCE EVALUATION DATA SETS 104

E.1 Gentoo/Xen Bridged . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

E.2 Gentoo/Xen Open vSwitch . . . . . . . . . . . . . . . . . . . . . . . . . . 106

x

E.3 Citrix XenServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

E.4 Microsoft Hyper-V (Standard Switch) . . . . . . . . . . . . . . . . . . . . 108

E.5 Microsoft Hyper-V (Cisco Nexus 1000v) . . . . . . . . . . . . . . . . . . 109

E.6 Proxmox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

E.7 VMWare vSphere ESXi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

E.8 Cisco 2950 Hardware Switch . . . . . . . . . . . . . . . . . . . . . . . . . 112

xi

LIST OF TABLES

3.1 Summary of the orignial test environment hardware. . . . . . . . . . . . . . 26

3.2 New virtual machines added to each hypervisor platform for Layer 2 DHCPattack testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Summary of hypervisor platforms and virtual switch configurations in-stalled to the new hardware. . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.1 MAC flooding attack results across seven test environments. 3indicatesthe platform was affected. . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.1 DHCP attack scenario results across seven test environments. 3indicates asuccessful attack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6.1 Switch spoofing attack results across the seven virtual test environmentsand a physical control system. 3indicates the attack was successful. . . . . 58

7.1 Physical double-tagging attack scenario results across the seven virtualtest environments. 3indicates that a frame was successfully sent from thephysical attacking system to a target virtual machine located on VLAN 20within the corresponding hypervisor environment. . . . . . . . . . . . . . . 67

7.2 Virtual double-tagging attack scenario results across the seven virtual testenvironments. 3indicates that a frame was successfully sent from the vir-tual attacking system to a target system located within a separate virtualnetwork on VLAN 20. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

xii

8.1 ARP Poisoning Man-in-the-Middle attack results across the seven virtualtest environments. 3indicates the platform was affected. . . . . . . . . . . 73

9.1 Summary of all attack scenario results across the seven different test envi-ronments. 3indicates that the platform is vulnerable to the correspondingclass of attacks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

xiii

LIST OF FIGURES

2.1 The Open Systems Interconnect (OSI) model. The solid line indicates thedata path from the transmitter to the receiver, the dotted lines represent thelogical connectivity between each of the seven layers on the sending andreceiving sides. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 An 802.3 Ethernet Frame. . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3 An 802.3 Ethernet Frame with an Added 802.1Q VLAN Tag. . . . . . . . . 14

2.4 Dynamic Host Configuration Protocol process. . . . . . . . . . . . . . . . 16

2.5 A basic bridge using a forwarding table to pass requests between two net-work segments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.6 A switch and its CAM table. . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.7 An 802.3 Ethernet Frame. Switches just need to match the destinationaddress from the frame to an entry in its CAM table in order to forward iton to the correct port. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1 A malicious virtual machine located on a multi-tenant virtual network. . . . 27

4.1 A malicious virtual machine running macof to flood a virtual network withrandom MAC addresses and malformed packets. . . . . . . . . . . . . . . . 32

xiv

4.2 Network diagram for MAC flooding attack scenarios. . . . . . . . . . . . . 33

4.3 Latency measured using the ping utility on a bridged virtual network duringa MAC flooding attack. The attack was launched at ICMP request 61 andterminated at ICMP request 241. . . . . . . . . . . . . . . . . . . . . . . . 34

4.4 A malicious virtual machine running macof on an Open vSwitch virtualnetwork and successfully sniffing HTTP traffic with Wireshark from an-other tenant virtual machine. . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.5 Latency comparisons against all tested platforms measured using the pingutility during a MAC flooding attack with a Cisco 2950 hardware switchused as a control. The attack was launched at ICMP request 61 and termi-nated at ICMP request 241. . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.6 Latency comparisons against all tested platforms measured using the pingutility during a MAC flooding attack with a Cisco 2950 hardware switchused as a control. The attack was launched at ICMP request 61 and termi-nated at ICMP request 241. . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.7 Box and whisker plot showing latency variations for each environmentwhile being subjected to the MAC flooding attack. . . . . . . . . . . . . . . 40

5.1 Duplicate addressing within a broadcast domain due to the presence of arogue DHCP server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.2 Presence of a poisoned DNS server on a network whose address is providedto clients associated with a rogue DHCP server. . . . . . . . . . . . . . . . 44

5.3 Malicious virtual machine configured as a router on a network whose ad-dress is provided to clients as a default gateway when associated with arogue DHCP server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.4 Malicious DHCP lease process leveraging ShellShock to issue the rm -rf /command using option 100. . . . . . . . . . . . . . . . . . . . . . . . . . . 47

xv

6.1 Attacker connects to free switch port and sends out DTP packet in order toestablish a trunk link with the switch. . . . . . . . . . . . . . . . . . . . . 53

6.2 A trunk link is formed between the switch and the attackers system. . . . . 53

6.3 The attacker can now send and receive traffic on all VLANs associated withthe trunk link. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6.4 Switch spoofing control scenario using a physical Kali 2.0 system to per-form a DTP attack on a physical Cisco 2950 switch in order to gain unau-thorized access to virtual machines on restricted VLANs. . . . . . . . . . . 55

6.5 Switch spoofing scenario where the attack is generated from a virtual ma-chine connected to a virtual switched environment that has a physical up-link to a Cisco 2950 switch in order to gain unauthorized access to othervirtual machines located on restricted VLANs within other hypervisor en-vironments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

6.6 Successful switch spoofing attack with Yersinia. . . . . . . . . . . . . . . . 57

6.7 sh int status output indicating the successful conversion of the port to trunkmode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

6.8 sh int trunk output indicating the port is now set to trunking mode with802.1q encapsulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

7.1 Comparison of a standard 802.3 Ethernet frame with frames containing802.1Q single and double VLAN tags. . . . . . . . . . . . . . . . . . . . 62

7.2 Network topology with two switches connected via a trunk link offeringaccess to VLANs 1,2, and 3. . . . . . . . . . . . . . . . . . . . . . . . . . 62

7.3 Double-tagged frame is sent on the network and reaches its destination onthe second VLAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

xvi

7.4 Double tagging scenario where the attack is generated from a physical Kali2.0 system connected to a Cisco 2950 switch with a second Cisco 2950switch located in between the first Cisco switch and the connected hyper-visor environments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

7.5 Double tagging scenario where the attack is generated from a physical Kali2.0 system connected to a Cisco 2950 switch in order to gain unauthorizedaccess to virtual machines located on restricted VLANs within connectedhypervisor environments. . . . . . . . . . . . . . . . . . . . . . . . . . . 64

7.6 Double tagging scenario where the attack is generated from a Kali 2.0 vir-tual machine within one of the connected virtual networks. A physicalCisco 2950 switch acts as the physical connectivity device located betweeneach of the connected virtual networks. . . . . . . . . . . . . . . . . . . . 64

8.1 ARP poisoning Man-in-the-Middle attack scenario diagram. . . . . . . . . 71

xvii

CHAPTER 1

BACKGROUND AND RELATED WORK

1.1 Introduction

With the growing popularity of Internet-based cloud service providers, many businesses

are turning to these services to host their mission critical data and applications. Cloud cus-

tomers often deploy virtual machines to shared, remote, physical computing resources.

Virtual machines running in cloud capacity are connected to the physical network via a vir-

tualized network within the host environment. Typically, virtualized hosting environments

will utilize either a bridged network interface or a virtualized switch such as Open vSwitch

(Pettit et al., 2010; Pfaff et al., 2009) for Xen and KVM based environments, or either

the standard built-in virtual switch or the Cisco Nexus 1000V (Cisco Systems Inc., 2014b)

series virtual switch for VMware vSphere and Microsoft Hyper-V environments. These

virtual switches are designed to emulate their physical counterparts, however, the majority

of them do not provide any of the Layer 2 protection mechanisms found in modern en-

terprise grade hardware switches. It is important for users of multi-tenant cloud services

to understand how secure their network traffic is from other users of the same cloud ser-

vices, especially given that virtual machines from many customers share the same physical

resources. If another tenant can launch a Layer 2 network attack and capture all the net-

work traffic flowing from and to their virtual machines, this poses a substantial security

risk. By understanding which virtual switches are vulnerable to which attacks, users can

1

evaluate the workloads they run in the cloud, consider additional security mechanisms such

as increased encryption and/or increased monitoring and detection of Layer 2 attacks.

In this research, I present the results of a systematic study to evaluate the effects of

MAC flooding, DHCP, VLAN hopping, and ARP poisoning attacks across five major hy-

pervisor environments with seven different virtual network configurations. First, I provide

some background information on the general network configuration options available to

virtualized environments. I then introduce the test environment that was put together to

perform the experiments, as well as discuss upgrades that were made to the environment

as the research progressed. Detailed descriptions of the attack methodology used for each

of the Layer 2 networking attack scenarios is presented along with the results of each at-

tack scenario against the seven virtual network configurations. I also discuss mitigation

strategies that could help to prevent each of the attacks from being successful.

1.2 Related Work

The concept of virtualization has been the subject of many academic publications go-

ing back as early as 1997 when the idea of running multiple commodity operating systems

simultaneously on a single multiprocessor system was first presented with Disco (Bugnion

et al., 1997). In 2003 the Xen hypervisor was introduced (Barham et al., 2003) which

proved to be a major contribution in the world of computing that facilitated the widespread

adoption of virtualization technologies in data centers across the globe. Since then mul-

tiple hypervisor and cloud computing platforms have been developed, and many papers

have been published evaluating either the performance or security of these environments.

In terms of virtualization security many of the papers currently in publication focus on at-

tempting to steal information from co-located clients by making use of side-channel attacks

2

(Zhang et al., 2012; Ristenpart et al., 2009) in order to extract information from the target

system located on the same physical host. Varadarajan et al. (2012) describe a class of at-

tacks which they refer to as resource freeing attacks which are able to modify the workload

of a co-located target virtual machine in order to free up resources for the attackers virtual

machines to use. There have also been publications that provide details on methods that

could be used in order to mitigate attacks that take advantage of vulnerabilities associated

with sharing computing resources in a multi-tenant environment. Two papers in particular

have introduced the idea of splitting up the hypervisor environment into multiple compo-

nents instead of having a single virtual machine manager that controls everything. Zhang

et al. (2011) propose the idea of adding a security monitor between the management do-

main and the user virtual machines. This monitor provides an additional layer of security

and isolation when handling resource allocation and sharing (CPU, memory, storage, etc.)

between client virtual machines. Colp et al. (2011) take the idea of breaking up the hyper-

visor further with the introduction of service virtual machines that separate the traditional

virtual machine manager tasks into multiple isolated virtual machines, each performing a

specific function.

There has already been a substantial amount of work studying the vulnerability of phys-

ical networks to Layer 2 attacks (Cisco Systems Inc., 2002; Rouiller, 2015; Yeung et al.,

2008; Altunbasak et al., 2005; Lauerman and King, 2010), but the impact on virtual net-

works has not received as much attention. This is beneficial in the fact that published

research previously performed on physical networks can serve as a model for testing in

virtual environments and comparisons can be made based upon the physical baselines. For

instance, Yeung et al. (2008) provide an overview of the most popular Layer 2 network-

ing attacks as well as descriptions of the tools used to perform them. This work was very

helpful in identifying possible attack vectors that could be emulated within a virtualized en-

vironment. Altunbasak et al. (2005) also describe various attacks that can be performed on

3

local and metropolitan area networks, as well as the authors idea of adding a security tag

to the Ethernet frame for additional protection. Cisco also published a white paper (Cisco

Systems Inc., 2002) regarding VLAN security in their Catalyst series of switches. The pa-

per discloses testing that was performed on the switches in August of 2002 by an outside

security research firm @stake which was acquired by Symantec in 2004. In the white pa-

per, they discussed many of the same attacks that were mentioned by Yeung et al. (2008),

however the authors also went into detail about best practices and mitigation techniques

that could be implemented on the physical switches in order to prevent the attacks from

being successful. The SANS Institute also published a paper entitled Virtual LAN Security:

weaknesses and countermeasure (Rouiller, 2015) which provided detailed information on

the different types of attacks that target virtual LANs and their effectiveness against physi-

cal switches. The paper is heavily based on work that was presented by a Cisco employee

at the Blackhat conference in 2002 entitled Hacking Layer 2: Fun with Ethernet Switches

(Convery, 2002), as well as the results provided by @stake in the Cisco white paper. Both

the SANS Institute paper and the Blackhat briefing provide an abundance of information on

the inner workings of common Layer 2 networking attacks that affect VLAN security such

as switch spoofing, double-tagging, and MAC address flooding as well as mitigation tactics

that can be put in place to prevent them from being successful within physical networks.

The documents served as valuable points of reference during this research, especially when

setting up and conducting similar experiments against multiple different enterprise grade

virtual switching devices.

4

CHAPTER 2

LAYER 2 NETWORKING

2.1 Introduction

The main purpose of a computer network is to allow multiple devices to efficiently

communicate with each other over some form of transmission media. The establishment of

communication channels and transmission of data over these networks typically happens

at blinding speeds. This makes it very easy to take for granted the many processes and ser-

vices that are working behind the scenes to facilitate the efficient transaction of information

between two systems on a communications network. These processes have been general-

ized and represented in the Open Systems Interconnection (OSI) model (Figure 2.1), which

is a theoretical model that illustrates what happens when two devices communicate on a

computer network (Dean, 2013).

The OSI model breaks up the processes involved in node-to-node communication into

seven different layers; Application, Presentation, Session, Transport, Network, Data Link,

and Physical. Each of the seven layers consists of a set of protocols that perform a specific

set of functions related to the transmission of data at that particular point in the commu-

nication stream. When a system transmits data to another system on a network the com-

munication path begins at the top of the OSI model at the Application layer on the sending

system, flows down through the other six layers in order, leaves the computers network

5

Figure 2.1: The Open Systems Interconnect (OSI) model. The solid line indicates the data pathfrom the transmitter to the receiver, the dotted lines represent the logical connectivity between eachof the seven layers on the sending and receiving sides.

interface at the Physical layer, and is then sent across the network media to the intended

destination. The receiving system intercepts the transmission at the Physical layer and then

sends it up through the next six layers in order, from bottom to top, until it reaches the Ap-

plication layer where protocols exist that interact with the programs that are expecting the

data. As the data flows down through the OSI model on the transmitting system informa-

tion is added at each layer in the form of headers and sometimes trailers. This information

is meant to be interpreted on the receiving system at the corresponding layer in the OSI

model. As the data moves down through the sending system this extra information is en-

capsulated at each new layer until the data and all of the corresponding headers become a

packet at the network layer. This is where the logical network information is added to the

data which consists of the source and destination IP addresses. The next step in the process

6

is the Data Link layer (Layer 2), which is where the focus of this research lies. At this layer

the packet is now encapsulated into a frame that is specific to the physical media that the

information will be transmitted across. The frame header contains the physical addressing

information for the sending and receiving systems in the form of a Media Access Control

(MAC) address. This address consists of a unique 48-bit hexadecimal number that is burned

into the network card at manufacture time. Once the frame is created it is sent down to the

Physical layer and transmitted across the network in the form of light pulses, radio waves,

or electrical signals. When the data reaches the receiver the frame is de-encapsulated at the

data link layer and pushed up through the rest of the layers which further de-encapsulate

the packet in order to read the header information that is pertinent to that specific layer.

This process repeats until the Application layer is reached and the data is passed on to the

requesting program.

Since the focus of this research work is on vulnerabilities in Data Link layer protocols

and devices, the other six layers of the OSI model are considered out of scope and will not

be discussed in detail. However it is important to recognize that each of these layers consist

of sets of protocols with their own vulnerabilities that are worth exploring against virtual

networks in future research endeavors. The rest of this chapter will provide background

information and details on common Layer 2 networking concepts, devices, protocols, and

attacks.

7

2.2 Collision Domain vs. Broadcast Domain

Layer 2 networking relies on transmitting and receiving frames between inter-networked

systems using the transmitting and receiving nodes physical MAC address for identifica-

tion. Unlike the logical addressing that occurs at the Network layer (Layer 3), MAC ad-

dresses are not route-able resulting in the inability for Layer 2 communications to leave the

Local Area Network (LAN). A LAN typically consists of a series of switches and connected

devices that make up a broadcast domain in which every connected device can communi-

cate with every other device via broadcast transmissions. A broadcast transmission is when

a system sends a frame on the network that is received by every other connected device that

is online. Broadcast domains can be separated by routing devices at Layer 3, which require

the MAC address of the system to be translated to a logical IP address so that the packet

can cross the boundary of the broadcast domain, and be passed on to a different broadcast

domain connected to another interface on the routing device. This process includes the

de-encapsulation of the frame at Layer 2 so that the router can examine the packet header

at Layer 3 and determine the destination. Then the router re-encapsulates the packet into a

new frame that is specific to the physical network media that it is being forwarded to.

Layer 2 networking also allows for unicast transmissions. In this case a system trans-

mits frames directly to the MAC address of the target destination instead of broadcasting it

to the entire network. Layer 2 switches allow for the separation and isolation of this traffic

by making each physical port a separate collision domain. A collision domain is an area on

a network in which all transmissions can be intercepted by each device within the collision

domain. This includes broadcast and unicast transmissions. Prior to the use of switches,

Physical layer devices called hubs were used in order to connect multiple systems on a

network. These devices provided no isolation between connected devices, and the entire

8

hub was considered as a collision domain. This allowed administrators along with mali-

cious and curious users to easily eavesdrop on all of the traffic passing through the device.

The implementation of Layer 2 switches instead of hubs corrected this problem with the

separation of the physical ports into their own collision domains. This provides a level of

protection for unicast traffic against eavesdropping or tampering, however in Chapter 4 I

demonstrate how this isolation can be broken due to a flaw in the design of most physical

switches.

2.3 Common Layer 2 Networking Protocols

2.3.1 IEEE 802.3 Ethernet

The majority of wired networks in use today are using the IEEE 802.3 Ethernet standard

(LAN MAN Standards Committee, 2012) which provides specifications for the transmis-

sion of data across twisted pair copper and fiber optic cable. Part of this standard is the

specification for the 802.3 Ethernet frame (Figure 2.2), which defines the structure that is

used to carry the encapsulated data across the network media. The 802.3 Ethernet frame

consists of a header that provides information about the frame such as where the transmis-

sion originated from as well as its intended destination. The frame also contains a trailer

after the encapsulated packet portion which consists of a Frame Check Sequence (FCS)

that is used to verify the frame integrity.

Without a standardization of the container that carries the data across the network,

systems would not be able to effectively communicate with each other. Two nodes on a

network have to be able to speak the same language in order to setup a communication

channel, and this is the reason why specific protocols and standards exist in networking.

9

Figure 2.2: An 802.3 Ethernet Frame.

This especially comes into play when manufacturing devices such as switches and net-

working cards that are to be used on Ethernet networks. They must be built around the

802.3 Ethernet standard which defines the specifications and guidelines as to how these

devices are expected to perform. The 802.3 Ethernet frame is designed in such a way that

it provides the information that Layer 2 networking devices require in order to send net-

work traffic to the correct destination efficiently. For example, Layer 2 switches typically

support one of two methods when forwarding a frame; cut-through and store and forward.

In cut-through mode the switch only processes the preamble and destination fields of the

frame, which are the first two fields. Once it reads the destination MAC address it immedi-

ately forwards the frame on without delay. In store and forward mode the switch processes

the entire frame and uses the Frame Check Sequence (FCS) information to verify the in-

tegrity of the frame. If it finds a fault in the frame it will ask the transmitting node to

resend the same frame again, otherwise it will forward it on to the destination. Both of

these methods of switching are compatible with the same frame type, however one is more

efficient at sending data across the network and the other is more reliable. The point here is

that switches that support either mode can be mixed on a Ethernet network and the frames

will be correctly forwarded because both modes of switching comply with the IEEE 802.3

Ethernet standard.

10

2.3.2 Address Resolution Protocol (ARP)

The Address Resolution Protocol (ARP) is a Layer 2 networking protocol that is used to

map the physical MAC addresses of connected devices within a broadcast domain to their

logical Layer 3 IP addresses. Each device on the network maintains an ARP cache which

is a table that is dynamically updated when a device discovers other devices located within

the same Layer 2 network. When a system is initially placed on a network, the ARP cache

is empty and is filled with new entries as the system begins to communicate with other

systems, either directly or via broadcast transmissions. Typically, the first entry added to

the ARP cache is the default gateway for the network. The process of updating the ARP

table is rather simple. If a system on a network does not know the physical MAC address

of another system within the broadcast domain, it will send out a broadcast transmission to

every connected device asking who has that specific Layer 3 IP address. Once the system

that is assigned the target Layer 3 IP address receives the Layer 2 ARP broadcast, it sends

a unicast reply back to the requesting system with its physical Layer 2 MAC address. The

requesting system then updates its ARP cache so that it does not need to send out the

broadcast request again when it needs to establish future connections to that particular

system. This process can be easily filtered and viewed in open source network traffic

monitoring programs such as tcpdump (Tcpdump Project, 2016) or Wireshark (Combs,

2016). The following example is filtered ARP traffic output from the tcpdump program

showing the ARP request broadcast and unicast reply process.

ARP, Ethernet (len 6),

IPv4 (len 4), Request who-has vader (ec:b1:d7:55:04:51) tell 10.42.0.10, length 46

ARP, Ethernet (len 6),

IPv4 (len 4), Reply vader is-at (ec:b1:d7:55:04:51), length 28

11

2.3.3 Spanning Tree Protocol (STP)

The Spanning Tree Protocol (STP) is a Layer 2 network protocol defined in the IEEE

802.1D standard (LAN MAN Standards Committee, 2004) for local and metropolitan area

networks to prevent loops from occurring within a Layer 2 network topology. The standard

also allows for redundant links to be created between switching and bridging devices on

a network that act as a fail-over if the main trunk link is down. When a spanning tree is

created within a broadcast domain there are typically multiple paths that are established

between any two nodes on a network. The Spanning Tree Protocol organizes the network

topology into a tree that reduces the amount of redundant paths between any two nodes

to a single active path. Other paths may still physically exist within the topology, but the

protocol blocks them until they are needed for fail-over or load balancing purposes. In

order to create the tree a root bridge is selected that acts as the root node of the tree. The

root bridge is identified as the device on the network with the smallest bridge ID and the

highest priority. The protocol attempts to calculate the lowest cost paths between the root

bridge and each network segment, as well as between each network segment, and uses the

interfaces connected at each end of those paths as the primary links. When a device or

link goes down the protocol automatically diverts the traffic to a higher cost path until the

original path is functional again.

STP uses Bridge Protocol Data Units (BDPU) as a method to exchange information

about the bridge IDs and path costs within the network. By default BDPU frames are

broadcast across the network every two seconds, and can be used to configure the spanning

tree as well as make changes to the network topology. When a switch port is configured

to use STP and accept BDPU frames it can be in one of five different states; blocking,

listening, learning, forwarding, and disabled. If the state is set to disabled then the port is

not participating in STP, in which case it ignores any BDPU frames that are sent to it. The

12

blocking state is similar to the disabled state however the port is still listening and receiving

the periodic broadcast of BDPU frames, however it cannot be used to pass traffic until the

block is released. A blocked port may be put into the forwarding state if another link path

fails and a message is sent in a BDPU frame that signals it to start up as a fail-over link due

to a failure in the lower cost path. Ports are considered to be in the forwarding state if they

are sending and receiving data as normal. When in the forwarding state the port still listens

to the BDPU broadcasts and adjusts itself as necessary depending on if there is a topology

change or not. The listening and learning states are intermediate states in which the port

listens for BDPU frames and populates its MAC address table. No frames are forwarded in

either the listening or learning states until the port is put into the forwarding state, in which

case it is clear to start transmitting frames across the network.

2.3.4 Virtual Local Area Networks (VLAN)

The IEEE 802.1Q standard (LAN MAN Standards Committee, 2005) provides support

for separating a physical network into multiple logical networks by allowing the creation of

multiple Virtual Local Area Networks (VLANs) within a single Layer 2 switching device.

This allows an administrator to use a set of physically connected switches to create multiple

(virtual) networks that span the same network hardware and media. Each of the separate

VLANs are considered as an isolated broadcast domain, therefore a Layer 3 routing device

has to be put into place if traffic needs to flow between them. In order to maintain isolation

between multiple VLANs running over the same switch, the 802.3 Ethernet frame supports

the addition of a 32-bit 802.1Q VLAN tag (Figure 2.3) which identifies the specific VLAN

that the frame belongs to in the tags VLAN ID (VID) field.

13

Figure 2.3: An 802.3 Ethernet Frame with an Added 802.1Q VLAN Tag.

When setting up a Layer 2 switching device to support 802.1Q VLANs there are two

types of port configurations to consider; trunk and access. A trunk port is typically used to

connect switches together in order to allow traffic from multiple VLANs to travel between

the devices over the same backbone channel. Access ports are those that are configured

for client access to the network, and more specifically to a certain VLAN. When a client

connects to an access port the connection to the virtual LAN is transparent. The client does

not require any special configuration to support being on the VLAN because the switch

automatically adds the VLAN tag to every frame that is sent over the port. The combi-

nation of trunk and access ports allow administrators to place clients located on separate

switches within the same VLAN. This provides flexibility when initially designing the net-

work topology, as well as when it comes time to scale it up in the future.

By default switches that support the 802.1Q protocol are configured to use a native

VLAN, which is usually VLAN 1 on the switch. The native VLAN allows for the passing

of untagged traffic and is typically configured to be used for management purposes only.

Every other VLAN that is configured on the switch is considered as an access VLAN. Ac-

cess VLANs require frames to have the proper VLAN ID in their 802.1Q tag in order to

14

be transmitted within the logical broadcast domain. If the 802.1Q tag does not exist or the

VLAN ID is incorrect the frame is dropped. By separating traffic into multiple VLANs,

and more specifically separate isolated broadcast domains, administrators are able to re-

strict access to network resources, quickly isolate problems, and increase overall network

performance.

2.3.5 Dynamic Host Configuration Protocol (DHCP)

The Dynamic Host Configuration Protocol (DHCP) is an Application layer (Layer 7)

protocol that relies heavily on Layer 2 broadcast messages in order to provide dynamic

Layer 3 addressing services within a single broadcast domain. DHCP simplifies the alloca-

tion of IP addresses and required network information to clients by completely automating

the process. This prevents errors that typically occur when manually assigning addresses to

network nodes which can be caused by typographical errors, omissions of required network

information, or by inputing the same address on multiple clients causing address conflicts.

The DHCP service can be configured to provision IP addresses from an IP address pool

that is reserved for this process. Clients that request an address from the DHCP server

are allocated one from the pool and are also provided with the proper subnet mask for

the network, the address of the default gateway, as well as Domain Name Service (DNS)

server information for name resolution. The Dynamic Host Configuration Protocol also

can be configured to push a wide range of other options to clients which are defined by the

Internet Engineering Task Force (1997) in RFC 2132. Currently the DHCP specification

supports up to two hundred and fifty five (eight bits) different options, many of which are

non-standard, application specific, or proprietary extensions.

15

Figure 2.4 illustrates the interaction that occurs when a client requests an IP address

from a DHCP server within the same broadcast domain. First the client sends out a broad-

cast to the network requesting a DHCP server to respond with an address lease offer. Once

a DHCP server on the network intercepts the client request broadcast it checks its pool for

an available address, then makes an offer to the client in the form of a unicast response to

the clients Layer 2 MAC address. The client then replies with an acknowledgment, and

accepts the IP address lease. The server then responds with the lease information consisting

of; the lease duration, client IP address, subnet mask, default gateway, and DNS informa-

tion for the network as well as any other options that it is configured to push. Once the

client receives this information it uses it to automatically configure its network interface

card appropriately to communicate with other nodes on the network.

Figure 2.4: Dynamic Host Configuration Protocol process.

2.4 Virtual Layer 2 Networking Options

There are two types of networking configurations that are typically used in virtualized

environments: bridging and switching. In this section we describe both options and discuss

how each one is applied within a virtualized network.

16

2.4.1 Bridging

Bridged mode is the simplest configuration providing an interface dedicated to virtual

machine use. A bridge connects two or more network segments at Layer 2 in order to extend

a broadcast domain and separate each of the segments into their own individual collision

domains (Seifert and Edwards, 2008). A forwarding table as described by (LAN MAN

Standards Committee, 2004; Seifert and Edwards, 2008) is used to list the MAC addresses

associated with devices located on each network segment connected to the bridge (Figure

2.5). Requests are forwarded based upon contents of this table and the destination MAC

address located in the Ethernet frame. A frame is forwarded across the bridge only if the

MAC address in the destination block of the frame is reachable from a different segment

attached to the bridge. Otherwise, the frame is directed to a destination address located on

the same segment as the transmitting device or dropped.

Figure 2.5: A basic bridge using a forwarding table to pass requests between two network segments.

17

In virtualized environments, guest machines utilize user-space virtual network inter-

faces that simulate a Layer 2 network device in order to connect to a virtual bridge. Typi-

cally, the virtual bridge is configured and bound to a physical interface on the host machine

that is dedicated solely to virtual machine traffic.

2.4.2 Switching

Physical switches have the capability of operating at Layer 2 or higher of the OSI

model. Switches can be thought of as multi-port bridges (Seifert and Edwards, 2008) where

each port of the switch is considered as its own isolated collision domain. Instead of a for-

warding table, switches employ a CAM (content addressable memory) table as described

by (Seifert and Edwards, 2008). Content addressable memory is specialized memory hard-

ware located within a switch that allows for the retention of a dynamic table or buffer that

is used to map MAC addresses of devices to the ports they are connected to (Figure 2.6).

When a packet arrives on a port from a new device, an entry mapping the device to

the port is added to the table. This allows a switch to intelligently send traffic directly to

any connected device without broadcasting frames to every port on the switch. The switch

reads the frame header (Figure 2.7) for the destination MAC address of the target device,

matches the address against its CAM table, then forwards the frame to the correct device.

If a MAC address is not found in the CAM table, a packet destined for it will be sent to all

interfaces.

The use of a CAM table and the separation of collision domains are key factors in pre-

venting eavesdropping of network traffic between devices connected to the switch. How-

ever, a physical switch is an embedded device and has a finite amount of memory available

18

Figure 2.6: A switch and its CAM table.

Figure 2.7: An 802.3 Ethernet Frame. Switches just need to match the destination address from theframe to an entry in its CAM table in order to forward it on to the correct port.

to its CAM table; once this memory is full, the switch must discard existing entries in order

to add new entries. The majority of physical switches in use today employ CAM chips

19

that are capable of holding up to 32,000 addresses (Seifert and Edwards, 2008) which can

easily be saturated by a single MAC flooding attack in a very short amount of time.

Virtual switches emulate their physical counterparts and are capable of providing fea-

tures such as VLAN traffic separation, performance and traffic monitoring, as well as qual-

ity of service (QoS) solutions. Virtual machines are connected to a virtual switch by the

way of virtual network interfaces (VIF) that are similar to the Layer 2 network devices used

in conjunction with virtual bridges.

2.5 Layer 2 Network Attacks

2.5.1 Media Access Control (MAC) Attacks

The most popular attack in this category is the MAC flooding attack. In this attack,

a switch is flooded with numerous random MAC addresses in order to fill up the content

addressable memory (CAM) buffer within the switch forcing it into a fail safe mode, other-

wise known as hub mode. When a switch is operating in hub mode, the inherent separation

of collision domains is broken and all frames passing through the switch are forwarded to

all connected devices. This allows for passive eavesdropping of all traffic passing through

the device. MAC flooding can be mitigated by enforcing port security on physical switches

which imposes a limit on the amount of MAC addresses that can send traffic to a specific

port (Cisco Systems Inc., 2015a). This feature is not implemented within the majority of

the virtual switches available today rendering them vulnerable to MAC flooding attacks.

20

2.5.2 Dynamic Host Configuration Protocol (DHCP) Attacks

The Dynamic Host Configuration Protocol (DHCP) is actually an Application layer

protocol that relies on the Layer 2 MAC address and the process of broadcasting to provide

IP addresses to clients located within the same broadcast domain. In order to perform

a Layer 2 DHCP attack an attacker must place a rogue DHCP server on a network in

hopes that clients in the broadcast domain associate with it rather than the legitimate DHCP

server. Once a client receives an IP address lease from a malicious DHCP server under an

attackers control, that client could also be seeded with the IP address of a poisoned DNS

server, an incorrect default gateway, or be forced to run malicious code. This type of

attack could also cause DoS situations where duplicate addressing occurs on the network

causing the resources bound to those addresses to be inaccessible. These attacks can be

mitigated by enforcing static addressing, or by using utilities to monitor the network and

alert administrators when this type of activity is encountered.

2.5.3 Virtual Local Area Network (VLAN) Hopping

VLAN hopping is the term used to describe an attack where an Ethernet frame is mod-

ified to force traffic to traverse (hop) VLANs in order to allow an attacker to gain unautho-

rized access to restricted portions of a network. More specifically, an attacker modifies the

VLAN tag embedded in the frame to contain the ID of the target VLAN on the network.

In order to perform this type of attack against a switch, the attacker must be connected to

a port on the device that is a member of the native VLAN, or one that is setup as a trunk

port. When the modified frame is passed through the switch, the VLAN tag is read and

the switch places it within the correct broadcast domain associated with that VLAN ID. If

successful, the attacker can then proceed to intercept traffic on the target VLAN or perform

21

other malicious activities such as DoS attacks against services running on that particular

portion of the network. VLAN hopping can be mitigated by enforcing strict VLAN con-

figuration of physical switch ports to prevent unauthorized access to the default VLAN or

trunk ports.

2.5.4 Address Resolution Protocol (ARP) Spoofing

ARP spoofing is a technique used by attackers that allows for the hijacking of traffic

destined for another host. Specifically, by using ARP spoofing an attacker can perform a

man-in-the-middle (MITM) attack that allows for eavesdropping between two hosts on a

network. An attacker can also broadcast fake ARP messages to a network to redirect traffic

or create DoS situations. ARP spoofing attacks can be mitigated by using static ARP entries

for all hosts on a network, however this is a cumbersome task and is typically avoided

unless absolutely necessary. More often that not savvy administrators will employ the

use of network monitoring utilities to watch for suspicious behavior and react accordingly

based upon the alerts.

2.5.5 Spanning Tree Protocol (STP) Attacks

In this type of attack, incorrect BPDU frames are sent to switches in order to modify

the spanning tree topology implemented on the network. By exploiting STP, an attacker

could perform a man-in-the-middle (MITM) attack allowing eavesdropping on traffic pass-

ing between two nodes on a network (Lauerman and King, 2010). The attacker could also

cause a denial of service (DoS) situation rendering services unavailable to legitimate users.

22

STP based attacks can be mitigated on Cisco Catalyst switches by enabling either the

Cisco Catalyst BPDU Guard or Root Guard feature on the device (Lauerman and King,

2010). By enabling BPDU guard on switch ports administrators are able to control the

STP domain boarders and prevent changes to the topology (Cisco Systems Inc., 2015c).

BPDU guard is enabled Cisco switches by configuring certain ports on the switch to use

STP PortFast mode. When a port is configured in STP PortFast mode devices connected

to it are not allowed to influence the STP topology (Cisco Systems Inc., 2015c). If the port

receives a BPDU while in PortFast mode it is put into a disabled state and a log entry is

made alerting the administrator that the port received a BPDU attempting to make changes

to the STP topology. Root Guard (Cisco Systems Inc., 2005) is another feature that Cisco

implemented in their modern switch lines that offers some additional protection against

STP topology attacks. When Root Guard is enabled on a switch port it blocks the port from

becoming the new STP root node of the tree. The STP topology can be augmented with

BPDU frames that specify when the root node should be changed to a different switch,

which also causes a recalculation in the tree topology and pathing. Root Guard monitors

the port for BPDU frames that are attempting to change the topology, and when one is

received on the protected port it shuts it down and creates an alert in the logs indicating

that there was an attempted STP root node change. Seeing as these are proprietary features

available only on Cisco Catalyst switches they offer no help in preventing these attacks

from being successful within a non-Cisco network that utilizes STP.

Virtual switches are currently unaffected by Spanning Tree Protocol attacks due to the

fact that the switches do not support STP and ignore any BDPU frames that they receive.

In fact some of the major hypervisor vendors have provided documentation that actually

informs administrators to disable STP on ports that their products are connected to by en-

abling PortFast mode on the interface (Citrix, 2016; VMWare, 2013). This helps to prevent

network service interruption to the virtual machines hosted within the hypervisor platform

23

when changes occur in the STP topology that cause a convergence. The STP convergence

process forces all switches in the STP domain to dump their forwarding tables and relearn

the entire STP topology including all related MAC addresses. This process could take up

to one minute before connectivity is restored (VMWare, 2013) causing downtime to each

of the devices connected to the network. If best practices are followed and switch ports that

are connected to virtual switches are properly configured in PortFast mode, this downtime

could be mitigated. This is especially important for large data centers that rely on virtu-

alization for their entire infrastructure. If STP is enabled on the network then this could

cause serious concerns for clients that have service level agreements (SLAs) in place where

there is an expected level of uptime that should be consistently maintained.

24

CHAPTER 3

RESEARCH ENVIRONMENT

3.1 Original Environment

The original research environment consisted of seven server class systems all located

on a test network that was isolated from local production networks to avoid impacting

them. I deployed an optimized installation of Gentoo Linux and the Xen 4.3 hypervisor

to three Dell PowerEdge 860 servers each equipped with a dual core Intel Xeon 3050

2.13GHz processor, 4 GB of memory, and a 500 GB hard drive. Each system contained dual

Broadcom NetXtreme BCM5721 Gigabit Ethernet PCI Express network interface cards

integrated into the motherboard. The first network interface was dedicated to the privileged

control domain on each server for administrative functions, and the second configured to be

utilized by guest virtual machines. Each servers 500 GB hard disk was divided into four

partitions; a 100MB ext3 /boot, a 10GB ext3 /, a 2GB swap, with the remainder allocated

to Logical Volume Management (LVM) storage for virtual machine deployment.

Four additional servers were configured with enterprise level hypervisor solutions: Cit-

rix XenServer 6.2, Microsoft Windows Server 2008 R2 with the Hyper-V hypervisor, Mi-

crosoft Hyper-V 2008 (free edition), and VMware vSphere (ESXi) 5.5 (free edition). The

hardware utilized for the Citrix XenServer 6.2 system was identical to the three Gentoo

25

systems, however the Microsoft Hyper-V and the VMware vSphere hypervisors were con-

figured on systems with different hardware configurations due to a lack of additional Dell

PowerEdge 860 systems. Both Microsoft Windows Server 2008 R2 along with the Hyper-

V hypervisor as well as the free version of Hyper-V 2008 were installed to identical Dell

PowerEdge 2950 server systems containing dual quad core Intel Xeon 5140 processors at

2.33GHz, 32GB of memory, and a 145GB SATA hard drive. VMware vSphere (ESXi)

5.5 (free edition) was deployed to a custom built server using a Supermicro X9SCL server

motherboard, a quad core Intel Xeon E3-1240 processor at 3.30GHz, 24GB of memory, and

a 500GB SATA hard drive. The Hyper-V and vShpere systems were each outfitted with two

network adapters in order to provide separate dedicated interfaces for administrative pur-

poses and virtual machine use. Though there are notably some variations in the hardware

configurations summarized in Table 3.1, it is important to note that these differences had

no impact on the results of the experiments that were performed.

Table 3.1: Summary of the orignial test environment hardware.

Hardware Specs

CPU Memory Hard NICsPlatform Type Size Disk

OS Xen w/ Linux Bridging Xeon 3040 4 GB 500 GB 2

OS Xen w/ Open vSwitch 1.11.0 Xeon 3040 4 GB 500 GB 2

OS Xen w/ Open vSwitch 2.0.0 Xeon 3040 4 GB 500 GB 2

Citrix XenServer 6.2 Xeon 3040 4 GB 500 GB 2

MS Server 2008 R2 w/Hyper-V Xeon 5140 32 GB 145 GB 2

MS Hyper-V 2008 Free Xeon 5140 32 GB 145 GB 2

VMware vSphere (ESXi) 5.5 Xeon E3-1240 24 GB 500 GB 2

For the MAC flooding scenario, two virtual machines were deployed to each virtualiza-

tion platform: one malicious virtual machine attempting to eavesdrop on the traffic of any

other tenant virtual machines and one victim virtual machine (Figure 3.1). The Kali Linux

26

security distribution was selected due to the plethora of network security auditing tools

that come pre-installed and configured. Two complete installations of Kali were installed

to each server on 20GB LVM partitions as hardware virtual machine (HVM) guests. The

systems were then allocated static IP addresses that positioned them on the same isolated

subnet as the servers and were completely updated.

Figure 3.1: A malicious virtual machine located on a multi-tenant virtual network.

The DHCP attack testing required a more elaborate setup. Specifically, four virtual

machines were created within each hypervisor platform. Each of these virtual machines

used a minimal installation of CentOS 6.5 and was configured for a specific purpose (Table

3.2). First, a virtual machine acting as a rogue DHCP server was setup and configured

using DNSMasq, a lightweight DHCP and DNS server. Second, a simple router using

iptables on a separate virtual machine was used to forward traffic between two broadcast

domains using NAT and two network interfaces. Third, a basic Apache web server was

setup to act as a malicious web server. Fourth, the final virtual machine was configured as a

minimal client that was left unpatched and vulnerable to shellshock (National Vulnerability

Database, 2014c).

27

Table 3.2: New virtual machines added to each hypervisor platform for Layer 2 DHCP attacktesting.

Operating Completely System VirtualSystem Updated Purpose Interfaces

CentOS 6.5 Yes DHCP/DNS Server 1

CentOS 6.5 Yes Simple Router 2

CentOS 6.5 Yes HTTP Server 1

CentOS 6.5 No Left Vulnerable to ShellShock 1

3.2 Upgraded Research Environment

In August of 2015 I acquired $30,000 worth of funding to build a new test environment

for this research. The previous systems that were used for the MAC flooding and DHCP

attack experiments were somewhat outdated and consisted of mismatched hardware which

made for an inconsistent test environment. Originally this was not a concern because of the

fact that the DHCP and MAC flooding attack experiments were not dependent on system

performance. However, it could be argued that the original MAC flooding performance

results were skewed due to the fact that some of the systems contained better processors

and more memory than others. Since acquiring the funding I have upgraded the entire test

environment to eight brand new identical SuperMicro server systems, and have installed

the latest versions of each of the hypervisor environments as of September 2015 in order

to continue the research in a uniform environment with the latest software available. Each

new rack mountable 1U SuperMicro server system consists of a quad core Intel Xeon X3-

1240V3 processor running at 3.4GHz, 32GB of memory, a 500GB Western Digital Enter-

prise 7400 RPM SATA hard drive, and four on-board Intel Gigabit Ethernet ports. Having

four Ethernet ports on each system allowed me to dedicate a port to the hypervisor oper-

ating system for management purposes, and also provided the flexibility to use the other

three ports for different virtual machine network configurations within each environment.

28

This especially became useful when conducting the VLAN hopping experiments since a

separate isolated network had to be created in order to run the attacks without affecting the

primary virtual machine test network. This isolated network was created by connecting the

third Ethernet port on each of the physical systems to a 48-port Cisco 2950 switch. There

was also an uplink from the 48-port switch to a second 24 port Cisco 2950 switch in order

to support multiple test scenarios. The initial interface and trunking configurations used for

each of the Cisco switches can be found in Appendix C.

As previously stated I upgraded all of the hypervisor environments and virtual switches

to the latest versions as of September 2015 and also added a few new configurations to the

mix. I swapped out two of the systems that were found to be redundant (Gentoo/Xen/OVS

1.11.0, and MS Hyper-V Free Edition) in favor of two new environments (Proxmox, and

MS Hyper-V with the Cisco Nexus 1000v) to add more variety since it was found that there

was no difference when testing those systems in comparison with the newer or more robust

solutions that were offered at the time. Kali 2.0 was installed to the eighth system in order

to have an independent physical test system to use in the VLAN hopping experiments.

Table 3.3 provides a list of the hypervisor environments and operating systems that were

installed to the new hardware along with the virtual switch configuration used within each

system.

29

Table 3.3: Summary of hypervisor platforms and virtual switch configurations installed to the newhardware.

Hypervisor Platform Virtual Switch

Gentoo OS Xen 4.5.1 Linux 802.1d Bridging

Gentoo OS Xen 4.5.1 Open vSwitch 2.4.0

VMWare vSphere ESXi 6.0.0 Standard ESXi Virtual Switch

MS Server 2012 R2 DataCenter w/Hyper-V Standard Hyper-V Virtual Switch

MS Server 2012 R2 DataCenter w/Hyper-V Cisco Nexus 1000v 5.2(1)SM3(1.1a)

ProxMox 3.4 (KVM) Linux 802.1d Bridging

Citrix XenServer 6.5.0 Open vSwitch 2.1.3

Kali 2.0 Standalone System No virtual switch

30

CHAPTER 4

MAC FLOODING ATTACK

4.1 Introduction

The Layer 2 Media Access Control flooding attack, also known as MAC flooding, is an

attack in which many packets are generated on the network with random MAC addresses

in an attempt to overflow the CAM buffer within a switch and thus force the switch into

a mode in which it broadcasts packets on all interfaces. This happens because the legiti-

mate MAC addresses are evicted from the CAM table in favor of the many random MAC

addresses generated by the attacker. This is referred to as hub mode and when a switch is

operating in hub mode, the inherent separation of collision domains is broken and all frames

passing through the switch are forwarded to all connected devices. This allows for passive

eavesdropping of all traffic passing through the device. MAC flooding can be mitigated by

enforcing port security on physical switches which imposes a limit on the amount of MAC

addresses that can send traffic to a specific port (Cisco Systems Inc., 2015a). This feature

is not implemented within the majority of the virtual switches available today rendering

them vulnerable to MAC flooding attacks.

31

4.2 Attack Methodology

The program macof from the dsniff package (Yeung et al., 2008) was used on a Kali

virtual machine to perform a MAC flooding attack (Figure 4.1) on the virtual network

within each test environment. This type of attack when performed on a physical switch

typically causes the CAM table on the switch to fill up forcing the device to go into a

fail safe or hub mode which in turn causes all packets on the network to be broadcast to

every node connected to the switch. Wireshark (Combs, 2016) was used to determine if

the attack was successful by monitoring the network for HTTP traffic which should not be

interceptable by other hosts on the virtual network.

Figure 4.1: A malicious virtual machine running macof to flood a virtual network with randomMAC addresses and malformed packets.

All tests were conducted in the same manner. Each server had two Kali Linux virtual

machines deployed on them and connected to the internal virtual network as illustrated in

32

Figure 4.2. During each experiment both virtual machines were brought online. On the

first virtual machine (Kali1) macof was started up using the command:

macof -i eth0

and left to run. Then Wireshark was started on the same virtual machine and an HTTP filter

was applied to only display sniffed HTTP traffic. The second Kali virtual machine (Kali2)

was then used to surf the web. If the attack proved to be successful then the HTTP traffic

from Kali2 should be viewable in Wireshark on Kali1.

Figure 4.2: Network diagram for MAC flooding attack scenarios.

4.3 Results

4.3.1 Linux 802.1d Bridged Interface

Running the attack within the bridged virtual network test environment resulted in a sig-

nificant performance degradation that impacted the usability of the tenant virtual machines,

33

essentially creating a denial of service (DoS) type of attack. This effect was observed as

a large increase in latency when attempting to interact with any of the virtual machines

on the system either through Secure Shell (SSH) or Virtual Network Computing (VNC).

While the MAC flooding attack was occurring remote connections to the virtual machines

became unstable due to the saturation of the virtual network with spoofed frames. This

effect was quantified by using the ping utility on the second virtual machine to measure

the transmission latency to a server located on the physical network while the attack was

occurring (Figure 4.3). The attack however did not result in the ability to sniff other vir-

tual machine traffic passing over the interface. This most likely comes from the fact that

the standard bridge interface is missing the CAM table that typically is found on switches

mapping known MAC addresses to switch ports, an essential element of the attack.

Figure 4.3: Latency measured using the ping utility on a bridged virtual network during a MACflooding attack. The attack was launched at ICMP request 61 and terminated at ICMP request 241.

34

4.3.2 Open vSwitch 1.11.0 Interface

When running the attack on the Open vSwitch 1.11.0 virtual network test environment

network performance degradation was also observed, and the attacking machine could also

successfully sniff traffic from another tenant machine. Figure 4.4 depicts the results of the

successful attack and provides substance to the claim that virtual switches are vulnerable

to some of the same Layer 2 attacks as physical switches.

Figure 4.4: A malicious virtual machine running macof on an Open vSwitch virtual network andsuccessfully sniffing HTTP traffic with Wireshark from another tenant virtual machine.

4.3.3 Open vSwitch 2.0.0 Interface

Running the attack on the latest version of Open vSwitch available at the time of this

research revealed that the vulnerability still existed and had not been addressed. The system

responded in the same way as the previous two attempts and the other tenants HTTP traffic

was viewable in Wireshark.

It should be noted that in February of 2015 I notified the Open vSwitch security team of

the discovery. They confirmed the vulnerability and immediately responded with a patch

35

(Pfaff et al., 2015; Pfaff, 2015a) to resolve the issue. Since then the patch has been merged

into every major branch of Open vSwitch from 2.0.0 on (Pfaff, 2015b). It is recommended

that any environment running any version of Open vSwitch prior to the patched version

of the 2.0.0 branch should be upgraded immediately, since both the vulnerability and ex-

ploitation technique have been made public.

4.3.4 Citrix XenServer 6.2

Citrix XenServer 6.2 utilizes an older version of Open vSwitch (version 1.4.6) to pro-

vide virtual switching services to its client machines. When the MAC flooding test was

attempted in the XenServer environment, it was also discovered that the flooding was able

to escape the virtual environment which caused all upstream physical switches to go into

hub mode as well. Not only did this allow the malicious virtual machine running Wireshark

to sniff traffic from other tenant virtual machines, it also was able to eavesdrop on traffic

from physical machines located within the same broadcast domain to which the physical

Ethernet adapter was connected. Since this test was performed on a university network well

over one hundred upstream switches were affected and put into hub mode which also had

the side effect of disabling all VLAN separation within the affected devices. This made

it possible to view the majority of the traffic flowing over the campus network from the

attacking virtual machine using Wireshark. After disclosing the issue to the IT staff they

promptly took action to enforce port security on all of their physical switches limiting each

port to learning at most fifty MAC addresses at a time to avoid the possibility of CAM table

overflow.

36

4.3.5 Microsoft Hyper-V Server 2008 R2

Testing under the Microsoft Hyper-V environment was performed both with and with-

out the Windows Firewall service enabled to identify if there was any effect on the results.

Both scenarios proved to be unsuccessful due to the fact that the virtual networking used

by Microsoft Windows Server 2008 R2 provides some minimal protection for virtualized

network traffic, this includes protection against MAC address spoofing (Microsoft, 2013).

Further testing was performed on the free version of Microsoft Hyper-V to see if the

protection offered by Server 2008 R2 is also built into the bare metal product. As with the

previous environment testing was performed both with and without the Windows Firewall

service enabled. It was concluded that under both conditions the free version of Microsoft

Hyper-V 2008 was also unaffected by the MAC flooding attack since it is built upon a

minimal version of Microsoft Windows Server 2008 R2 entitled Server Core. The Core

version of Microsoft Server 2008 R2 still provides the same level of network protection as

the full version, but only allows for the installation of specific server roles to the operating

system (Microsoft, 2008), in this case the Hyper-V hypervisor.

4.3.6 VMware vSphere (ESXi) 5.5 - free edition

All testing within the VMware vSphere environment was performed identically to the

previous trials for completeness. Testing was performed on the free version of ESXi using

the default virtual networking configuration. The results show that this particular config-

uration was not vulnerable to the MAC flooding attack in terms of a malicious user being

able to eavesdrop on another tenants network traffic.

37

4.3.7 Summary of MAC Flooding Results

It can clearly be seen from the results summarized in Table 4.1 that any virtualized

network environment built upon the Open vSwitch virtual switch could be vulnerable to

MAC flooding attacks, and has the potential to expose its client traffic to eavesdropping.

Therefore, if a virtual machine is transmitting sensitive information over a virtual network

that uses Open vSwitch precautions should be taken such as using encryption in order to

ensure that the information in transit remains confidential.

Table 4.1: MAC flooding attack results across seven test environments. 3indicates the platformwas affected.

Results of Attack

ImpactedEavesdropping Network

Platform Allowed Performance

OS Xen w/ Linux Bridging 3

OS Xen w/ Open vSwitch 1.11.0 3 3

OS Xen w/ Open vSwitch 2.0.0 3 3

Citrix XenServer 6.2 3 3

MS Server 2008 R2 w/Hyper-V

MS Hyper-V 2008 Free

VMware vSphere (ESXi) 5.5

Figures 4.5, 4.6, and 4.7 illustrate the effects of the MAC flooding attack on each of

the virtual environments utilizing the same ping latency test that was previously described

in Section 4.3.1 and depicted in Figure 4.3 for the bridged interface. The MAC flooding

attack performance benchmarks were conducted on the upgraded environment to ensure

consistency among the different platforms, as well as to isolate any possibility of differ-

ences in hardware affecting results. This also resulted in the performance evaluation of the

newer environments against the attack, specifically the ProxMox and Cisco Nexus 1000v

38

virtual networks were added. A control element is also provided in the following graphs

via a Cisco 2950 hardware switch which underwent the same testing. All of the exper-

iments were performed using Gigabit network interfaces to maintain uniformity in flood

rate. As can be seen by the results any environment using a bridged or Open vSwitch based

virtual network was heavily affected by the attack in terms of network performance. The

Microsoft Hyper-V virtual switch, the Cisco Nexus 1000v, and the ESXi virtual switch

all maintained consistent performance throughout the entire test which was very similar to

the performance of the Cisco 2950 hardware switch which remained unaffected during the

attack.

Figure 4.5: Latency comparisons against all tested platforms measured using the ping utility duringa MAC flooding attack with a Cisco 2950 hardware switch used as a control. The attack waslaunched at ICMP request 61 and terminated at ICMP request 241.

39

Figure 4.6: Latency comparisons against all tested platforms measured using the ping utility duringa MAC flooding attack with a Cisco 2950 hardware switch used as a control. The attack waslaunched at ICMP request 61 and terminated at ICMP request 241.

Figure 4.7: Box and whisker plot showing latency variations for each environment while beingsubjected to the MAC flooding attack.

40

4.4 Mitigation Techniques

As stated previously the traditional way to prevent these attacks on physical switches is

to utilized port security to lock down switch ports so they can only learn a limited amount

of MAC addresses to avoid CAM table overflow. It is also wise to configure port security

to limit network connectivity to authorized MAC addresses only. Since this feature is not

possible on the majority of enterprise grade virtual switches in use today other mitigation

techniques need to be investigated. The most obvious solution would be for the virtual

switch developers to start incorporating port security features into their software switches

as a default feature that is easily configurable. Another simpler solution would be to prevent

the switches from going into a hub mode state in the first place by not emulating physical

switch CAM table limits. Bare-metal hypervisor platforms are typically deployed on sys-

tems with large amounts of memory. Unlike physical switches which have a limitation on

available resources, server systems have an abundance of memory that could be allocated

to the task of providing dynamic CAM tables to the virtual switching devices that could be

configured to adapt to a flooding situation and alert an administrator if the memory usage

rises above a certain threshold. This would eliminate the possibility of information leakage

due to the virtual switch going into a hub mode state. As stated previously most modern

physical switches have a CAM table capacity limit of 32,000 MAC addresses. With each

MAC address comprising of 48 bits this would require around 187.5MB of memory in or-

der to store all of the addresses. This memory requirement is insignificant on most modern

server systems today which could easily provide a larger more dynamic buffer to the virtual

switch.

41

CHAPTER 5

DHCP ATTACKS

5.1 Introduction

Layer 2 Dynamic Host Configuration Protocol (DHCP) attacks typically consist of an

attacker placing a rogue DHCP server on a network that essentially competes with the

legitimate DHCP server when responding to client addressing requests located within the

same broadcast domain. Setting up a DHCP server on a Linux virtual machine is a fairly

simple task that only requires installing the dnsmasq or dhcpd services and making a few

minor configuration changes. The entire process can be completed in a short amount of time

allowing an attacker to quickly place a rogue DHCP server within a multi-tenant virtualized

environment. Since DHCP requests are broadcast across the network the attacks rely on

the attackers DHCP server to respond to the address request before the legitimate DHCP

server in order to be successful.

Once a client obtains an IP address lease from a malicious DHCP server, that client

could also be seeded with the IP address of a poisoned DNS server, an incorrect default

gateway, or be forced to run malicious code. This type of attack could also cause Denial

of Service (DoS) situations where duplicate addressing (Figure 5.1) occurs on the network

causing the resources bound to those addresses to be inaccessible, or allow for the execution

42

of Man-in-the-Middle attacks where traffic is first sent to an attacker and then onto the

original destination.

Figure 5.1: Duplicate addressing within a broadcast domain due to the presence of a rogue DHCPserver.

5.2 Attack Methodology

Three different attack scenarios were duplicated across each of the original seven test

environments in order to evaluate the impact of the following Layer 2 DHCP attacks; seed-

ing clients with a poisoned Domain Name Server (DNS) server, providing an invalid or

malicious default gateway, and remote execution of code.

5.2.1 Seeding Clients With a Poisoned DNS Server

When a client receives an IP address lease from a DHCP server it usually is also pushed

DNS server information for the network. If the client is seeded with the IP address for a

43

poisoned DNS server under an attackers control the client could potentially be directed to

spoofed websites and services setup by the attacker to capture private information from the

user such as user names, passwords, credit card numbers, and other personally identifiable

information (PII) that the attacker could use to further penetrate the organization.

For this scenario, the DNSMasq server was used to seed the client system with a

poisoned DNS server through DHCP. Since DNSMasq also provides DNS server func-

tionality the rogue DHCP server doubled as the poisoned DNS server that was passed

to clients receiving addresses. The DNS server was setup to direct all traffic destined to

www.gmail.com to be redirected to the malicious web server (Figure 5.2). A command

line web browser called elinks (ELinks, 2014) was then used in the client virtual machine

to visit www.gmail.com in order to observe the effect.

Figure 5.2: Presence of a poisoned DNS server on a network whose address is provided to clientsassociated with a rogue DHCP server.

44

5.2.2 Providing Clients With an Invalid or Malicious Default Gateway

If clients on a network obtain incorrect default gateway information from a rogue DHCP

server under an attackers control they will be unable to route traffic outside of the local

network creating a denial of service situation for resources on other subnets or the Internet.

An attacker can also push the IP address of a default gateway that leads the client into a

malicious honeypot network that can be setup to mirror another subnet on the network.