21c3 NOC Overview - blackwingblackwing.de › files › vortraege ›...

18
The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview Concepts, Implementation and Hardware Christian Carstensen, Sebastian Werner & The 21c3 NOC Crew

Transcript of 21c3 NOC Overview - blackwingblackwing.de › files › vortraege ›...

Page 1: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18

21c3 NOC Overview

Concepts, Implementation and Hardware

Christian Carstensen, Sebastian Werner & The 21c3 NOC Crew

Page 2: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 2/18

Overview

What will we cover:n Routing Terms explainedn Recall 20c3n Solving the Problemsn Networking requirementsn BCC Networklayout how it should ben Networklayout reality

Page 3: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 3/18

Networking terms

n Layer 2 OSI Data Link Layer. e.g. Ethernet or 802.11an Switch Layer 2 based interconnection device between

physical networksn Layer 3 OSI Network Layer. e.g. IP or IPXn Router Layer 3 device that connects Layer 2 segments

logicallyn Layer 4 OSI Transport Layer. e.g. UDP or TCPn LAN Provides physical network connectivity.n VLAN Devides a LAN into several logical/virtual LANs using

the same physical link.n Flow based routing Routing Switching on Layer 2 after a route

lookup using MAC instead of IP

Page 4: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 4/18

Recall 20c3 - Situation

n New Building with unknown problems...n about 20 different rooms with specific access profilen 4 floors interconnected through floor Dn different network hardware arrivedn lack of facility documentationn rogue services (dhcp) and hardware (access points!!)

Page 5: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 5/18

Recall 20c3 - Consequences

n Layer3 networks connected via L2 backbonen 2 routers did all routing workn Initial cabling insufficientn WLAN got flakyn DHCP became unreliablen A lot of extra work

Page 6: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 6/18

Recall 20c3 - Reasons

n Many VLANs that got “trunked“n Attacks on flow based routing equipment (TCAM full!)n Hardware (HP, Foundry) got overloadn Patching cables on undocumented panels is hardn Too many nodes in the WLAN and too powerful transcieversn Lack of network monitoringn Lack of user (available) documentationn Finally: fatigued NOCpeople...

Page 7: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 7/18

Solution strategy

Keep it simple!n Smaller collision domains (Layer2 segments)n Avoiding tagged (dot1q) / trunked (isl) vlansn Routing not on L3 switches but on real full-featured routersn Reduced trust in 802.11b (Do NOT expect it to work!)n Focus on 802.11an Explicit effort to ensure documentationn NOC Help Desk

Page 8: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 8/18

Special demands

n Entrance needs to be exclusively linked to the Orga Arean Network-Jacks for speakers need highly-available uplinkn WLAN (Soekris) need dedicated cabling (PoE!)n Helpdesk and Public Terminals should have high-available

uplinkn Video streams should be privilegedn Projects need “dynamic VLANing”n Wireless Mesh needs WLAN Channel 10 exclusively

n Server storage/housing for projects

Page 9: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 9/18

Network Services

n DomainNameService (recursive & authoritative) 82.130.23.35

n User DNS Registrationhttps://yourname.congress.ccc.de

n DHCP Service https://yourname.congress.ccc.den IPSEC Frontendhttps://illuminatheros.congress.ccc.de

Page 10: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 10/18

BCC Network Layout - LogicalUplink

Lützowstr Uplink BCC

C91

A85

B90.01

C57

A87

D57

Juniper N ETWORK S

TM

LT

M5

P

I nternet rocessor

AUX/MODEM

CONSOLE

MGMT PIC 0/3 PIC 0/2 PIC 0/1 PIC 0/0

R ETHERNET 1000 BASE-LX LINE

RX

AC

TI V ITY

RX

TX

STA

TUS

1000SX

PoE Switch Server Video NOC

ETHERNET 100BASE-TX

STA

TUS

PO

RT 1 R

X

LINK

PO

RT 0 R

X

LINK

PO

RT 2 R

X

LINK

PO

RT 3 R

X

LINK

ETHERNET 1000 BASE-LX LINE

RX

AC

TI V ITY

RX

TX

STA

TUS

2mbit

bcc.gate

l2.core

Blinken

Haecksen

Engel

Lockpick Workshop

Wikipedia Art&Beauty

POC

Wireless

Helpdesk Saal1

Public C

Public B

Public C Saal2

Saal3

CERT

Kasse

INFO

Hackcenter 1

Hackcenter 2

Funk

Orga

Soekris C

Soekris B

Soekris A

c91.core

a87.core a85.core

b90.core

c57.core

trust.core

d57.core IPSec

Uplink Netz

Backbone (Gbit SX Trunk) BCC Hausnetz Gbit Netz Wlan Netz Kassen Netz Patch Verkabelung

Page 11: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 11/18

BCC Network Layout - OSPF

Page 12: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 12/18

Hardware

n Inhouse Internet Uplink: Juniper M7in D57 (Core): Cisco Catalyst 6509n C57 (Ebene C): Cisco Catalyst 4507n B90 (Ebene B): Cisco Catalyst 4506n A85 (HackCenter 1): Cisco Catalyst 6513n A87 (HackCenter 2): Cisco Catalyst 4006n Access Layer: HP ProCurve 5308xl, Cisco 3750, Cisco

3550, Cisco 4908

Page 13: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 13/18

Implementation

n OSPF between core layer devicesn Multiple gigabit (etherchannel) interconnectsn VLAN Trunking for access layer devicesn DHCP forwarding from every VLAN to the DHCP via

‘ip-helper‘

Page 14: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 14/18

Internet uplink

n 1000.baseLX uplink (Thanks to Versatel!)n Own AutonomousSystemNumber (temp. AS34254)n Everyone gets a world reachable IP (temp. 82.130.0.0/18)n 3 Juniper Network M7i routersn internal BGP between thosen external BGP sessions from 2 routersn Native peerings with interroute21, Cogentco

Page 15: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 15/18

IP Uplink Topology

DECIXDHOSTING

BCCNETSIGN

S1000.baseTX1000.baseSX

Juniper M7i

berlin.gate

STM1

1000.baseLX

Juniper M7i

istanbul.gateBCIX

1000.baseSX

CogentcoInterroute21

IN-Berlin

DFN

1000.baseSX

Versatel

1000.baseLX

Juniper M7i

bern.gateCORE_D

Page 16: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 16/18

IPSEC Realisation

n IPv4 and IPv6n Based on OpenBSD isakmpdn X.509/ssh cert-/key-based authenticationn Anonymised usersn Non platform specificn Work in progress

Page 17: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 17/18

Using and abusing the network

n Staticly add MAC of your gatewayn Have you ever thought about ICMP route redirects?n Contact NOC Helpdesk for network problems: Phone

1234-NONETn Spanning tree HAS a purpose - YOU destroy YOUR network!

Page 18: 21c3 NOC Overview - blackwingblackwing.de › files › vortraege › 2004_12-21c3-NOC_Overview.pdf · The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 1/18 21c3 NOC Overview

l Overview

l Networking terms

l Recall 20c3 - Situation

l Recall 20c3 - Consequences

l Recall 20c3 - Reasons

l Solution strategy

l Special demands

l Network Services

l BCC Network Layout - Logical

l BCC Network Layout - OSPF

l Hardware

l Implementation

l Internet uplink

l IP Uplink Topology

l IPSEC Realisation

l Using and abusing thenetwork

l Sponsors

The 21c3 NOC Crew, January 9, 2005 21c3 NOC Overview - p. 18/18

Sponsors

n Internet routers

n Backbone routers

n Routing equipment

n Switches

n Upstream connectivityn Interroute21 - Upstream connectivity

n Upstream connectivity

n Internet uplink