Engineered for UptimeStrategies for Designing Reliable Data Centers and Networks

2

Agenda

• Engineering your Data Center for Uptime

• Leveraging 10 Gig in the Data Center

• Design Considerations

• TIA, ISO, IEEE Standards

• Pathways & Spaces

Data Center Design

Engineering your Data Center for Uptime

4

Outline

• Data Center Applications

• Data Center Infrastructure Design and Standards

• Network Engineer Concerns

• Data Center Standards

• Summary

5

Data Center Defined

• A Data Center is dedicated area within a building for connecting servers to the internal and external network

• Key Functions/Applications

– Storage of important data (websites, transactions, video, voice, logs, records)

– 24x7x365 immediate access to information from anywhere

– Reliability and Redundancy

6

Trends impacting the Data Center

• IP Convergence (More devices in LAN requiring Ethernet connections)

• Blade Servers (higher density servers)

• 10 Gigabit Speeds (copper and fiber)

• Storage Area Networks (Massive Hard Drive Arrays for the long term storage of information)

• Key Point: All of the above are very positive trends for high end Structured Cabling – Why?

7

Baseline IP

Bursty

VoIP

WAP

Security

Lighting

HVAC

IP Convergence Drives More Cable into LAN and Data Center

More Networked Devices drive additional Equipment in Data Center

8

Blade Servers increase the density of cabling in the data center

• Blade Servers (IBM shown below):

– Increase density of cabling as there are physically more servers per cabinet

– Increase heat generated in cabinet (Airflow becomes more important)

                                                                               

           

9

10 Gigabit Speeds require upgraded cabling

• No one will debate the fact that Data Centers will be the first to deploy 10 Gigabit

– They are already doing it over fiber

– Active equipment vendors recommending all new copper cabling be Augmented Category 6 to support 10GBase-T

• 10 Gig over Fiber – Laser optimized MM

• 10 Gig over Copper – Augmented Category 6

10

Why Design for 10 Gigabit?

• Philosophy of “Building for Bandwidth”

– Cabling expected life: 10-15 Years (12.5 yr Avg.)

– Active Equipment expected life: 3-5 Years (4 yr Avg.)

» Force10 Targeting extended up to 10 year life

– Conclusion: Cabling needs to be design to last at least 3 generations of active equipment

• Recent release of 10GBase-T (IEEE 802.3an) in June of 2006

– Equipment manufacturers working on early prototype solutions, expect the early commercial availability of NIC’s and Switches in 2007

11

Why Design for 10 Gigabit?

• Interesting Statistics

– IEEE Higher Speed Study Group is looking at 100 Gbps Ethernet over fiber in the 2010 timeframe

12

Storage Area Networks (SANs) require a high density of fiber

• SANs generally connect a high capacity hard disk arrays to the data center via Fiber Channel (not Ethernet, yet)

• Number of connections demand high density fiber cable management solutions

13

Network Engineer Concerns within the Data Center

• Scalable

– Density of equipment, cabinets, frames

– Fast and Accurate Moves, Adds, and Changes

• Thermal

– Problematic in most data centers (designed before Blade Servers)

– Poor Air Flow a Problem in many Data Centers

• Reliability/Uptime

– Cannot afford any downtime, even during expansion

• How can structured cabling positively impact each area of concern?

14

Scalability

• Managed Density – Proper design and cable management enhance the ability to track and maintain circuits.

• Fast and Accurate MAC work – Products should be designed to be easy to install and manage.

– The four fundamentals of Fiber Management Address these issues

15

Foundation of Fiber Connectivity

• Four Key Elements of Fiber Cable Management

– Bend radius protection

– Cable and connector access

– Intuitive cable routing paths

– Physical protection

16

Thermal

• Proper Cable Management promotes good air flow

– Overhead Fiber cabling through fiber raceway eliminates “Air Dams” below the raised floor

– Cable Management organizes cables so they don’t restrict airflow to switches and servers

17

• High Density Chassis Switches with very poor cable management

• 2:1 ratio Patch Panel to Horizontal Manager

Inadequate Cable Management

18

Overhead Fiber Raceway Example

• FiberGuide

– The key to optimizing air flow in the Data Center

– Provides bend radius protection, physical protection, and segregation of fiber optic jumpers

– Excels at all the key attributes of a good Fiber Raceway solution:

» Flexibility

» Speed of installation

» Speed of deployment

» Durability

» Fiber protection

19

Reliability of Data Center Equipment is Directly Tied to Proper Cooling

Proper Data Center Design deployment of Hot Aisle/Cold Aisle Cooling

– Good Airflow/Proper Cooling = Optimal Performance of Servers and Switches

20

Reliability of Data Center Equipment is Directly Tied to Proper Cooling

But what happens if poor cable management blocks airflow?

21

Reliability of Data Center Equipment is Directly Tied to Proper Cooling

Cables blocking air inlets and exits will raise the temperature of switches and servers lowering their reliability!

22

Cabling the Data Center

• Copper or Fiber? Both!

• Fiber Optic Cabling

– Backbone cabling

– Horizontal cabling over 100 meters

– SAN

– Recommended Types: Laser Optimized Multimode, Singlemode, or Hybrid

• Copper Cabling

– Horizontal Cabling of 100 meters or less

– Recommended Types: Category 6, Augmented Category 6

23

Choosing the Correct Cabling Type

• Category 6

– TIA-942 standard recommends Cat 6 as the minimum standard for Data Center installations

– Limited support for 10GBase-T (37 meters unmitigated per TSB-155)

• Augmented Category 6

– Not a ratified standard yet, although draft standards are available and IEEE 802.3an is ratified

– Full support for 10GBase-T to 100 meters in standard operation and 30 meters for Short-Reach mode

• Laser Optimized Multimode Fiber

– Recognized by TIA and IEEE 802.3ae

– Full support 10 Gigabit Ethernet over fiber for 300 meters or more depending on grade of fiber

24

Greatest Myths regarding 10GBase-T Cabling

• Myth: You can only run 10GBase-T on Augmented Cat 6

• Reality: 10GBase-T will run on Cat 6 for limited distances

• Myth: Alien Crosstalk is the only concern in 10GBase-T transmission

• Reality: All electrical parameters are a concern, especially attenuation and return loss

25

Greatest Myths regarding 10GBase-T Cabling

• Myth: You have to use shielded cable to run 10GBase-T

• Reality: Unshielded designs can easily run 10GBase-T and overcome all electrical issues associated with running 10 Gig

26

TIA-942 Data Center Standard

TIA-942 - Telecommunications Infrastructure TIA-942 - Telecommunications Infrastructure Standard for Data CentersStandard for Data Centers

• Site selection and sizing

• Cabling infrastructure administration

• Architectural and structural considerations

• Security and fire protection

• Electrical, grounding and mechanical systems

• Application distance limitations

• Access-provider coordination and demarcation

27

TIA-942 Data Center Model

Backbone Cabling

Horizontal Cabling

28

Interconnection vs. Cross-Connect in the LAN

29

Migration of Data Center Cabling Design

Generation 1: Direct Connection

– Difficult to Scale

– Done incorrectly impacts thermal performance

ServerSwitch

30

Migration of Data Center Cabling Design

Generation 2: Interconnect

– Prevalent architecture in LAN, scales adequately in smaller Data Centers

– For larger, Tier III and IV Data Centers design limits scalability

Switch Server

31

Migration of Data Center Cabling Design

Generation 3: Cross-Connect

– The optimum solution for enhanced cable management and reliability

– Allows for scalability (addition of equipment) by only changing patch cords

ServerSwitch

32

• Ideally a Data Center should not only be designed and installed in a tidy user friendly manner

• But also be maintained to the same level of installed manageability!

When it’s done right the first time!

33

Centralized Fiber Management in the Data Center

34

Fiber Termination OptionsField Term, Splice or MPO/MTP

• Field Termination

– Allows polishing of fiber in the field

• Splicing

– Fusion splicing of factory terminated fiber pigtails

• MPO Solutions

– Allow fast termination with factory polishing

35

Data Center Design Observations

• Data Centers are fundamentally different from typical LANs in that they require…

– Different Architecture

– Different Level of Reliability

– Different Standards

– Different Cable Management Requirements

– Different Structured Cabling Solution

36

The Data Center Design Dilemma:Density vs. Manageability and Reliability

Fundamentally, you cannot use the same structured cabling products originally designed for low density LANs and expect them to perform to the level required in a Data Center

37

The Data Center Design Dilemma:Density vs. Manageability and Reliability

Direct Connect

Cross Connect

38

The Solution: Cabling Systems Designed Specifically for Data Centers

• Solves both the Density and the Management issues

• Manages Ethernet cables (copper or fiber)

• Manages cables from both Switches and patch panels

• Designed to meet the Four Fundamentals of Fiber Cable Management

39

ADC TrueNet Data Center Solutions Summary

• Data Centers are fundamentally different than LANs

• Structured Cabling can positively or negatively effect Data Center reliability

• Proper design, adherence to standards, and the right product solutions will result in an optimal Data Center design

• Learn more at http://www.adc.com/truenet

-

• Are you an infrastructure designer or specifier?

• Would you like dedicated support and design tools free of charge with no obligation?

– Join PACE - ADC’s Collaborative Program for Professional Architects, Consultants and Engineers

For more information or to join today simply visit: adc.com/partners or contact us at: pace@adc.com