Through Modular Data Center Planning - BICSI Modular Data Center Planning Tan Puay Leong, ... •...

December 5, 2007

Space and Time SavingsSpace and Time SavingsThrough Modular Data Center PlanningThrough Modular Data Center Planning

Tan Puay Leong, RCDD Tan Puay Leong, RCDD

SnrSnr Industry ManagerIndustry Manager

pltanpltan@[email protected]

December 5, 2007page 2 /

Presentation Objective

• Data center operational efficiently presents many challenges

– Downtime is costly and disruptive

– Space is at a premium

– Time is expensive

• Proven and New Modular technologies can significantly improve time and space efficiency.

– Cabling designs can provide floor space savings and/or reclamation

– Installation and operational savings can be achieved

– IT assets and change documentation can be automated.

• This presentation will provide insight through a case study review.


Data Center Environment

• Many Data Centers have been built on an Ad-hoc basis

– By budget year

– By department

– By available space

• Point to point cabling

• Pathway confusion and congestion

• Documentation minimal or non-existent

– Rarely up-to-date

• Each Add, Move or Change has Space and Time Implications


• Ad-hoc growth and cabling

• Point to point cabling

Data Center Environment

Network Cabinets

Server Cabinets

Server Cabinets

Server Cabinets

Server Cabinets

• Pathway confusion

• Documentation


Cable Management? Look Familiar?


Data Center Topology – designing for order


Value of Space

• Data Center Floor Space in CBD average $ 70/sq ft per year just to operate.

– Includes:

• Raised floor, HVAC, Lighting, Power

• No electronics, rack/cabinet hardware or service contract

– A 3,000 sq ft data center would average $210,000/yr just for the facility usage.

• Collocation sites can run much higher $ per sq. ft.

– Off-site provisioning, back-up, redundancy, disaster-recovery, etc. benefits

The objective is to make better use of your space, rather than needing to build out

or build new facilities.


• An average 7’ rack or cabinet houses 40 rack spaces for network devices or connecting hardware

• Copper RJ45 patch panels

– 24ports per rack space

– Alternating spaces used for horizontal cable management

• 20 spaces connectivity, 20 for cable management

– 480 passive copper ports per rack

• Floor Space Dedicated per rack:

– 9 sq. ft. Traditional Room – Avg $ 630/yr ops. cost

– 20 sq. ft. Hot/Cold Aisle – Avg $1,400/yr ops. cost

Value of Space: Rack/Cabinet Footprint


960 passive copper ports per rack – high-density

(48 port 1u panels)

Flat or Angled 48 port panels will enable 960 ports in 1 rack foot print.

Angled panels can manage 960 ports in half a rack through elimination of horizontal managers.

Provides upfront and ongoing footprint savings

– Saves one rack/cabinet space per 960 port

– @ 9 – 20 square feet per rack footprint

– $ 630.00 - $1,400.00 per saved rack per year

Value of Space: High-density Connectivity


Data Centers aren’t built 1sq.ft. at a time

• Consolidating the

footprint can:

– Reduce initial fit-out

costs

– Reduce ongoing

charges

– Reduce complexity

– Ease troubleshooting

– Minimize impact of

growth


480 passive copper ports per rack -traditional (24port panels)

Average termination time per RJ45 port = 8 min.– includes prep, termination, dress, insert & label

• Termination time alone = 64 hours / rack• 6.4 hours per 48 ports

Value of Time


Shift to 48-port 1u 4-pair to RJ45 panels

960 passive copper ports / rack – high-density

(48port panels)

But what about time?But what about time?

– Termination time alone = 128 hours / rack

• 6.4 hours per 48 ports

Value of Time


Update to: 48Update to: 48--port 1u MRJ21port 1u MRJ21--RJ45 modular panel RJ45 modular panel

960 passive copper ports / rack – high-density panels

What about time?What about time?

Average termination time per MRJ21 port = <2 min.

– Each MRJ21 = 6 GbE ports

– Includes push-on keyed connection, tightening 2 jack-screws, &

label

• Connection time alone = 5.33 hrs / rack vs. 128hrs for 4-pair• 16 minutes per 48 ports

Value of Time


Pathways: Cable Management

• Cables should be placed in pathways adhering to recommended fill rates

– BICSI recommends 60% maximum fill for cable tray

• Cable trays should be sized to accommodate growth or change if necessary

• Separation should be in place to accommodate media or change.

– Many data centers used tiered cable trays:

• Top tray - Most accessed cables

• Lower tray (s) – Fixed / Permanent cabling

• Plan pathways to accommodate media OD and weight for structure

– 6 4-pair = .714” CMA (OD) 1 25-pair = .5” CMA (OD) 30% ↓↓

– 6 4-pair = 168lbs. / m ft 1 25-pair = 130-135lbs. / m ft 20% ↓↓


Case Study – Collocation Client SiteBackground

• Tier 4 Data Center

• Client needed to add-on 5,000 sq ft in build-out.

• Previous section used point to point cabling

– Non-existent documentation

– Trouble shooting is extremely time consuming

– Traditional cabling densities in switch and server cabinets

– Incorporated 4-pair and 25-pair copper, duplex and multi-pair fiber

cable


Case Study – Collocation Client SiteBackground

• Added build-out space to be utilized for additional 210 cabinets

– Roughly 85% server, 15% network

• Needed documentation and 24/7 rapid recovery – wanted real-time

visibility from HQ

• New build-out is fully modular

– Design uses HDA as a central patching area for administration and

documentation

– All server and network cabinets cable to HDA

– Automated documentation AMPTRAC with IM software incorporated

at HDA


Data Center Install – Co-lo Facility HDAReference site

• 30 48 port panels

per 7’ rack

• 1,440 ports

• 240 trunk cables

• 1 rack replaced 3:

≈66% floor space

savings

• Built-in intelligence

• Automatic

documentation

database updates –

copper or fiber


Data Center Install – Colo FacilityHDA


Data Center Install – Colo FacilityServer Cabinet

• Common design with high-density panels located top of rack

• Fiber Optic Panel located every 3rd cabinet

• Common Patch cord lengths at every cabinet

– Simplified initial BOM and site assemblies


Modular Cable Bundling

• Group Cables in logical and manageable bundles

– Reference site used 4 assembly group (24 ports/ left or right half panel)

• Expedites the site installation at panels

• Pre-terminated cable assemblies, pre-tested, pre-labeled bundles.

• Each assembly terminates 6 ports (tighten 2 screws)

•• ReRe--usable assemblies and panelsusable assemblies and panels


Cable Bundling – Reference Site

• 30% smaller cable bundle than

comparable 4-pair cables

• Provided adequate bend radius

• Cable lay in vertical and horizontal

pathways

– Vertical pathways 8”x8” back to

back

• Eases Change and tracing

• Did not over-tighten bundles


AMPTRAC Patchcords on Front Panel

• Important to plan for volume and

size of patchcords into cable

managers


Case Study End Result:

Client realized:– Overall space savings of approximately 30%

– Installation time significantly reduced – termination in panels ≈10x

faster through modular copper and fiber optic installation

– Better utilization of electronics ports – fully configured switches

– Remote visibility to port connection, IT asset status, network

performance with real-time change updates

– Technician patching done only at central patch area

• Switches and servers are undisturbed – cabinets locked

• Security maintained throughout facility and access


• Modular cabling laid-in pathways

Star-wired to HDA to cabinets

• Equipment added as needed

Modular Data Center Environment

Network Cabinets

Server Cabinets

Server Cabinets

Server Cabinets

Server Cabinets

Central Patch

• Growth and change managed with patch

cords at HDA

• Automated documentation installed @ HDA


Summary

• Modular Data Center Planning provides space and time savings

• Data Center Standard simplifies management, troubleshooting and

documentation

• New high density patch panels enable reclamation of valuable floor space

• New network electronics based connector technologies provide modularity

previously unavailable.

• Modular high-density connectivity enables rapid installation and recovery of

site cabling

• The combined effects of space and time savings provide a simplified growth

path for network migration and planning

December 5, 2007

Data Center Technology OverviewData Center Technology Overview


Network Vendors with MRJ21 Equipment

• Alcatel:

– OmniSwitch

– Alcatel TiMetra

– Lucent / Riverstone

• Barco Xenia

• EIT

• EntriSphere

• Foundry Networks

• Force10 Networks

• Fujitsu

• IBM

• Motorola

• NMS Communications

• Proworks

• Shanghai Electronics

• Stratex Networks

• Tellabs / Vivace

• UT Starcom

Force10 E120090 GbE ports/card

Foundry BigIron RX-4, 8, & 1648 GbE ports/card

Note: The vendors listed and shown have publicly launched MRJ21 equipment into the marketplace. Those not listed should be contacted directly for program status updates.

IBM eServerBladeCenter Copper

Pass-thru module 5 GbE ports Alcatel OmniSwitch

NI48 card


MRJ21 Copper Cabling System

High Performance copper cabling solution

• High-density, small form factor connectivity

• Modular pluggable solution

– Trunk cable and cassette

• Supports 10/100/1000BASE-T applications

• Factory terminated and tested solution

• Savings on

– Space

– Time

– Maintenance


MPO Fiber Optic System

MPO System Features

• Standard compliant

– IEC/EN 61754-7 (MPO connector family)

– TIA-568-B.1-7

• Modular system

– Pre-terminated fiber optic components

– Trunk cables and cassettes

– Plug and Go solution

• High Density System

– Compact and high fiber count MPO connector

• Factory Terminated and Tested

– Quality consistency and assurance


MPO Fiber Optic System

MPO System Applications

• High bandwidth LAN connectivity

– Ethernet – 1Gbps, 10Gbps,

– Next Generation Ethernet – 40Gbps, 100Gbps

• Parallel optics

• SAN connectivity

– Fiber Channels

– InfiniBand


MPO Configuration Overview

MPO Trunk Cable Assemblies

• Based on Industry standard MPO connector

• Pre-terminated with high fiber count

• MPO Connectors based on push-pull type

• Available length – 10m to 300m

MPO connector


• Up to 72 fibers in one

MT-ferrule/connector housing

• Applications

– Jumpers and trunk cables

– Interface to transceiver

– SAN Very Short Reach

applications

– Also available for custom

applications

MPO Product Overview

24-fiber (2 rows of 12)

72-fiber (6 rows of 12)

Original 12-fiber


MPO Configuration Overview

MPO System Configuration

2 Pieces


High Density Connect Cable Management

• Recommended to be used at the

MDA and/or HDA

• Has been designed to effectively

manage the density of cables

• The Bend control design ensures

that the Installed Patch cords

automatically achieve the

Minimum Bend radius

requirement.

• The Bend control also features

accessible cable retention that

ensures maximum user flexibility.


High Density Cabling System


Intelligent Infrastructure Management System

• Tracks all connectivity changes in

real-time

• Maps and documents the physical

infrastructure ‘end-to-end’

• Discovers devices connected to

the network

• Logs, reports, sends alarms and/

or alerts for all changes to assets

and physical layer connectivity

changes


Intelligent Infrastructure Management System Overview

I/O Cables

Patch Panel

Switch with Sensor Strip

Patch Cord

Analyzer

Ethernet

• Analyzers

– tracks Connections and communicates to the Database

• Patch Panels

– with integrated Sensor Contacts

• Sensor Strips

• I/O Cables

– link Panels / Switches and Analyzer

• Patch Cords

– 9th Wire Cords interconnect between Sensor Pads

• Intelligent Management Software

– Controls the Analyzers & Database

• Intelligent Management Server

System Components


Wiring Closet View

Complete and real

time information:

Graphic View

Floor Plan with user‘s

work place location and

it‘s assets.

Control Panel System Overview

Trace View

Show the circuit

connectivity thru the

network...

Data View

Component data

Locate View

Showing database

with network

hierarchy


Summary

Important Considerations & Best Practices

• Capacity Planning - future growth and expansion

• Bandwidth demand - technology supported today and future

• Management - density, flexibility, segregation and maintainability

• Legal Compliance – Standard compliance, Security and local codes

• Administration and management

Through Modular Data Center Planning - BICSI Modular Data Center Planning Tan Puay Leong, ... •...

Documents

Transcript of Through Modular Data Center Planning - BICSI Modular Data Center Planning Tan Puay Leong, ... •...