Rochester Institute of TechnologyRIT Scholar Works

Theses Thesis/Dissertation Collections

2009

Data center design & enterprise networkingChristian Mahood

Follow this and additional works at: http://scholarworks.rit.edu/theses

This Thesis is brought to you for free and open access by the Thesis/Dissertation Collections at RIT Scholar Works. It has been accepted for inclusionin Theses by an authorized administrator of RIT Scholar Works. For more information, please contact ritscholarworks@rit.edu.

Recommended CitationMahood, Christian, "Data center design & enterprise networking" (2009). Thesis. Rochester Institute of Technology. Accessed from

Data Center Design & Enterprise Networking

By

Christian Mahood

Thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in

Networking & Systems Administration

Rochester Institute of Technology

B. Thomas Golisano College of

Computing and Information Sciences

February 25th, 2009

Rochester Institute of Technology

B. Thomas Golisano College of

Computing and Information Sciences

Master of Science in Networking and Systems Administration

Thesis Approval Form

Student Name: Christian Mahood

Thesis Title: Data Center Design & Enterprise Networking

Thesis Committee

Name Signature Date

Luther Troell Chair Charlie Border Committee Member Sylvia Perez-Hardy Committee Member

Thesis Reproduction Permission Form

Rochester Institute of Technology

B. Thomas Golisano College of Computing and Information Sciences

Master of Science in

Networking & Systems Administration

Data Center Design & Enterprise Networking I, Christian Mahood, hereby grant permission to the Wallace Library of the Rochester Institute of Technology to reproduce my thesis in whole or in part. Any reproduction must not be for commercial use or profit. Date: ___________ Signature of Author: __________________________

Abstract:

Today’s enterprise networks and data centers have become very complex and have

completely integrated themselves into every facet of their represented organization.

Organizations require Internet facing services and applications to be available at any

part of the day or night. These organizations have realized that with centralized

computing and highly available components, their technological presence with

customers can be greatly enhanced. The creation of an infrastructure supporting

such high availability takes numerous components and resources to function

optimally. When an organization makes the decision to design a data center, they

utilize resources to provide insight into what components to deploy. Much of this

information is based off of recommendations made by third party vendors or limited

past experiences. This research provides a course offering as a solution to help

provide students with the information to design and comprehend the major

components within a modern data center. The information included in the course

offering has been compared with industry accepted standards and various other

resources to provide reliable and accurate information. Course topics have been

architected around eight major topics. These topics covered are network design,

electrical systems, HVAC (Heating, Ventilation, and Air Conditioning) systems,

security, management, redundancy, disaster recovery and site planning. The

information contained within the lectures has been compiled from multiple sources to

provide a single location for information. Furthermore, the course offering will utilize

class discussions, case study analysis and activities to re-enforce key points.

Providing such a course for students to learn about data center components will

provide organizations with the ability to rely less on outside information and design

scalable data centers that can provide years of growth.

TableofContents

1Introduction .................................................................................................................... 11.1Problem ............................................................................................................................................................................11.2Importance......................................................................................................................................................................3

2Reviewofcurrentresearch.............................................................................................. 3

3Background ..................................................................................................................... 8

4ResearchDesignandMethodology................................................................................ 104.1AssumptionsandLimitations ..............................................................................................................................104.2ResearchMethodology ...........................................................................................................................................124.3TestSystemDesign...................................................................................................................................................145Results .......................................................................................................................... 155.1CourseTextbook........................................................................................................................................................155.2TopicDesign ................................................................................................................................................................175.3TopicExpansion.........................................................................................................................................................195.3.1NetworkDesign.......................................................................................................................................................195.3.2Electrical ....................................................................................................................................................................225.3.3HVACSystems ..........................................................................................................................................................245.3.4Security .......................................................................................................................................................................275.3.5DisasterRecovery...................................................................................................................................................285.3.6Tools.............................................................................................................................................................................30

6Conclusion..................................................................................................................... 31

7FutureWork .................................................................................................................. 328.1CourseSyllabus ..........................................................................................................................................................338.2CourseICCForm........................................................................................................................................................398.3CourseLectureSlides ..............................................................................................................................................438.3.1Week2Slides............................................................................................................................................................438.3.2Week3Slides............................................................................................................................................................528.3.3Week4Slides............................................................................................................................................................608.3.4Week5Slides............................................................................................................................................................718.3.5Week7Slides............................................................................................................................................................80

8.4MidtermExamExampleQuestions ...................................................................................................................898.5MidtermExamExampleQuestions ...................................................................................................................919References: ................................................................................................................... 93

2

Table of Figures

Figure1 ............................................................................................................................ 12

Figure2 ............................................................................................................................ 17

Figure3 ............................................................................................................................ 18

Figure4 ............................................................................................................................ 30

1

1 Introduction

Organizationalnetworkshavebecomevastandcomplex.Manycompaniesthroughoutthe

worldhavedatacentersthathaveevolvedfromsingleroomswithinlargercompany

operationcentersintoenormousstandalonebuildingsthathousethousandsupon

thousandsofservers.Thereisnolongeranotionthatadatacenterbelongsonlytothelarge

organizationsoftheworldassmallerandsmallerbusinessesarebeginningtorelyonthe

centralizationandbenefitsofrunningadatacenterthatcanassistwiththeiroverall

efficiency.Datacentersrangeinvaryingsizeandcapabilitiestodayandalsoarecomprised

withvaryingcomponents.Thereareanumberofdifferentpieceswithinadatacenterthat

mustmeshtogetherseamlesslyforperformancetobeoptimal.Inorderforanorganization

toimplementandbuildadatacenter,heavyplanningisinvolvedtounderstandtheneedsof

theorganizationandtodeterminethefullscaleoftheendresult.Someorganizationsmay

payparticularattentiontofinancialsideofthedatacenter,whileotherorganizationsmay

payattentiontooverallspacethatcanbeachieved.Thisparticularneedwillvaryfrom

organizationtoorganization,howevertherearemultiplekeyareasthatmustbeaddressed

todesignafunctionaldatacenter.

Thisresearchaimstoprovideasolutionforacademicinstitutionstoprovidestudentswitha

locationtolearnaboutdatacenterdesignandnetworking.Byprovidinginformationabout

datacenterdesignandnetworking,studentswillbeabletoutilizeexperiencesand

informationfromtheclassinfuturecareersfororganizationsthatwishtoreconfigure,plan

orretrofitenvironmentsforusageasadatacenter.

1.1 Problem

Datacentersrequirealargeamountofplanning,interactionwithdifferentresourcesanda

largeunderstandingofdifferentareasofexpertisetoimplementfromstarttofinish.Thereis

2

alargeamountofcostandtimededicationtounderstandalltypesoftechnologyand

componentsthatmustbeimplementedwithinadatacenter.Thereisinformationavailable

acrossmanydifferentplatforms(theInternet,books,articles,etc,)howevertherelacksa

resourcethatcanhelpstudentsunderstandthecomponentsandtheimportanceofdesign

intheplanningofadatacenter.Thedevelopmentofacourseofferingthatcoversthese

topicswillallowstudentstoshareexperiencesthattheyhaveencounteredandalsowill

allowstudentstocollaborativelylearndifferentfundamentalstogetherthroughdiscussion.

Thereisampledataavailableinlibraries,theInternet,variousbooks,andmagazinearticles

thatcanassistintheplanningandimplementationofadatacenterforanorganization.

Manyoftheseresourcesaredifficulttoobtain,containinformationonasinglepieceofthe

infrastructure,lackcomprehensiveinformationinonesourceanddonotprovideadequate

recommendationsbasedonexperiences.Theinformationcanbedifficulttoobtaindueto

financialinvestmentandalsocantakelongamountsoftimetocorrelate.Thelargeamounts

ofdatamakeitdifficultforausertodecipherexactlywhatcomponentsarecriticalandvital

tothesuccessoftheproject.Itisimportanttodesignthedatacenterforcurrent

organizationalneeds,yetplanforthegrowthofthedatacentertomeettheneedsofthe

organizationsforthenexttentotwentyyears.

Acourseofferingthatcompoundstheinformationandallowsindividualstudentexperiences

tobesharedisoneapproachtodecreasetheneedforvariousdifferentexternalresources.

Thisresearchhasnotfocusedonthebenefitstocentralizationofinformationinregardsofa

datacenter,butpresentsasolutiontoassiststudentscomprehendandunderstandthe

necessarycomponentswithinadatacenterandtakesalargestepofgatheringthe

informationoutofthepicture.Manysourcesdiscusshowtocreateadatacenter

surroundingcurrentorganizationalneeds.Adatacenteristhendesignedsurroundingthese

“needs,”howeverwithintwotothreeyearstheorganizationcouldrealizethatithasrunout

ofspaceandneedstodeterminetheircourseofactionyetagain.Howcouldtheresponsible

personthengobacktotheorganizationandaskagainforhundredsofthousandsofdollars

3

foryetanotherdatacenter?Thoughtfulplanningandestimationthroughoutthedesign

processcouldhavemitigatedthisbyunderstandingscalabilityconcepts.

1.2 Importance

Planningthroughouttheentireprojectofadatacenterisimportantasitallowsforthe

correctestimationandscopeofthevariouscomponentssupportingthedatacenter.The

infrastructureofthedatacentercanallowflexibilitiesinthesystemcomponentsinstalledin

thedatacenterifplannedcorrectly.Thevariouscomponentsfromelectrical,andheating,

ventilationandairconditioning(HVAC)systemsinstalledinadatacenterareusually

modifiedtoprovidehighercapacitiesandmanyorganizationscouldutilizetheskillslearned

inancourseofferingtoprovideconsciousdecisionsduringprojectssurroundingtheirdata

centers.Thisresearchwillbenefitanyorganizationthatislookingtobuild,retrofit,ormodify

adatacenter.Itwillalsobenefitanyinstitutionlookingtoimplementsuchacoursetoassist

studentsinunderstandingdatacenterdesignandmajorcomponents.

2 Review of current research

Researchregardingthecomponentsofadatacenterandtheirimportancehasbeencovered

inindustryandacademialedresearch.Thereisanimportanceunderstoodthatthedata

centeritselfprovidesmanyopportunitiesanduniquefeaturesthatothertypesofcomputing

donotfeature.Highlypoweredcondensedserverclustersandhighavailabilityhardware

makethedatacenteracriticalcomponenttotoday’sorganizationalsuccess.Howcanthese

typesofenvironmentsbeplannedcorrectlyforfullscalability?Furthermore,howdoallof

thesecomponentsworktogetherinafashionthatcanbemanagedsuccessfully?Inthis

section,priorresearchispresentedaboutdatacentercriticality,poorsiteplanning,network

components,managementandlastlyscalabilityimportance.

4

(Alger,2005)focusedontheneedforadministratorstorealizeandfullyunderstandthe

criticaltieofaDataCentertothebusinessitsupports.Thepurposeofhispublicationwasto

informadministratorsandexecutivesoftheplanninginvolvedwithaDataCenter.Theother

focusofthepublicationwastoaddressthemagnitudeofcomponentsthatgointoaData

centerandhowtochoosethebestsolutionfortheneedsofthebusiness,orcustomer.His

researchwasqualitativeregardingthecomponentsandincludedquantitativemethods

involvingdesigninganalysis.Thebookwaswrittenpullingtheexperiencefromtheauthor

andthosefromtheCiscoInfrastructureTeamtogiveadministrators,professionalsand

executivesabetterunderstandingofaDataCenter.Thebookstructuresitselfintothekey

areasofthedatacenter;thesite,thespace,theelectricalsystem,thenetwork,theairand

temperatureofthesite,andthemanagementoftheDataCenter.Theresultsofthe

publicationaremultipledifferentviewsandopinionsaboutthedesignandimplementation

oftheDataCenterandtheavailableoptionstothecustomerallinasinglepublication.This

publicationisuniqueinthatitcombinesalmostallaspectsofthedatacentertogetherinan

analysistothedepththatallkeypointsarecovered.Thetechnicalflawofthispublication

wasthatitdoesnotcoverthedepthnecessaryinDataCentersfornetworkingsegregation

andflowlayout.

Anothersimilarpublication(Snevely,2002)discussesthoroughlyaboutthelackof

constructionplanningwhenapproachingaDataCenter.Themainfocusofhisresearchwas

toprovidethoseinterestedinplanningtheconstructionorre‐constructionofaDataCenter

andevenasminuteasreconfiguringorretrofittinganexistinginfrastructurewiringcloset.

HisattemptwasnottocovertheallaspectsofthedesignofaDataCenter,buttoprovide

resourcestohelpassistthedesignofaDataCenterbasedoffoftheneedsandrequirements

oftheDataCenterforconstructionalpurposes.Hisresearchincludedqualitativemethodsof

constructionanddesignfrompreviouscasestudies,experienceanddocuments.The

materialcoveredinsidethebookrangesfromexteriorspacedesignsdowntothewiring

insideaDataCenterrackandevenincludestrashcontrolmethods.Robextensivelycovers

andstressestheimportanceofplanninginaDataCenter.Eachsectionwithinthebook

5

coversatleastonetiporkeypointtomakenote.Thebookalsocoversindepththeneedto

followlocalandfederalbuildingcodeinregardstotheconstructionandimplementationof

componentsinDataCenters.Thisisauniquefeaturethatshouldbetakennoteduring

planningstages.TheconclusionthatSnevelymadeduringhisbookwasthatwhilethe

planningofDataCenteriscomplicated,ifdonecorrectlytheDataCentercanprovidea

scalableandmodularenvironmentforanysizeandcapabilityneededbythecustomer.The

otherresultofthebookisapublicationthatcanprovidequickreferencefordesign

requirementsandtips.Thedifferenceinthispublicationincomparisontootherworksisthat

whileitdidnotcomprehensivelycoverservercomponents,ornetworkcomponentsitdid

coverthemajorityoftheenvironmentalcomponentsnecessarytodatacenterdesignand

shouldlookedatasavalidresourceforstudents.

(Berktowitz,1999)presentedanin‐depthanalysisofenterprisenetworkingandcomponents

ofenterprisenetworks.Inparticular,thispublicationfocusedfullyonenterprisenetworking

ratherthannetworkingingeneral.Theauthorrecognizedaproblemwithnetworking

“cookbooks”thatclaimtoteacheverything,butlackthefollow‐throughofexplainingthe

“whatandwhy”ofnetworking.Inthispublication,particularlychapterssevenandeight,the

authordistinguishedthedifferentlayersofthenetwork.Itisauniquepublicationthattakes

directlyintoconsiderationtheflowofthenetwork,theperformance,andmostnotable,the

implementationofnetworksegmentation.Theresultofthissectionwasacomparison

betweenthedifferenttechnologiesavailableandtherecognitionofcostintheequation

duringplanninganddesignstages.Theauthoralsorecognizedthevastarrayofdifferent

technologiesavailabletothecustomeranddiscussestheneedtounderstandthebusiness’

customer’sneedspriortodesigningthenetworkportionoftheDataCenter.Thispublication

directlyaddressedtheneedforenterprisenetworkdesignandwentintogreatdetailabout

thistopic.ThepublicationdoesnotaddressanyintegrationorcasestudiesofDataCenter

integrationandalsodoesnotmentionotherportionsofnetworkingintheDataCenterother

thanLayers2andLayer3oftheOSImodelcommonlyreferredtoinnetworking.Whilethe

6

studyisusefulforasingleimportantcomponentwithinthedatacenter,itdoesnotpresent

anyinsightintotheintegrationofanyothercomponentsrelevanttoadatacenter.

AgroupatCarnegieMellonUniversity(Sun,2007)focusedonmanagementofDataCenters.

TheyfoundthatmonitoringisacrucialcomponenttoaDataCenter’sfunctionalityand

maintenance,yettherewerenotcomprehensivesolutionstoaddressmanagement.The

teamatCarnegieMellonthendevelopedanalgorithmthatwillassistinthemanagementof

theDataCenterwithlessofanimpactontheadministrator.Thepurposeofthisresearch

wastointroducetothearenaofmonitoringanothertoolthatprovidedadaptiveandreliable

resultsfortheDataCenter.Theirresearchusedmixedmethodstotakeexistingknowledge

abouttheSNMPprotocolandmanagementsoftware,whilenewmethodswereproposed

basedupontheseexistingmethods.ThealgorithmwastestedonalargeclusteratCarnegie

Mellontotestfunctionalityandscalability.TheirimplementationutilizedSNMP,whichis

oneofthemostubiquitousprotocolsavailableformonitoringpurposes.Thefocusoftheir

algorithmwastoassistinandfurtherenhancetheconfigurationportionofmonitoring,the

reasoningalgorithmusedtodetermineifaneventisanomalous,andthestorageofthe

historicaldata.Theresultsoftheirstudyincludedresultsthatwerenotonlypositive,but

exceededwhattheyfeltweretheirgoalsintheproject.Theirresearchwasrelevanttothe

topicofthisthesisinthatmanagementandmonitoringarecrucialaspectstoaDataCenter.

Al‐Fares,Loukissas,andVahdat(Al‐Fares,2008)discussedthoroughlytheneedforScalable

interconnectivityinclusters,economicavailability,andbackwardscompatibilitywithlegacy

protocolsandsystems.Theirprojectaddressedaconcernforconnectivityinhierarchical

networkswherebandwidthiscrucialandexpensive.Thepurposeoftheprojectwasforthe

teamtointroducetheproblemtothenetworkcommunityandproposeaneconomic

solutionthatutilizedexistingequipmenttosatisfybandwidthneeds.Theirsolutionwasa

technologycalledfat‐tree,whichsegregatesthenetworkinto“Pods”andtwo‐levelrouting

thatallowslookupsformulti‐pathrouting.Theprojectimplementednewcodeontoexisting

routersandswitchesthathelpedandallowedthemtoperformthetwo‐levelroutingand

7

lookups.Theimplementationandtestingofthistechnologywassuccessful,howevernot

ideal.Thecore‐switchinglayercanbecomecongestedduetocollisionandroutingdecisions

madebytheroutersdownstream.Theotherresultthatwasnotoptimalwasthenumberof

cablesrequiredtocreateameshtopologyforthenetwork.Thisprojectvisitsaunique

problemthatoccursinDataCenters.Theneedforscalablebandwidthforinterconnectivity

isstillnotascapableastheclustersandtechnologythatisavailableforusage.

(Sharma,2008)addressedtheimportanceofcooling,power,andcomputinginaDataCenter

withtherecentchangestoserverequipmentanddesign.Sharma,Shih,BashandPatel

describethechangesinserverhardwareandthatithasbecomemoreandmoredenseand

powerfuloverthelastdecade.Thishascreatedahigherlevelofnecessaryresources

availablefortheseserverstooperatesuccessfullyandefficiently.Toaddressthisissue,the

groupfromHewlett‐PackardproposedacontrolenginetocreateaDataCenterthatcan

throttleandchangeitsrequirementsforcooling,powerandcomputing.Theauthorsalso

discusshowthissolutionvariesfromtraditionalDataCentermethodologyinthateachgiven

areainaDataCenterissegmentedfromothersviacontrolengines.Thethreeareas(Power,

Cooling,andComputing)canbethrottledupordownbasedoffalgorithmsthattheauthors

havespecified.Ifahigherdemandisnecessaryforonesection,otherscanthrottledownto

allowresourcestobedistributedtotheotherworkloadareasintheDataCenter.Theyalso

discussedthatthethreeneededtobemanagedasanensembleandnotseparateentitiesfor

benefitstobeobtained.Theoutcomeofthisprojectwastherealizationthatenergy

consumptionandflowswillcontinuetobecriticalinDataCentersandthatthemanagement

ofthesecommoditieswillbecomemoreandmoreofafocusduringthedesignoftheData

Center.

8

3 Background

Inpastimplementations,manydifferentgroupsfromdifferentexpertiseandbackgrounds

havedesignedthevariouscomponentsindatacenters.Eachdifferentgrouphadanichein

theirrespectivefieldandcompletedtheirinstallationbasedoffofinformationthatwas

providedtothembytheprojectmanagerorleadoftheproject.Serverrequirementsinthe

pastweremuchdifferentthanthatofserversandequipmenttoday.Theequipmentwas

morebulky,andmostsystemsdidnotresideinrackcabinetsoftoday’sstandards.Most

systemscalledmainframeshadentirecomputerroomsbuiltaroundthemrequiringhigh

amountsofpowerandHVACsystemsforoperation.Theirfunctionwastoprovidehighly

processorintensivecalculationsforanorganization.Theyalsoallowedforhighdatatransfer

andmulti‐clientconnectionasthedevelopmentofthesesystemsprogressedthroughthe

years.SomeofthemostwellknownofthesesystemswerebuiltandinstalledbyIBM

(Mainframe.)Thecomponentswerenotasdemandingincomparisontotoday’sdatacenter

ascomputingpowerwasneitherhighlycondensednorscalable,butthepowerandHVAC

requirementsofthesemainframeswereuniqueandrequiredspecialdesignsfor

implementation.Computershaveadvancedconsiderablysincethepeakofthemainframein

the1980’sandmanysystemshavebeenreplacedforserversystemswithinthesamerooms

ordatacentersthatoncehousedtheselargemainframes(Mainframe.)

Asthedemandfortheclient‐serverbasedarchitecture,theInternetandindividual

computershasgrownexponentiallyinthepasttwodecades,theneedtodeploysystemshas

alsoincreased.Whilethesenewersystemsdonothavethefullcomputingpowerthat

comparablemainframeshave,theirfootprint,technologicalflexibilityandlowerindividual

costcreatesalargedemandthusmoreandmoreserverswereimplementedinshort

amountsoftime(DiMinico.)Withfurthertechnologicaladvancetheseserversnowcanbe

placedinracksthatcanhouseasmanyas50serversperrack.Theirfootprintcondensesthe

heatoutputandelectricalintake.Withthismanysystemsinasinglerack,onecanclearlysee

thechangesandstressthatinfrastructuresnowplaceonthesameenvironmentalsupportof

9

thesedatacentersthatwereoriginallydesignedformainframeusage.Becauseofthese

changes,manyorganizationschoosetoredesign,rebuiltorstartfreshwiththeirdata

centers.

Asenvironmentsarere‐built,ithasbecomeanorganization’sabilitytoplanforthescalable

futureofitsinfrastructure.Fromtheexperiencecompanieshavegainedfromconversionof

mainframetoserverclustersandimplementations,itcanbeexpectedthatthetrendof

higherpowerinasmallerlocationwillcontinue.Thiscreatesuniquerequirementsfor

electricityandHVACsystem,whicharethetwomostcriticalenvironmentalfactorsinadata

center(Alger.)Planningforthesetwosystemsmustprovideadequatescalabilityforatento

fifteenyearperiodinregardstoanorganizationalgrowthoutlook.

Inthepastdatacenterimplementationsindividualsresearchcomponentsduringtheproject.

Componentsareresearchedonaneedtoknowbasisandsolutionsareprovidedbythese

individualsorbyspecializedthirdpartycontractors.Individualswiththeunderstandingofall

facetsofthedatacenterarehardtofindandcanchargehighratestouseforconsultation.

Informationforindividualsororganizationstocompleteaprojectofthisnaturehastypically

beendispersedinmultiplelocationswithinbooks,Internetarticlesandpersonal

experiences.Havingthisinformationavailableisalargetoolthatcangreatlyassistinthe

planningofthedatacenter.Havingtheinformationavailable,inonelocationand

understoodbyanemployeeorgroupofemployeeswithinanorganizationcanonlyassist

withtheplanningandallowforabetterprojectoutcome.Thisisanimportantpieceinadata

centerprojectandisafocusofthisresearch.

10

4 Research Design and Methodology

Inthissection,anapproachtoacourseofferingfeaturingdatacenterdesignandenterprise

networkingispresented.Theapproachthattheresearchfollowedwasbasedonthe

compilationofvariousinformationsourcestoprovidecoverageofcrucialdatacenter

componentsincoursepresentations.Alsopresentedistheconfigurationofatestsystemto

showstudentsvarioustoolstoutilizeformanagement.Section4.1providesassumptions

andlimitationsforthediscussedportionsofresearch.Theremainingsections4.2and4.3

discusstheapproach,andexplanationoftheresearchandtestsystem.Insection5,the

deliverableresultsarepresented.

4.1 Assumptions and Limitations

Thisresearchdoesnotintendtospecifythatthetopicscoveredbythespecifiedcourseare

theonlysolutionavailabletoorganizationsorcustomers.Thereforeitisassumedthatthe

researchwillbeappliedgenerallytoorganizationsasawaytoensurethatallcrucial

componentsarecoveredandtoprovideassistancewhereweaknessesinaspecifictopicmay

exist.

Itisalsoassumedthatwhilebrandnamesmaybeutilizedwithinthematerialofthecourse,

therearenorecommendationsimpliedbythisresearch.Thestudentororganizationshould

decideequipmenttypesbasedoffofinternalpartnershipsandvendorsandalsowhich

equipmentfitstheirprojectorenvironmentoptimally.

Itcanalsobeassumedthatbyhavinginformationinasinglelocationbeingthecourse,less

timewouldbespentresearchingandpollinginformationfromdifferentresources.Itisalso

assumedthatbyutilizinglesstimeperformingresearchtounderstandtopicfundamentals,

11

theindividualcanthenspendmoretimelearningintricatedetailsandsolutionstoachievea

bettersolutionforthatdatacenter.

Anotherassumptionforthisresearchisthatthefundamentalsofbasicnetworking

technologieswillbepriorcovered.CoveragesuchastheTCP/IPprotocols,basiccomputer

networking,interconnectivityandbasicadministrationresponsibilitieswillnotbecoveredby

thisresearch,butwillbenecessarytounderstandsometopicsoutlined.

Therearevastdifferencesinorganizationsandcustomerneedsthatwillnotbemetbyall

aspectsofthisresearch.Manysolutionscomprisedwithcomponentswillfitafew

customers’needs;howeveroverallnotallsolutionsprovidetheanswertoalldatacenter

projects.Crucialcomponentsinrelationtothedatacenterwillbecovered,andbasedoffof

thesecomponentsafunctionaldatacentercanbeachieved.

Theidealcourseofferingregardingdatacenterdesignandnetworkingcoverseachfacetof

thedatacenterfromtheoutsidebrickandmortartotheindividualpatchcableconnectinga

serveruptoapatchpanelwithinarack.Therearemanyreasonsthatacourseshouldcover

allofthesedifferenttopics.Thefirstandforemostreasonisthatwithinadatacenter,all

componentsmeshtogethertocreateafullsolutionthatisaimedatprovidingandprotecting

digitalinformation.Eachdifferentpartofthedatacenterplaysakeyroleinaccomplishing

thattask.Ifonecomponentisarchitectedorimplementedinawaythatdoesnotsuitthe

datacentercorrectly,itcouldcompromisetheintegrityandreliabilityofthatdigital

information.Anotherreasonthatcomprehensivecoverageisimportantisthatwithoutthe

fullunderstandingofhowcomponentswork,anorganizationmaybeunabletocreatea

scalabledesignthatwillfitthegrowthandfinancialneedsiftheorganizationoveraperiod

ofyears.Whilethisresearchwillcoveralargeamountofinformation,itwillnotencompass

thefulldatacenterprojectfromstarttofinishasoutlinedabove.Theprojectwillfollowa

setofdeliverablesthatisshowninfigure1below.Thesedeliverableswillbediscussedinthe

resultssection,butothertopicsinrelationtothecoursewillnotbediscussedforresearch

12

timeconsiderations.ThesedeliverablesarespecifictotheRITenvironmentanddonot

outlinegeneraldeliverablesforallcoursedevelopment.

Deliverables

Comprehensivelistofcoursetopics

ICCform

Textbookchoice

ListofCasestudies

LectureSlidesfor½ofcourse(5weeks)

Figure1

4.2 Research Methodology

Tocompileinformationthatwillserveagoodbasisforacourseofferingfordatacenter

designandnetworking,coverageofkeycomponentsneedstobecoveredtooffer

informationthatstudentscanutilizeasabaselinefortheirunderstanding.Thissectionwill

explaintheprocedurestodeterminethesekeyareasfortheresearch.

Thecoursewillbedesignedaroundbuildinguponthecoreservicethatitwillprovidewhich

isdata.Thenetworkisthefirstcomponentthatshouldbethoughtofduringthedesignof

thedatacenter.Whilethephysicalspecificationsofthedatacenterareindeedimportant,

withoutanetworkdesign,theremainderofthelayoutmustbemadepriortothe

understandingofthedatarequirements.Thephysicalconsiderations,whichincludethesite,

theelectricalsystems,HVACsystems,physicalsecurity,firesuppressionsystemsand

flooring,shouldbedesignedandimplementedutilizingspecificspecialiststhatunderstand

minutedetailsandspecificationsofthosecomponents.Duringthecourse,keypointswillbe

re‐enforcedtoprovidealevelofunderstandingsurroundingthesedifferentareasof

complexinfrastructuredeterminedduringthecoursedevelopment.

13

Itcanbeseenintheparagraphabovethatthereisanimmenseamountofdatatobe

compiledtofullycompleteaprojectofthisscale.Section1makesthestatementthatthere

isalotofinformationavailableregardingthesetopics,andlikewisefulfillingthesolutionof

centralizingthoseresourcesforstudentswillbeastrongfocusoftheresearch.

Astherearemanydifferentcomponentsthatareimplementedintoadatacenter,alltopic

coveragewillbecompiledusingthesamedefinedmethods.Consistencyfortopiccoverage

depthwillalsobeutilizedtoensurethatinformationisportrayedallowingenoughtimein

thecourseforadequatecoverage.Topicdecisionswillbebasedoffofthecoursetextbook

offerings,andindustrystandardcomparisons.TheTelecommunicationsIndustry

Association’s(TIA)documentTIA‐942isanIndustrystandardforDatacenterdesign.Its

topicswillbereviewedasasourceagainstthetextbookforcoursetopics.Eachtopicwillbe

researchedforitsbackground,differenttypesoftechnology,andimplementationintoadata

center,componenthardeningforreliabilityandredundancyandexamplesofthegiven

component.Thisinformationwillbecompiledfromvarioussourcesprovidingacceptable

informationbasedoncomparisonswithothersources,thecoursetextandindustry

standards.Providingadequatecoverageofeachtopiciscrucial,buttoextendthatcoverage

toshowhowthecomponenttiesintothefullschemeofthedatacenterprovidesamore

realisticapproach.Thementionedstepswillbeutilizedforeachcomponentandwillthus

provideequalunderstandingwhetherthecomponentisthemostcriticaloraddseaseof

managementtothedatacenter.

Toprovideabetterunderstandingforthecourseandforstudentstoreferenceinformation

outsideoftheclassroom,acoursetextbookisnecessary.Searchesonlineandthroughlibrary

databaseswillbeutilizedtolookforasolutionthatprovidesthebestcoverageandinsight

intoalloftheresourcesmentionedabove.Thetextbookwillbecomparedtoothersources

fromresearchandbasedonthebestcomparisonthebookwillbechosen.Topicstobe

concernedwitharedepthofknowledgeportrayedinthesource,numberoftopicscovered,

relevancetodatacenters,casestudypresenceorpersonalinsight,anddateofcoverage.

14

4.3 Test System Design

Todiscussandprovideinsightofdifferenttoolsutilizedincertainlecturepresentations,aset

ofsystemswillbecreatedwithtoolsinstalledfordemo.Thetoolsinstalledontothedemo

systemsaretoolsthatarecommonlyutilizedbydatacenteradministratorstoprovideease

ofdutiesonadailybasis.Toolsassistadministratorsdailywithoperations,monitoring,and

recordkeepingofassets,events,andlogicaldataensuringthatreproductionofsystemsand

designscanbecompletedintheeventoflossordisaster.Thesetoolsarealsocommonly

utilizedtoproviderecordkeepingandchangemanagementofthenumeroussystemsthat

arehousedwithinthedatacenter.Toprovideexamplesofthesetoolsduringlectures,a

systemcomprisedofWindowsXPandanotherpartitionwithadistributionofLinuxcalled

CentOS5willbeutilized.ThesesystemswillbebuiltonaVMwareworkstationplatformto

allowfortransparentusageduringlectures.Avirtualplatformallowsforsoftwaretoolsto

remaininstalledwhilechangesaremadetothesystemthatmayjeopardizetheir

functionality.Quickreversioniscapablewhichprovideseasyfixestomodelsoftwareduring

classes.

15

5 Results

Forthisresearch,informationwascompiledandcentralizedtoprovideresourcesfor

studentstounderstanddatacentercomponents,designstepsandnetworking.The

informationwillalsoallowstudentstounderstandcommonserviceswithinthedatacenter

environmentsuchasreplacinganddesigningrackstobeutilizedco‐existentwithother

systems.Studentswillbeabletodesignnetworkingcomponentsfromthelowestrack

cabinetwithinthenetworkhierarchyuptotheserviceproviderlevel.Toprovideadequate

understandingofmaterialscovered,asetofdeliverableswassetforthintheproposalofthis

projectandisprovidedinfigure1.Thesedeliverableswillbediscussedwithinthisresults

section.Thecourseproposedtextbookwasthefirstdeliverablediscussedtoprovidea

locationforstudentstoreferencematerial.Thisdeliverableisdiscussedinsection5.1and

providesthetextbookchosenfromtworeputablesources.Acomprehensivetopiclistwas

alsocreatedtoprovideahighleveloverviewofwhatwouldbecoveredinthiscourse.This

topiclistwillbecoveredinsection5.2andalsocanbefoundinAppendix8.1underthe

coursetopicssection.Aselectionoffivelecturepresentationsispresentedtoprovide

contentcoverageandmaterialexpansioninrelationtothecourseobjectivesandtopics.

Thesepresentationsarecoveredinsection5.3andalsoareshownintheirentiretyin

appendix8.3.Theremaindersofthedeliverablesoutlinedincludehalfofthemidtermand

finalexam,whichisincludedinappendix8.4and8.5.AlsoincludedisthedepartmentalICC

form,whichhasbeencompletedandprovidedasappendix8.2.

5.1 Course Textbook

ThecoursetextbookwaschosentobeBuildtheBestDataCenterFacilityforyourBusiness

byDouglasAlger.ThebookbyDouglasAlgerwascomparedtoEnterpriseDataCenterDesign

andMethodologywrittenbyRobSnevely.BothofthesebookscoverthemajorityofData

Centerdesignandnetworking,butAlger’sbookcoversbetterin‐depthmosttopicsandalso

16

injectspersonalexperienceintoeverychapterofthebook.Snevely’sbookpertainsmoreto

theup‐frontinformationanddoesprovideabetterinsightintonationalbuilding,fireand

electricalcodes.InformationfromRobSnevely’sbookwillbeutilizedintothelecturesand

willbeprovidedasanoptionalreferenceasthebookhasbeenreleasedasafreepublication

fromSunCorporation(Snevely.)

Anotherdifferencebetweenthetwobooksistheusageofdiagrams,mathematical

calculationformulasandcomparisonsbetweentechnologies.Theseareallimportantfactors

whenbuildinganddesigningadatacenter.Theorganizationorcustomerneedsto

determinethecorrectmaterialsandsizesthatwilloptimallyfittheirdatacenter.Differences

betweenproductsmeancostsavingsorincreases,whichultimatelyaffectsthebudgetofthe

datacenter.Instillingtheabilitytoperformthecalculationsanddesignspermitsfaster

designtimesandalsoallowsforverificationofbidsestimatedbythird‐partycontractors.

Thisisanimportantcost‐savingsapproachtoensurethatthesolutionisgearedexactly

towardstheorganizationalrequirementsandgoalsoftheproject.Itmaybefoundthata

solutionrecommendedorputinplansisnotadequatefortheenvironmentorinopposite

cases,ismuchtoolargefortheenvironmentfortheorganization’sgrowthscale.

Diagramsutilizedinthecoursetextbookallowsforstudentstovisualizeconceptsthatare

beingdescribedwithinthechapters.Agivenexamplewouldbeexistentinthedescriptionof

airhandlerplacement.Whilereadingabouttheplacementofanairhandlerperpendicularto

rowsofserverracksprovidesmaybeaninsightastudentmaynothavepreviouslyknown,it

doesmaynotdepictaclearunderstanding.Placingadiagramthenrepresentsanexampleof

theairhandlerandallowsdifferenttypesoflearnerstounderstandthesameconcepts

visually.Alger’sbookutilizesdiagramsmoreeffectivelyforconceptsandmaterialsthat

shouldbeunderstood.Thisprovidesabetterplatformforstudentstounderstandthebasic

conceptsandcomponentsfordatacenterdesignandnetworking.

17

5.2 Topic Design

Thecoursetopicsweredesignedbasedoffofthecoursetextbookandcomparisonswiththe

TIA‐942standard.Whiletherearemultipledifferenttopicsandcomponentsthatmakeupa

datacenter,majorsectionsofthedatacenterwillbecoveredtoprovidethefeasibilityto

supportabasicdatacenter.Thetopicsthatarediscussedindepthinboththeindustry

standardandthebookareshowninfigure2below.Thesetopicswerethemostprevalent

topicsinthetwosourcesandhavebeenreviewedbymanyinindustry.Togetherthese

topicsprovideover80%oftheinfrastructurepresentinadatacenter.Thetopics

themselvescontainsub‐topicsthatallowsforabetterbreakdownofexplanation.Thesesub‐

topicscanbefoundinappendix8.1.

Withthetopicsdetermined,overallcourseobjectivescouldbegintobesetforth.While

manyofthesearegeneralized,theypresentthegoalsthatstudentsshouldbelooking

towardsattheverybeginningofthecourse.Attheendofthecourse,studentsshouldbe

abletofulfillthecourseobjectivesoutlinedinFigure3below.Thesecourseobjectiveswill

aimtoassiststudentsinunderstandingandfocusingonkeycoursematerialofthecourse.

Theyalsoservedtoprovideguidanceduringtheresearchperiodsofthisprojectto

determinewhatpiecesofmaterialwererelativetothecoursedevelopment.

Identifyandunderstanddifferententerprisenetworkfundamentals.Understandthefundamentalprocessesandproceduresofplanning,anddesigningacomplexdatacenterand/orenterprisenetwork.Defineresourceneedsofadatacenterandnetworkneeds.

Cabling/Networking HVAC Electrical Redundancy

Management SiteDesign Security DisasterRecovery

Figure2

18

Describethedifferentcomponentsinanenterprisedatacenterandproviderecommendationstoothersaboutthesecomponents.

Design,plan,manageandimplemententerprisedatacentercomponents.Compareexistingsolutionstodeterminewhatwillfulfillthedefinedneedsofthecustomerororganization.Understandhierarchicalnetworksandtheirimportanceinscalableandsuccessfuldatacentersandenterprisescenarios.Writecleardocumentationthatallowsbettermanagementofimplementedsolutions.

Figure3

19

5.3 Topic Expansion

Thetopicsthatwerechosenwerethendevelopedintocourselectures.Theselecturesare

presentedinawaythatrampsthestudent’sunderstandingofcomponentsfromthe

individualcomponentuptotheirfullintegrationinadatacenter.Asoutlinedinsection4,

thebackgroundofthecomponentiscoveredofeachtopicandthenthedifferentoptionsof

thetechnologyarepresented.Afterthecomponentisdiscussedasitself,thecomponentis

thendiscussedasitsintegrationtothedatacenterandwhyitisimportanttothedata

center.Waysforthecomponenttobemeshedintoahighavailabilityorredundancystate

arediscussedandthensecurityimplicationsinregardstothecomponent.Highavailabilityor

redundancyistheabilityforagivencomponenttoremainonline,stable,andavailablewhen

it’sotherhalforanotherportionofthecomponenthasfailedorisdownformaintenance.

Thistermiscommonlyusedwithindatacentersbecauseofthedemandforservicestobe

available24hoursaday.Manycomponentsarethusdesignedtoprovidesomeformof

redundancyorhighavailability.Thisisafeaturethatisdiscussedwithineachofthetopics

fromFigure1above.Tofinalizeeachcomponent,examplesordiagramsarereferencedto

showthecomponentinanactualsetting.Somecomponentswillbediscussedwithrelation

tocasestudiesandstoriestoassistwithkeypoints.

5.3.1 Network Design

Coveringthedifferenttypesofnetworkingthatoccurswithinthedatacenterandhowto

designanetworktobeutilizedinadatacenterwasthefocusofthislecture.Byfarthedata

networkisthemostcriticalcomponentforserviceoffering,transfersbetweendifferent

systemsandforstorage.Whilethereasonswillbediscussedlaterinthissection,thedata

networkprovidestheabilityforserverstoperformworkandalsotoprovideresults,data,

andservicestoendcustomersand/orothersystems.Datacentersusenetworking

componentssimilartoothertypesofnetworkingenvironments;howeverthedifferencelies

inthehardwaredevicesthatinterconnectsystemsandthewaythatthenetworkhierarchyis

designed.Therearealsotypesofnetworksthatarepredominantlylocatedinthedata

20

center,whichprovideuniqueabilitiestoothersystems.Thesetypesofnetworksare

referredtoasstorageareanetworks.Theremainderofthissectionwilldiscussthislecture

anditscomponents.

Oneofthemostimportanttopicstobediscussedwithinthislectureisthenetworkhierarchy

thatisusedwithinthedatacenter.Thethreetiersthatarefocusedonarethe‘edgelayer,’

the‘distributionlayer,’andthe‘hostoraccesslayer’(Oppenheimer.)Theselayersprovide

meanstosegregatetrafficflowandtoallowresourcestobeprovidedinahighavailability

mode.Discussedwillbethedifferentprotocolsutilizedatthedifferenthierarchiesofthe

design.

TheedgelayerofthenetworkprovidesthefirstlayeredaccessfromtheInternetservice

providertotheinternalnetworkofthedatacenter.Thislayerisutilizedwithprotocolssuch

asbordergatewayprotocol(BGP)andEnhancedInternalGatewayProtocol(EIGRP)to

providefailovertoleranceandalsotoprovidethebasicserviceofroutingtrafficonthe

Internetamongdifferentautonomoussystems(Snevely.)Thelayeralsoprovidesthe

locationwherethemajorityofthedatacenter’sroutersarelocated.Theyusuallyprovide

aggregationfrommultiplevendorsandcouldprovideahigherlinkspeedtoproviders.

Thedistributionlayerallowsnetworktraffictobetransferredtodifferentsub‐networks,

customers,andin‐houserequirementsandtoprovidedifferenttypesofnetworkresource

allocations(Kerr.)Thedistributionlayerisalsotheprimarylayerwheretrafficfiltering

occurs.Thisactsasawaytokeepcertaintrafficfrompollutingthenetworkbandwidthand

alsohelpstolimitbroadcastandmulticastdomains.Thedistributionlayerutilizesamixture

ofroutersandswitchestosegregatedifferenttypesofnetworktraffic,whileallowinglower

branchesofthehierarchydifferentaccesslevelstotheInternetoredgelayerservices.The

distributionlayerisalsoconsideredthegatewaylayerasdifferentrouterscommunicate

withinthislayerutilizingprotocolssuchasOSPF(OpenShortestPathFirst)orRIP(Routing

InformationProtocol)tocommunicatenetworkinformation.Hostswithintheaccesslayer

21

utilizegatewaysinthedistributionlayertoaccessresourcesbasedontheirclassification.

Thelecturecoversthislayeranddenotesdifferenthardwaretypesanddiagramstoshow

howconnectionscouldexist.

Thehostandaccesslayeristhethirdandlowestlayerwithinthenetworkhierarchy.This

layerprovidesdifferentnetworkingcomponentssuchasswitching,hubs,firewalls,and

occasionalrouters.Theprimaryfocusofthislayeristoprovidetheconnectionfromtheend

device,intothedatacenternetwork.Attheaccesslayer,separatenetworksmaybefound

thatdonotconnectsystemstothedistributionlayerandedgelayer.Thesenetworksmay

providestoragefeatures,andtransportfrombackendsystemstosystemsthatconnectto

theaccesslayer.Thesenetworktypesserveastheremainderofthelectureforthatclass

period.

StorageAreaNetworks(SAN)andprivatenetworksmakeupthemajorityofothernetworks

withintheaccesslayerofthenetworkhierarchy.SANnetworksprovidestoragesolutions

overtraditionalnetworkingmediums.Thesestoragesolutionstypicallyaresharedstorage

thatcanallowmultiplesystemstoaccessthesamestorageatthesametime.Systemscan

accessthesestoragecollectionsovertraditionalhardware(NetworkInterfaceCards,Fiber

Cards,HostBusAdapters),whichallowsforlowercostimplementationandredundancy.This

topicwillbediscussedinaseparatelectureinfurtherdetail,butthenetworkportionofthe

technologyshouldbearchitectedinawaythatprovidesmultiple(2ormore)connectionsto

thesystemforredundancyandhighavailability.Privatenetworksprovidedatatransfers

acrossbackendsystemstofront‐endsystemswithoututilizinglimitedbandwidth.These

networksusuallyareimplementedtoprovidebettersegmentationandcanalsobeutilized

formanagementpurposes(Alger.)

Thefinalportionofthiscomponentistodiscusstheneedfornetworkredundancy.

Redundancywithinthedatanetworkcanoccurinmanydifferentways.Itcanoccuronthe

hostlevelwithmultipleportconnectionstoswitchesorhubs,oritcouldbeastransparent

22

asrunningmultiplerouterswithintheedgelayerutilizingtheHSRPprotocol.Althoughthe

HSRP(HotStandbyRouterProtocol)isaCiscoproprietaryprotocol,itcanbeutilizedwith

differenthardwarevendorsandsystemsandcanprovidetransparentrouterredundancy

withinthenetwork.Redundancyincreasestheoverallavailabilityofthenetworkto

customersandserversthemselves.Theoverallgoalistocontinueserviceintheeventone

serviceprovider,onecable,ornetworkportfails.Therearedifferentrequirementsfor

availabilityofthenetworkindifferentdatacentertiers.Thesetierswillbediscussedwithin

depthduringthelectureandwillprovidestudentswiththeunderstandingthatwhatever

datacentertheirorganizationfunctionstheyshouldbeawareofnetworkavailability.

5.3.2 Electrical

Electricalinfrastructureandconfigurationisalsoacrucialcomponentofadatacenterasit

providesthenetworkhardware,servers,HVACsystems,andlightswiththenecessaryenergy

tofunction.Thislecturefocusesonthediscerningofdifferentelectricalspecificationsaswell

astheinfrastructurethatisnecessarytoprovideaconsistent,reliableandscalableelectrical

infrastructureforthedatacenter.Itwaschosenasakeytopicbecauseofitsimportancein

theoveralldesignofthedatacenterandbecauseitisusuallynotfocuseduponby

individualswithnetworkingbackground.Thegoalofthelectureandthistopicistoprovide

studentswiththeabilitytoanalyzeinformationpresentedtothemaboutelectricaldesignin

thedatacenterandtobeabletocalculateneedsforpowerrequirements.Overallthemost

criticaltopicregardinganelectricalinfrastructureistheredundancyandstandbypower

offering.Thisisnottheonlysectionofelectricalinfrastructurethatiscovered;howeveritis

themostcrucialtoprovidinghighavailabilitywithinthedatacenter.Similartoredundancy

inthedatanetwork,electricalredundancyfocusesonprovidingmultiplecircuitriesthatcan

servesystemsanddevicesintheeventofafailedcircuit,electricalproviderorstandby

system.Thisbecomesanimportanttopicwhendiscussingdifferenttypesofelectrical

power.Thereisalargedebatesurroundingalternatinganddirectcurrentelectricalpower

andbecomesforefrontduringthediscussionofstandbyandredundantpower.Thereisa

23

lossofelectricalpowerbetweenACandDCpowerduringconversionwithinuninterruptable

powersupplies(UPS).

Voltageselectionbecomesadebateinmanydatacenterenvironments.Manyexperts

believethatdirectcurrentisabettersolutionthanthattoAC(Rasumussen.)Thislecturewill

coverthedifferencesinbothtypesofelectricalpowerandwilloutlinethebenefitsand

drawbackstoboth.Theclasswillultimatelybeabletodecidewhichtechnologyisbetterfor

thedatacenter.Aslidewithinthelectureshowsthecomparisonofthesetwotechnologies

andthedifferentfinancialimplicationsforeithertechnology.Althoughpreferenceisafactor

inthedecisionprocess,costtotheorganizationorindividualusuallyisthewinningfactor.

Thissectionwillprovideuniquediscussionandwillreflectuponpersonalimplementations

fromstudents,andthebook.

Anothertopiccoveredinthislectureistheabilitytoforecastandprovideadequate

electricityforthedatacenter.Therearemanytoolstoutilizeforthisandmanytoolsranging

fromgovernmentprovidedtoolstoindividualproprietortoolswouldbediscussed.Manyof

thesetoolsutilizekilovoltamperageasthemetrictodetermineneed,asitallowsforan

industrystandardtodevelopsolutions.Theabilityforanorganizationtounderstandtheir

requirementsandforecastneedscanattributetoascalableinfrastructurethatwillprovide

longevityinreturnsfrompurchasedequipment.Scalableinfrastructurealsolowerscost

overallbycreatingmeansforanorganizationtoaddcapacitieswithlittleoverheadcosts.An

exampleofscalabilitycoveredinthelecturesistheadditionofcircuitpanelsthatleaveroom

forextracircuitstobeallocatedforrackspace,networkdevices,andHVACsystems.Whips

aredefinedascablingthatisrunfromcircuitpanelstotheendsystemforconnectiontothe

electricalsystems.Thesewhipsusuallyarecomprisedofasingleordualcircuitthatdoesnot

shareitselectricalloadwithothersystems.Thisusuallyconsumeselectricalcircuitsatafast

ratebecauseofdedicationtoindividualsystems.Theabilitytoforecastandimplementthe

capabilitytocontinuesupportingsinglecircuitwhipswhilemaintaininghealthyelectrical

24

loadsconstitutesgoodforecastingfromadesignstandpoint.Examplessuchastheseare

coveredwithinthelecturestoprovidebetterinsightfortheimportanceofscalability.

Thelastsectionthatisimportantforelectricalsystemswithinadatacenteristheabilityto

sustainpowerintheeventofpowersourcefailure.Therearemanydifferenttypesof

solutionsthatcanprovideadatacenterwithmeansofelectricityintheeventofan

extendedoutage.Someofthesesolutionsproposeutilizingbatterysystemstoensurecritical

systemsremainonlineduringtheoutage.Thesesystemsprovidealimitedextensionof

electricity,asbatterycapacitiesaregenerallynotasextensiveasothersolutions.Ensuring

thatthereisatleastathirty‐minutewindowforadministratorstoprovidetriagetosystems

isagoodmeasurementtoscopebatteryimplementation.Anothersolutionforbackup

electricityisforagenerator.Thesegeneratorsusuallycanprovideelectricalmeansforadata

centertoallowsystemtoremainonlineforhoursandevendays.Manyofthesystemsutilize

typicalfossilfuelsasdiesel,naturalgasorgasolinetogenerateelectricity.Thesesystems

shouldhaveagreementswithfuelprovidersintheareatoguaranteeprovisionsintimesof

outages(TIA‐942.)

Manydatacentersuseacombinationoftheabovesolutionstoprovideafullyredundant

systemthatcanallowthedatacentertoremainindependentofthirdpartelectrical

providers.Whetherthesystemisbatteryorgenerator,itshouldbescaledlargeenoughto

powertheentiredatacenterintheeventofanoutage.Thisassiststoensurethedata

center’sSLAtocustomersandincometotheorganizationisnotlostduetodowntime.

Thesetopicswillbefullycoveredwithinthislecture,andtheabilityforstudentstoreflecton

experienceswilladdtothediscussionofelectricalimportance.

5.3.3 HVAC Systems

Thislecturewilldiscusstheimportanceofheating,ventilationandairconditioning(HVAC)as

itrelatestothedatacenterenvironment.Ensuringthattheserversandotherequipmentof

thedatacenterfunctioninaconstanttemperaturewithcorrectamountsofhumidityisa

25

keycomponent.Fluctuationswithintheenvironmentcancauseequipmentfailuresandin

certainscenarioscausedamagetodata.Becausethisisatopicthatisalsonotcoveredin

depthinmanyoftheothertypesofclassestakenclassicallybyindividualsofnetworking

background,thistopicwaschosenasafocusforalecture.Thegoalistoprovidestudents

withthecapabilitytomakeconsciousdecisionswithintheirdesignplansandtoalsohave

theabilitytocriticallyanalyzedesignsproposedbythirdpartyproviders.Thebiggest

componentwithintheHVACsystemofadatacenteristhecoolingcomponentsasbynature

computersgenerateheat.HandinhandwiththeHVACsystemisthewaythatthedata

centerhandlesairflow.Thisishandledwiththeinstallationofoverheadorunderfloorair

plenums,whichprovidesupplyandreturnairtotheHVACsystems.Theseplenumsarethe

areawithinthedatacenterwhereairflowandpressureoccurstoprovidetheairflow

necessarytoregulatetheenvironmenttemperature.

Inpastgenerationsofhardware,temperaturecontrolwasveryimportantintheoperationof

thedevices.Ifthedevicebecametoowarm,stabilityofthedevicewascompromised

(Sharma.)Thesedeviceshavesincebeenrevisedandimprovedtooperateintemperatures

thatarewarmerandcanallowthesystemstoremainstable.Ithasbeenestablishedin

industrythattemperatureswithintherangesof64degreesto81degreesFahrenheitare

acceptabletemperatures(TIA‐942.)Thisismainlytoprovidecomfortabletemperaturesfor

employeestoperformworkwithinthedatacenterandisnotlimitedbecauseofdevices.

Asmentionedearlier,themostimportantfacetoftheHVACsystemisthecoolingsystem

(Alger.)Therearemanytechnologiesthathavebeencreatedtoensurethatsystemscan

remaincool.Thesesystemsdiscussedthoroughlyinthelectureslidesinappendix8.3.3will

assiststudentsrealizethatnotonlyaretheredifferenttechnologiesavailable,butthat

certaintechnologieswillworkbetterinvariousgeographicallocations.Different

technologieswillalsoofferdifferentlevelsofredundancytoprovidecontinuouscooling

capabilitiestothedatacenter.Somesystemsaswillbediscussed,providecoolingtothe

datacenterthatisfreeandprovidesmonthsworthofcostfreeenvironmenttemperature

26

regulation.Thispresentslargecostsavingstotheorganizationforutilitybillsandpresents

uniquedecisionstobedeterminedbythedesigner.

RedundancywithintheHVACsystemprovidestheabilitytocontinuallyprovideenvironment

regulationintheeventofsystemmaintenanceorsystemfailures.Manysystemscannow

utilizethesamecoolingtowersorcoolingsystemswhilemaintainingseveralbackend

systems.Thisallowsforfailoverwhilestillprovidingthesamelevelofservicetothe

environment.

Asdiscussedearlier,theredifferentdesignstodelivercoolingtothesystemswithinthedata

center.Oneofthesedesignswhichisunderfloorplenumwithreturnairaboveutilizesthe

fanpowerofmanyserverstomoveairfromthegrounduptothereturnductsintheceiling.

Thesedifferentapproachestodelivertheairarediscussedindepthwithinthelectureslides

andprovideexamplesofwhatcanhappenifairisnotdistributedcorrectlyorevenly

throughoutthedatacenter.TomitigatemanyoftheproblemswithHVACsystems,the

lecturealsocoversdifferenttoolsutilizedbydatacenterstolocatepocketsofmismanaged

airandthenmitigatetheproblembychangingairflowdesignswithintheplenum.Thermal

designprovidesthebestwayforsystemstoreceivecooledair;howevertheenddesign

dependsonthefacilitythatisprovided.Therearebenefitsanddrawbackstooverheador

underfloordesigningeneral,butgiventherightscenario,bothsystemdesignscanbe

beneficial.

27

5.3.4 Security

Securityprovidesprotectiontoallsystemsthatsupportandexistwithinthedatacenter.

Therearemanytypesofsecuritywithinthedatacenterrangingfromphysicalsecurityofthe

premisestopersonnelsafetyandaccess.Itwasdeterminedthatbecausethereissuchavast

amountofdatasurroundingthistopic,thatitwouldbeagoodfocustocoverinalecture

aboutitsintegrationintothedatacenter.Thetopicwasalsochosenbecauseitinherently

tiesbackintotheinfrastructuresystemofthedata,whichassistsinitsmanagementofthe

datacenter.Thegoalofthelectureisprovideinsightintothecrucialsystemssurrounding

security.

Physicalsecurityisthemostprevalentfeatureofadatacenter.Inmanyways,datacenters

presentthemselvesassecurelocationsbyensuringhighqualitylockingmechanismsarein

place,hightechnologyaccessvalidationsystemsareputinplaceandsurveillancesystems

areprovidedtodetermaliciousactivityaroundandinternallytothedatacenter.Different

technologieswillbepresentedassolutionstothestudentsandbenefitsofthosesolutions

willbeoutlined.

Heatandwaterpreventionisanotherkeytopicdefinedbysecurityofthedatacenter.

Withoutthesetwopreventionmechanisms,thedatacentercouldbevulnerabletofiresand

flooding.Thesetwohazardspresentuniquechallengestoovercomeinadatacentersetting.

Typicallyfirepreventionsystemsareimplementedbyusingwatersprinklersystemstodouse

fireswhentheyaredetected.Withthelargeamountofelectricalcomponentswithinthe

datacenter,itisdifficulttousewater,asalossofdataisalargerisk.Alternatively,watercan

presentchallengesbecauseitcouldshortcircuitelectricalcomponentsunderraisedflooring

withoutnoise,orwarning.Therearemanysolutionsthatareavailabletopreventbothof

thesehazards.Therearemanysolutionstopreventfiresthatarepresentedinthelecture

slidesinappendix8.3.4.Inverselysystemslikeawaterdetectionsystemareavailableto

preventwaterdamagetoflooringsystemsandelectricalcomponents(Alger207.)Toensure

28

thatthesesystemsareworkingcorrectly,continuoussamplingmustbecompletedtoprotect

thedatacenter.

Thelasttopiccoveredwithinthesecuritylecturewillbetodiscusspersonnelsafetyitems.

Therearemanydifferentitemswithinthissectionthatpertaindirectlytothedatacenter.

Thebiggestofthesesectionsistoprovidepolicymanagementandenforcement.Thesetwo

managementitemsprovideastructuralsensetoenforceruleandprovideinformationto

othersofhowthedatacenterfunctions.Thisisimportantforthirdpartyprovidersto

understandwhentheyenterthefacilityandperformservicesforthedatacenter.Ifpolicies

arenotimplementedandenforced,controlandregulationofresourcesisnotfeasible.These

differentpolicyitemsareincludedwithinthislectureandwillprovidethestudentswiththe

abilitytosafeguardthedatacenterandthosewhoworkdailywithinthefacility.

Anothersectiontobecoveredistheadherencetolocal,stateandfederalbuilding,fireand

electricalcodes.Thesecodesareimplementedtoprotectindividualswhomworkinthese

facilitiesandtoensurethatdevicesandinfrastructureareinstalledwithmaterialswhose

integritywillmatchstressputonthem.Thisinformationwascompiledfromthenationalfire

protectionassociation(NFPA),NFPA70(Snevely.)Thiscodeisalargereferenceforthose

workingwithdatacentercomponentsandelectricalcomponents.Thegoalofthissectionis

toprovidestudentswithresourcestolocatecurrentcodesandregulationsasmanyfacetsof

thedatacenterarestillundercodedevelopment.

5.3.5 Disaster Recovery

Thelastlecturewithinthedeliverablesisalecturesurroundingdisasterrecoveryofthedata

center.Whilethistopicisonethathopefullyorganizationsneverhavetoutilize,itis

imperativethatpreventionmechanismsbeutilizedandputinplacetodefendagainstdata

loss.Becausethistopicisauniquetopicinrelationtoadatacenter,itwasincludedasahigh

focustopictobringawarenesstostudentsofitsimportanceindatacenteroperation.The

goalofthislectureistoprovidestudentswiththeabilitytodesign,implementandmanage

29

disasterrecoveryinfrastructuresthatcanbeutilizedintheeventofaccidentaldatalossor

disaster.

Theprimaryfocusofthelectureistocoverthedifferentsolutionspertainingtobackup

systemsinthedatacenter.Solutionssuchasonsitetapebackupsandoff‐sitestorage

solutionswillbediscussedprovidingthebenefitsanddrawbackstobothsituations.

Solutionscomprisedofremovablemediaotherthanthatoftapesystemswillalsobe

discussedandhowtheyfitintothedatacentertoday.Off‐sitestoragesolutionswillbe

discussedinrelationtotherotationalperiodsinwhichmediashouldbestoredoff‐site.

Discussionstodeterminewhatsolutionsstudentshavetestedwillbeengagedtopromote

informationsharing.

Anotherimportantsituationthatshouldbenotedwithindisasterrecoverysystemsisthe

wayfordatacenterstotestanddevisetestprocedurestoensurethatthesystems

implementedworkinthewaysthatarespecified.Manysystemscanbeimplementedwithin

thedatacentertopreventdisasterfromoccurring,howeverifthesesystemsdonotfunction

thewaythattheyweredevised,theyprovidethedatacenterwithnobenefit.Toassistwith

thesetestprocedures,documentationshouldbestoredinsecurelocationsoncontacts,

emergencypersonnelcontactsandguidelinesonhowtorestorethefunctionalityofthedata

centerintheeventoftotalcatastrophe.Thelecturecoversdifferentfacetsofthedata

centershouldprovideproceduresanddocumentationfortoensurethatstudentscanthen

documentandprovidemeaningfulpreventiontoanydisasterthatcouldoccurwithinthe

datacenter.

30

5.3.6 Tools

Throughoutthecoursemiscellaneoustoolswillbepresentedtothestudentstoshowthe

vastcapabilitiesthattheycanprovidefordailyadministrativefunctionsandthedesigningof

adatacenter.Thesetoolsprovidetherobustabilitytoforecastentiredatacenterpower

requirementstothesimplicityofmanagingIPaddressreservationsandsubnetinformation.

AnexampleofsomeofthetoolsutilizedwithinthecourseisshowninFigure4.Thetoolswill

bedemoedonavirtualsystemdiscussedinsection4.3.Thetoolsthatwillbeshownalsoare

opensourcetoolsorfreewareandcanbedistributedtoprovidefuturereference.The

impactofthetoolsfortheadministratoristhattheywillshortenandsimplifymanagement

tasks.Anexampleofthesetoolsisthepowercalculatorspreadsheetthatisdistributedby

thegovernmenttoassistwithcalculatingenergyneeds.Ratherthantakeinventoryofall

systemsmanually,thischecklisthelpstoestimateandforecastfutureneeds.Thisprovides

aneasierwaytoobtainquotesfromvendorsandwillcuttimetodevelopsolutionswithin

theplanningstagesofthedatacenter.

RackTables ClockingIt.com

PassSafe SharePoint

Nagios IPPlan

DellRackCalculator LinkandSpeed

TestingTools

Figure4

31

6 Conclusion

Datacentersprovidehugeamountsofrevenuefororganizationsandhostvastamountsof

datathatiscriticalfororganizationstofunction.Theirdesignandimplementationisan

importantfeaturefororganizationsthusthisresearchisimportant.Theproblemof

providingacourseofferingforstudentssurroundingdatacenterdesignandnetworkingcan

besolvedbyusingsolutionspresentedbythisresearch.Studentsandorganizationsthat

employstudentswithnetworkingbackgroundcanbenefitfromthiscourseofferingbecause

itpresentsinformationthatallowsthemtodesignandmakedecisionsbasedoffoftheirown

requirements.Informationisplacedwithineasyaccessandprovidespersonalinsightthat

canbeofassistancewhenconfrontedwithtechnologyorrequirementsthatmaynotbe

clear.Further,theresearchcoverstheneedtounderstandallfacetsofthedatacenter.

Whileunderstandingsinglesystemsisanimportantskillsettohave,understandingthe

integrationofmultiplecomponentstogetherallowstheindividualtohavebetterinsightof

howthedatacentermeshestogethertoprovideservices.Allfacetsofthedatacentermust

meshtogethertoprovidetheworkingdatacenter.Ifonesystemfails,theintegrityofthe

datacenterisnolongeroptimal.Thisiswhyredundancyisastressedimplementationwithin

thedatacenter.Withouttheredundancy,serviceswouldnotbeashighlyavailableasmany

hostedserviceshavebecometoday.Focusingontheredundancyandsolidimplementation

ofdatanetworks,electricalinfrastructure,HVACsystems,andsecuritywillprovidethe

meansforanorganizationorstudenttodesignandprovidesolutionsforacustomizeddata

center.

32

7 Future Work

Ithasbecomeatrendinrecentyearstoutilizevirtualcomponentstoprovideabetter

scalableinfrastructurewithbetterserviceprovisioning.Infutureresearch,attentionshould

befocusedontheintegrationofthesevirtualenvironmentsintothedatacenter.Ifthenew

environmentcouldsupportandpresentvirtualredundancyaswellasphysicalredundancy,it

thenwouldpresentanN+1redundancythatcouldgreatlypreventfailuresandbolsterhigh

availability.Virtualtechnologymayalsobeabletotakeadvantageofthemultiplecore

processingtechnologiesthusallowingforhighercondensationofresourcesintoasinglerack

location.Thiswillpresentnewscenariosfordatacenteradministratorstoovercome,asthe

concentrationofresourceswillrequirenewelectrical,HVAC,andnetworkresources.One

couldassumethatvirtualtechnologywouldlowerthestrainondatacenterresources,but

theconversionofagingsystemstonewerconcentratedsystemsmayactuallyincreasethe

requirementsondata,electricalandHVAC.Furthermore,itcouldbeusefultodetermine

howprimarilynetworkresourceswillbeimpactedastheserequirementswillbeeasily

condensedintosystemstoprovidededicatedlinkstovirtualmachinesandtheclusteringof

networkresourcesonsingleresources.

Anewtypeofmodulardatacentersisaimingtoprovidemobilesolutionstofulfillcustomer

needs.Thesetypesofdatacentersincorporateallnecessarycomponentsofthedatacenter

intoonelocationtoprovidevendorswiththeabilityofbuildingadatacenterinacontainer

andthenshippingthatcontainertoalocationthatprovidesmodularconnectivity.Thistopic

couldbeutilizedinfutureexperimentstodeterminethetypesofinfrastructurerequiredto

implementsuchadesignandtoalsotheimplicationitwouldhaveonthetopicsspecifiedby

thisresearch(Hamilton.)

33

8 Appendices

8.1 Course Syllabus

CourseSyllabus4055­883

EnterpriseNetworking200CourseSyllabus

REMINDER: The information presented in this syllabus is subject to expansion, change, or modification during the quarter Instructor: Name Office:Bldg.70‐emailaddress:

Office Hours: M:Orbyappointment.

http://mycourses.rit.edu CourseTextandMaterialsRequiredTextbook:DouglasAlger,BuildtheBestDataCenterFacilityforyourBusiness.CiscoPress,2005.–ISBN:1‐58705‐182‐6OptionalMaterials:RobSnevely,EnterpriseDataCenterDesignandMethodology.SunPress,2002–ISBN:0‐13‐047393‐6ImportantRITDeadlines Last day of add/drop is March 16th. Last day to withdraw with a grade of “W” is May 1st – The deadline for withdrawing from a course with a W grade is the end of the 8th week of the quarter. Forms may be obtained from your department office and need your instructor’s signature. NOTE: The department policy states that a student has one quarter to challenge any grade. After that, grades cannot be challenged. CourseDescription

Thiscoursewillprovidestudentswiththeknowledgeandunderstandingtoapplydesign

34

techniquesandlogicalsegmentationtolarge‐scaleenterprisenetworks.Theoreticalconceptsoflarge‐scalenetworkswillbediscussedandstudentswillcreatedesignsbasedonthistheory.Casestudieswillbeutilizedtohelpunderstanddifferentcomponentsandtheirimportanceintheoverallscheme.Thiscoursewillprovidestudentswiththeknowledgeneededtoapplyavailabletoolsformodelingnetworkfunctionalitytodeterminetheimpactofnetworkinfrastructuremodification,devicereconfiguration,andtheimpactofnewapplicationrollouts.Design/casestudyprojectrequired.CourseObjectives

General

Attheendofthiscourse,studentswillbeabletoeffectivelyuseandimplementtoolsneededforthemanagement,designandcreationofenterprisescalenetworksanddatacenters.Studentswilldevelopprocedurestoensurethattheintegrityofthenetworkisusableattherequiredlevels.

Specifically: 1.Identifyandunderstanddifferententerprisenetworkfundamentals2.Understandthefundamentalprocessesandproceduresofplanning,anddesigningacomplexdatacenterand/orenterprisenetwork.3.Defineresourceneedsofadatacenterandnetworkneeds.4.Describethedifferentcomponentsinanenterprisedatacenterandproviderecommendationstoothersaboutthesecomponents.5.Design,plan,manageandimplemententerprisedatacentercomponents.6.Compareexistingsolutionstodeterminewhatwillfulfillthedefinedneedsofthecustomerororganization.7.Understandhierarchicalnetworksandtheirimportanceinscalableandsuccessfuldatacentersandenterprisescenarios.8.Writecleardocumentationthatallowsbettermanagementofimplementedsolutions.CourseOrganizationMyCourses

The course is organized by using RIT's myCourses platform. You are required to have a DCE account to access myCourses at mycourses.rit.edu. The myCourses is not only used by faculty to organize, create and manage the course activities and course materials, but also by students to communicate with peers and instructors, access the course content, assignments, course grades, group discussion and feedback. myCourses drop boxes are used for the submissions of homework, projects and exams. The instructor will NOT make use of the FirstClass mail and conferencing system. Please check mycourses’ announcement area at least twice a week. CourseCommunication

35

Course communication relies heavily on the content area and the discussion area of myCourses. Course syllabus, outline, lectures, assignments and exams will be posted at the content area. The weekly discussion topics will be posted in the discussion area. Students are highly encouraged to participate in the discussions by posting your comments and/or questions related to that week’s material to that area.

TheinclasslecturesarescheduledonMondaybetween6:00PM‐7:50PMforstudentswhoenrollin4055‐883‐39.TheseinclasslectureswillberecordedduringclassandpostedinMyCoursescontent‐area.However,Istronglyencouragestudentsfromsection90toattendtheselectureseitheronsiteorremotelyviaconnect.rit.eduduringthelecturetimeforliveinteraction.TheURLforattendingtheConnectconferenceswillbepostedeveryweekinMyCoursespriortothelectures.ThecoursemayalsoutilizeateleconferenceusingAdobeConnect’spremiereglobalservicesespeciallyfortheonlinestudents.Theexacttimeofthephoneconferencesalongwiththeinstructionsfordialinginwillbedeterminedandannouncedwhentheclassprogresses.WrittenExamsTheonlinewrittenexamswillincludemultiple‐choicequestions,shortanswers,casestudyanalysisandothercontentformatasappropriate.Theexamsaredesignedtobeopenbook,opennotesandopenreferences.Studentsarerequiredtofinishtheirexamsindependentlyandinthetimeframegiven.IndividualHomeworkandAssignmentsTheinstructorthroughoutthequarterwillassignindividualhomeworkandlabs.Theseassignmentsaretobecompletedindependentlyunlessnoteddifferentlybytheassignment.GroupProjects/PresentationsStudentsmaybeassignedtogroupstoworkonthegroupprojectassignmentsthroughoutthequarter.Thegroupmaymeetanddiscussproblems/issuesusingthediscussionareaand/orchatfunctionofMyCourses.Studentsarerequiredtopresenttheirgroupreportslivethroughtheclassroomorthroughconnect.rit.edu.Eachmemberofthegroupisrequiredtoparticipateinthegroupeventsactively.Peerevaluationswillbeadministeredthroughouttheprojects.Ifthereisaproblemthatappearsonthegroupevaluationswhereamemberisnotcooperatingordoingtheirfairshare,thenthatperson’sgradeonthatportionoftheprojectwillbesubjecttoreductionatthediscretionofyourinstructor.Somakesurethatyouareinvolvedandincontactwithyourothergroupmembers.

Course Topics 1. Principles of Enterprise Networks and Data centers 2. Networking Design 2.1 Cabling design and types

2.2 Network Hierarchy 2.2.1 Edge

2.2.2 Distribution

36

2.2.3 Host/Access 2.3 Network redundancy and fault tolerance

2.4 Storage Area Networks 2.5 High Availability

2.5.1 Time systems 2.5.2 EIGRP

2.5.3 IGP 2.5.4 BGP 3. Physical Design Considerations

3.1 Power 3.1.1 220v/120v

3.1.2 Electrical Segmentation 3.2 Cooling

3.2.1 HVAC systems 3.3 Flooring

3.3.1 Raised 3.3.2 Ground

3.3.3 Weight Factors 3.4 Air/Thermal Design

3.4.2 Hot/Cold Design 3.4.3 Hot/Hot Design

3.4.4 Air Sampling 4. Disaster Planning / Recovery

4.1 Backup systems 4.1.1 Onsite storage

4.1.1.1 Tape and Disk Backup systems 4.1.1.2 Removable media

4.1.2 Offsite storage 4.1.2.1 Long-term storage solutions 4.1.3 Test procedures

4.1.3.1 Policy development 4.1.4 Redundancy

4.1.4.1 Electrical 4.1.4.2 Data

4.1.4.2.1 Multiple ISP 4.1.4.2.2 Protocols (BGP, eBGP) 4.1.4.3 Storage

4.1.4.3.1 Storage Area Networks 5. Security 5.1 Physical Security

5.1.1 Surveillance 5.1.2 Safeguard systems

5.1.3 Access control systems 5.2 Heat/Water Prevention

5.2.1 Fire Suppression 5.2.2 HVAC/Air Quality control

5.3.3 Water/Moisture prevention 5.3 Personnel Safety

5.3.1 Policy management & enforcement 5.3.2 Fire codes

5.3.3 Building codes 5.3.4 Access control management 6. Management

6.1 Remote Management

37

Component Weight

Discussions 10FinalExam 25Midtermexam 25FinalPresentation 25Assignments 15

Range Grade

>=90.0% A>=80.0%&<90%

B

>=70.0%&<80.0%

C

>=60.0%&<70.0%

D

<60.0% F

6.1.1 Nagios or Network Monitoring 6.1.2 Microsoft Operations Manager/System Center Operations Manager

6.1.3 SMS and E-mail Alerting 6.2 Onsite Management

6.2.1 Organizational Tools 6.2.1.1 Rack Tables

6.2.1.2 IP Plan / Spreadsheets 6.2.2 Server and environment monitoring

6.2.2.1 Service health monitors 6.3 Scalability

6.3.1 Growth capacity planning 6.3.2 Cost Recovery modeling

7. Future Technologies Grading

Thegradingscaleusedalongwiththegradingcriteriaisasfollows:

TentativeCourseScheduleWeek Topics/Exams AssignedReading Activities1‐2 Introduction to Data Centers and

Enterprise Networks

- Network Design - Segmentation - Capacity Planning

Textbook:Chapter1,2&7Lecturenotesandonlinematerial/articles

MyCourses:IntroductionAssignment1(Week2)

3 Physical Design Considerations - Site planning

Text book: Chapter 3,4 Notes, Tools, Slides

MyCoursesDiscussion#1

4 Physical Design Considerations - Components

Text book: Chapters 5 &6

MyCourses Discussion #2

5 Security - Physical & Environmental

Text book: Chapter 13,14 Notes, Tools, Slides

Assignment 1 Due Midterm Exam

38

6 Security/Disaster Recovery - Personnel - Onsite Systems

Notes, Tools, Slides Assignment 2 MyCourses: Discussion #3

7 Disaster Recovery - Proactive vs. Reactive

Text book: Chapter 8, 9 Slidesandonline

MyCourses:Discussion#4

8 Management‐RemoteandOnsiteManagement

Textbook:Chapters10,11,15

MyCourses:Discussion#5

9 Management/FutureTechnologies

OnlineArticles Assignment2Due(endofweek9)

10 StudentTopics‐Presentations

Text book: Chapter 12

ReviewforfinalExam

11 FinalExam

Cheating Policy: Please review the departmental policy on cheating as described at http://www.it.rit.edu/policies/dishonesty.html Student Responsibilities: Please review the general student responsibilities as outlined at http://www.nssa.rit.edu/~netsyslab/Responsibilities.htm

39

8.2 Course ICC Form

B. Thomas Golisano College of Computing and Information Sciences

Department of Networking, Security, and System Administration

REVISED COURSE: 4055-883 1.0 Title: Enterprise Networking Date: 2/25/2009 Credit Hours: 4 Prerequisite(s): N/A Co-requisite(s): N/A Course proposed by: Chris Mahood 2.0 Course information:

Contact hours Maximum

students/section Classroom 2 12 Lab Active Learning/Active Learning Extended Other (specify DL) Online delivery 12

Quarter(s) offered Fall X Winter X Spring X Summer Students required to offer this course: None Students who might elect to take the course: Matriculated students in the M.S. in Networking, Security & Systems Administration.

3.0 Goals of the course: Today’s large organizations heavily rely on data services, high availability, reliability and scalability to deliver products and services to their customers. As the online audience continues to grow, more of these organizations are realizing that they must convert existing offerings into digital and online services to enhance their profits and exposure. The challenge many of these organizations face is that they either do not have an existing infrastructure that can host the services necessary, or their existing infrastructures can not accommodate the growth in technology. In the past, many organizations retrofitted rooms within their existing infrastructures to accommodate servers and network equipment. These rooms were usually

40

small and held a few computers or servers. These challenges have been met by enterprise data centers that are wholly owned, built and designed by organizations, or met by enterprise data centers that are owned by third party companies whom rent space to other organizations. Data centers present a unique difference in the way that resources or networks are traditionally allocated. This course will explore the numerous facets of a data center along with the important design steps necessary to provide a smooth and reliable operation. At the end of this course, students will be able to describe multiple different technologies used within a data center and will also be capable of designing hierarchical networks that can interact with multiple systems. This course will consist of a combination of lectures, discussions, case study analysis, and a field trip to a dynamic collocation datacenter. 4.0 Course description: This course will provide students with the knowledge and understanding to apply design techniques and logical segmentation to large-scale enterprise networks and data centers. Theoretical concepts of large-scale networks will be discussed and students will create designs based on this theory. Case studies will be utilized to help understand different components and their importance in the overall scheme of a data center and enterprise network. This course will provide students with the knowledge needed to apply available tools for designing an effective enterprise network that may be implemented in a data center. The course will also present different technologies available currently for usage in a datacenter as well as some emerging technologies aimed towards the enterprise. Design/case study project required.

Class 4, Credit 4

5.0 Possible resources (texts, references, computer packages, etc.) 5.1 Douglas, Alger. Build the Best Data Center Facility for your Business. Cisco

Press, 2005. ISBN: 1-58705-182-6. 5.2 Snevely, Rob. Enterprise Data Center Design and Methodology. Sun

Microsystems Press, 2002. ISBN: 0-13-047393-6. 6.0 Topics (outline) 6.1 Principles of Enterprise Networks and Data centers 6.2 Networking Design 6.2.1 Cabling design and types

6.2.2 Network Hierarchy 6.2.3 Network redundancy and fault tolerance

6.2.4 Storage Area Networks 6.2.5 High Availability 6.3 Physical Design Considerations

6.3.1 Power 6.3.2 Cooling 6.3.3 Flooring

41

6.3.4 Air/Thermal Design 6.4 Disaster Planning / Recovery

6.4.1 Backup systems 6.5 Security 6.5.1 Physical Security

6.5.2 Heat/Water Prevention 6.5.3 Personnel Safety 6.6 Management

6.6.1 Remote Management 6.6.2 Onsite Management 6.6.3 Scalability

6.7 Future & Emerging Technologies 7.0 Intended learning outcomes and associated assessment methods of those

outcomes 7.1 Identify and understand different enterprise network fundamentals 7.2 Understand the fundamental processes and procedures of planning, and designing a complex data center and/or enterprise network. 7.3 Define resource needs of a data center and network needs. 7.4 Describe the different components in an enterprise data center and provide recommendations to others about these components.

7.5 Compare existing solutions to determine what will fulfill the defined needs of the customer or organization. 7.6 Fully understand hierarchical networks and their importance in scalable and successful data centers or enterprise scenarios.

8.0 Program outcomes supported by this course

8.1 Program Objective 1: Design, deploy, and manage data center environments to meet the goals of an organization or customer.

8.2 Program Objective 2: Communicate and document clear information that can be used throughout an enterprise organization.

9.0 Other relevant information

None 10.0 Supplemental information None

42

Approvals: Date

Date

43

8.3 Course Lecture Slides 8.3.1 Week 2 Slides

44

45

46

47

48

49

50

51

52

8.3.2 Week 3 Slides

53

54

55

56

57

58

59

60

8.3.3 Week 4 Slides

61

62

63

64

65

66

67

68

69

70

71

8.3.4 Week 5 Slides

72

73

74

75

76

77

78

79

80

8.3.5 Week 7 Slides

81

82

83

84

85

86

87

88

89

8.4 Midterm Exam Example Questions

4055-883 Data Center Design and Networking Midterm Exam Questions

1. Please describe the hierarchy commonly referred to in data center network design.

Explain activities that occur at those layers, their function and types of devices that exist at these layers.

2. This type of cabling design provides a hierarchy of cabling:

a. Cat 5e b. Distributed c. Aggregation d. Direct e. None of the above

3. _____________ provides the ability for systems to remain online while another

portion of the network may be failed.

4. This type of fiber optic cable provides faster data transmission speeds over longer distances.

a. Multimode b. Single mode c. Mega mode d. Dual pair

5. Scenario: The organization you work for has purchased new switches for the data

center that run only on DC power. Your existing switches utilize AC power and you do not have any DC power available within the data center. Provide a solution to utilize these switches within the data center and provide justification for your answer.

6. Name and explain two benefits to utilizing DC power within the data center.

7. Name and explain two benefits to utilizing AC power within the data center.

90

8. Scenario: The organization that you are designing a data center for is comparing raised floor versus flat floor installation. The maximum height of the raised floor that they could install would be 17 inches due to building code. Based off of this information please provide a recommendation for raised flooring or flat flooring. Discuss reasons for your choice and reasons why the other choice was not made.

9. Ensure that UPS devices or PDUs are not utilized above _____ percent

10. Which of the following cooling technologies would benefit a data center in Rochester

the best? a. Air Cooled b. Water Cooled c. Glycol d. Fan Based e. None of the above

11. Normal humidity operating range is from 20% to ____% humidity

12. An air __________ is an enclosed space utilized in data centers to provide airflow.

91

8.5 Midterm Exam Example Questions

4055-883 Data Center Design and Networking

Final Exam Questions

1. Scenario: You are consulting for an organization where the finalization of security designs is occurring for a new data center and the organization is looking to you to provide recommendations on physical security. The organization is a web hosting organization that processes credit card payments through a series of e-commerce sites. The organization would like to implement a maximum of three (3) different physical security technology solutions to provide access control over not only the data center in general, but to the server room as well. Please provide three different recommendations of physical security to protect the payment card systems and justify the reasons to recommending these systems.

2. What is usually the most susceptible location to physical break-ins of a data center:

a. Back door b. Front door c. Loading dock d. Roof access e. None of the above

3. _____________ prevent someone from propping a door open and allowing free access

to a server room or maintenance room.

4. This tool can be utilized to check air flow patterns and where hot pockets are forming within the data center.

a. Radar gun b. Manometer c. Multimeter d. Bilometer

5. Scenario: In the data center you work within, your building monitoring system

continually warns of smoke particles in the environment, however all systems have been tested to be clean. This occurs frequently when other employees are located within the server room or maintenance is being performed. Discuss what steps should be taken to remedy this situation. Justify your answers.

92

6. Name two techniques to prevent water damage to the data center.

7. Which of the following organizations provides a standard for the protection of computer equipment?

a. National Electrical Contractors Association b. Institute of Electrical and Electronics Engineers c. National Fire Protection Association d. Telecommunications Industry Association e. Electronics Industries Alliance f. None of the above g. All of the above

8. This protocol can provide redundancy and failover capabilities at the edge layer

a. OSPF b. RIP c. BGP d. IGP e. None of the above

9. This type of tape library system has historically provided the most interior space with

the least footprint in the data center a. Helical b. Linear c. Tower d. Hexagon e. None of the above

10. Scenario: A data center is being implemented in New York City, NY. The

organization also owns an existing data center located in Seattle, WA. The organization wants to create a site replication solution to provide critical files and data base backups to the existing data center in Seattle. Explain two different solutions that the organization may utilize to connect and transfer files of these types to the remote location and how they can easily replicate their data center at this location. You can assume that their topology at both locations will be identical and that no configuration of systems will be necessary.

93

9 References: "80PlusProgram."The80PLUSProgram.13Jan.2009<http://80plus.org/80what.htm>."AC,DCandElectricSignals."ElectronicsClub.Ed.JohnHewes.4June2007.13Jan.2009

<http://www.kpsec.freeuk.com>.Al‐Fares,Mohammad,AlexanderLoukissas,andAminVahdat."AScalable,

CommodityDataCenterNetworkArchitecture."Thesis.AScalable,CommodityDataCenterNetworkArchitectureACM(2008):63‐74.

Alger,Douglas.BuildtheBestDataCenterFacilityforYourBusiness.NewYork: CiscoPress,2005.Arregoces,Mauricio,andMaurizioPortolani.DataCenterFundamentals.NewYork: CiscoP,2003.Berkowitz,HowardC."DefininganArchitecture."DesigningRoutingandSwitching ArchitecturesforEnterpriseNetworks.ByHowardC.Berkowitz.GrandRapids: NewRiders,1998.Blazek,Michele,HuiminChong,WoonsienLoh,andJonathanG.Koomey.DataCenters

Revistited:AssessmentoftheEnergyImpactofRetrofitsandTechnologyTrendsinaHigh‐DensityComputingFacility.Thesis.ASCE,2004.JournalofInfrastructureSystems,2004.

Caldwell,Chris."PowerSupplies:MakingtheLeaptoHigherEfficiency."Presentationto APEC.Texas,Austin.4Feb.2009.Creswell,JohnW.ResearchDesign.ThousandOaks:SAGEPublications,2003.DataCenterEnergyBenchmarkingCaseStudy.Oakland:LawrenceBerkelyNational Laboratory,2007. "DataCenterSecurity."ePaymentsNetwork.20Jan.2009.20Jan.2009

<http://www.epaymentsnetwork.com/DataCenter/tabid/124/Default.aspx>.DiMinico,Chris."TelecommunicationsInfrastructureStandardforDataCenters."TelecommunicationsInfrastructureStandardforDataCenters.25Jan.2009.IEEE.Dunlop,Jodi."DataCenterCaseStudiesFocusofASHRAEBook."ASHRAE.7May2008.

ASHRAE.20Jan.2009<http://www.ashrae.org/pressroom/detail/16715>.

94

Hamilton,JamesR.AnArchitectureforModularDataCenters.Thesis.Microsoft,2008.

Redmond:MicrosoftCorporation,2008.IEEE.13Jan.2009<http://www.ieee802.org/3/hssg/public/nov06/diminico_01_1106.pdf>.

Kerr,Devon.NetworkHierarchy.Kovacs,Brian.DataCenterOutsourcingCaseStudy.Rep.Carmel,IN:NFrame,2004."MainframeHistory."MainframeHistory.IBM.4Feb.2009<http://www‐

03.ibm.com/ibm/history/exhibits/mainframe/mainframe_intro.html>.Mitchell,RobertL."CaseStudy:WellsFargo'sFreeDataCenterCoolingSystem."Computer

World.5Nov.2007.4Feb.2009<http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=299301&pageNumber=2>.

Patrizio,Andy."CleanYourDatacenter,CutYourElectricBill."InternetNews.29Jan.2009.4

Feb.2009<www.internetnews.com/hardware/article.php/3799451>.Rasmussen,Neil.ACvs.DCPowerDistributionforDataCeters.Publication.American

PowerConversion,2007.Oppenheimer,Priscilla.Top‐downNetworkDesign.NewYork:CiscoP,2004.GoogleBooks.Google.12Jan.2009<http://books.google.com/books?id=YKWfuGGSmXMC&pg=PA141&lpg=PA141&dq=access+layer+hierarchical&source=web&ots=HklxUBvIep&sig=M8h96f9fK_9rBLmt‐nb4sWZvVCE&hl=en&ei=Q5qPSfW4NJ6DtweQxtGDCw&sa=X&oi=book_result&resnum=2&ct=result#PPA143,M1>.Sharma,RatneshK.,RockyShih,CullenBash,andChandrakantPatel."OnBuilding NextGenerationDataCenters."ACM(2008).Snevely,Rob.EnterpriseDataCenterDesignandMethodology.UpperSaddleRiver: PrenticeHallPTR,2002.Sun,Jimeng,EvanHoke,JohnD.Strunk,GregoryR.Ganger,andChristos

Faloutsos."IntelligentSystemMonitoringonLargeClusters."Thesis.IntelligentSystemMonitoringonLargeClustersACM(2007):47‐52.

TIA‐942DataCenterStandardsOverview.Publication.Minneapolis:ADC,2006.

"WhatisNetworkCabling."WhatisNetworkCabling.14May2008.UniversityofSouthernFlorida.12Jan.2009<http://fcit.usf.edu/network/chap4/chap4.htm>.