Nutanix Complete ClusterA Technical Whitepaper
NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Table of Contents
ExecutiveSummary............................................................................... 3
Introduction........................................................................................... 4
LimitationsofCurrentArchitecture........................................... 4
TheGoogleApproach................................................................. 4
NutanixCompleteClusterArchitecture.............................................. 6
MainPillars.................................................................................. 6
ArchitectureOverview................................................................ 7
NutanixScale-OutConvergedStorage–KeyComponents...... 8
KeyFeatures........................................................................................ 11
CapacityOptimization.............................................................. 11
EaseofManagement................................................................ 11
PerformanceandScalability..................................................... 12
HighAvailability........................................................................ 13
Conclusion........................................................................................... 15
EX
EC
UT
IVE
SU
MM
AR
Y
3NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Executive SummaryStorageisthebiggestchallengeinvirtualizeddatacenterstoday.Thenetwork
storagearchitecturedesignedfifteenyearsagoforphysicalserversistoo
expensiveandcomplexforvirtualmachines.Withoutanenterprise-class
alternative,organizationsareforcedtousetraditionalsolutions,whichcan’t
keepupwithvirtualmachinesthataredynamic,growrapidlyinnumberand
continuetodemandnewlevelsofperformanceandcapacity.
NutanixCompleteClusterisascale-outcomputeandstorageinfrastructure
thatallowsorganizationstovirtualizetheirdatacenterswithoutrequiring
networkstorage(SANorNAS).Builtfromthegroundupforvirtualmachines,it
providescompletecomputeandstoragecapabilitiesalongwithenterprise-class
performance,scalability,availabilityanddatamanagementfeatures.Itleverages
industry-standardhardwarecomponents,solid-statedrivesandmarket-leading
hypervisorstoprovideanout-of-the-boxsolutionthatmakesvirtualization
extremelyeasyandcosteffective.
INT
RO
DU
CT
ION
4NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
IntroductionLimitations of Current Architecture
Organizationsarebuildingtheirvirtualizationinfrastructureusingthe
traditionalservers-connected-to-storage-over-a-networkarchitecture,which
can’tadapttotheever-changingdemandsofvirtualization.Inadditiontoslow
performance,networkstoragehasbecomethesinglebiggestsourceofcostand
complexityinvirtualizedenvironments.Thenetworkstorage-basedarchitecture
workedwellforphysicalserversthatservedrelativelystaticworkloads.
Virtualization,andnowCloudComputing,hasmadedatacentersextremely
dynamic;virtualmachinesarecreatedonthefly,movefromservertoserverand
dependheavilyonsharedresources.Thesecharacteristicsmakethemanagement
ofvirtualmachinesandtheirunderlyingphysicalinfrastructureextremely
complex.
Datavolumesaregrowingatarapidpaceinthedatacenter,thankstotheease
ofcreatingnewVMs.Intheenterprise,newinitiativeslikedesktopvirtualization
contributetothistrend.Serviceprovidersdealwithanevenlargernumberof
VMsastheybuilddatacenterstoservecustomerswhocan’taffordthecost
andmanagementoverheadthatvirtualizationrequires.Thisgrowingpoolof
VMsisexertingtremendouscost,performanceandmanageabilitypressureon
thetraditionalarchitecturethatconnectscomputetostorageoveramulti-hop
network.
Theriseofsolid-statedrivesisanothertrendthatisrapidlywideningthegap
betweencomputeandstoragetiers.UseofSSDsthatare100X-1000Xfaster
thantraditionalharddiskswillmaketheexistingnetworkbottlenecksand
networkcomplexityevenworse,ifvirtualmachinesneedtoaccessthemovera
network.ManySAN/NASvendorsareaddingSSDstotheirsolutions,charginga
heftypremiumforthesedrivesandrequiringadditionalinvestmentsinnetwork
bandwidthtoaccessanalreadyexpensivetierofstorage.
The Google Approach
Googleandotherleadingcloud-generationcompaniessuchasAmazon,Yahoo
andMicrosoft(Azure)realizedthatanetwork-storagebasedapproachwould
INT
RO
DU
CT
ION
5NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
notworkfortheirdatacenters.Theybuiltsoftwaretechnology(suchasGoogle
FileSystem)thatcouldgluealargenumberofcommodityserverswithlocal
storageintoasinglecluster.ThisapproachallowedGoogletobuildaconverged
computeandstorageinfrastructurethatusedcommodityserverswithlocal
storageasitsbuildingblock.GoogleFileSystemrunsacrossaclusterofservers
andcreatesasinglepooloflocalstoragethatcanbeseamlesslyaccessedby
applicationsrunningonanyserverinthecluster.Itprovideshighavailability
toapplicationsbymaskingfailuresofharddisksandevencompleteservers.
GoogleFileSystemallowedGoogletobuilddatacenterswithmassivelyscalable
computeandstorage,withoutincurringthecostsandperformancelimitations
associatedwithnetworkstorage.
Nutanixhastakenasimilarscale-outapproachtobuildanenterprise-ready
computeandstorageinfrastructurethatisdesignedfromthegroundupfor
virtualmachines.
NU
TA
NIX
CO
MP
LE
TE
CL
US
TE
RA
RC
HIT
EC
TU
RE
6NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Nutanix Complete Cluster Architecture Main Pillars
NutanixCompleteClusterwasdesignedfromscratchtosolvestoragechallenges
forvirtualmachinesbybuildingasystemthatleveragesthelatestadvancesin
systemarchitecture,hardwareandsoftwaretechnologies.Therearethreecore
pillarsoftheNutanixarchitecture:
Distributed Computing
TheNutanixarchitectureissimilartoGoogle’sarchitectureinthatitisascale-
outcomputeandstorageinfrastructurethateliminatestheneedfornetwork
storage.Atthesametime,NutanixbuildsuponGoogle’sarchitectureand
providesanenterprise-classsolution.WhileGoogleFileSystemisacustom
solutionthatworksforGoogle’sinternalapplications(search,Gmail,etc.),
Nutanixprovidesageneral-purposesolutionforvirtualizedenvironments.In
additiontoitsscale-outcapabilities,ithasthesameorbetterenterprise-class
datamanagementfeaturesthatarecommonlyprovidedbyadvancednetwork
storagesolutions,includinghighavailability,backup,snapshots,anddisaster
recovery.
Virtualization
TheNutanixarchitecturewasdesignedforvirtualmachinessoitsupportsall
hypervisorfunctionsthataresupportedbythetraditionalnetwork-storage
basedarchitecture,includingliveVMmigrationandhighavailability.In
addition,becausetheNutanixarchitectureisVM-aware,itovercomeslimitations
oftraditionalsolutionsthatwereoptimizedtoworkwithphysicalservers.For
example,whilethingsaremanagedonaperVMbasisoncomputeside,theunit
ofmanagementonstoragehastraditionallybeenaLUN.WhenaLUNisshared
bymanyVMs,itbecomesmoredifficulttoperformstorageoperationssuchas
backup,recovery,andsnapshotsonaper-VMbasis.Itisalsodifficulttoidentify
performancebottlenecksinaheavily-sharedenvironmentduetothechasm
betweencomputingandstoragetiers.TheNutanixarchitectureovercomesthese
limitations.
NU
TA
NIX
CO
MP
LE
TE
CL
US
TE
RA
RC
HIT
EC
TU
RE
7NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Solid-State Drives
TheNutanixarchitecturewasdesignedtotakeadvantageofenterprise-grade
solid-statedrives(SSDs).Itisimportanttonotethatthetraditionalstorage
systemsweredesignedforspinningmediaanditishardforthemtoleverage
SSDsefficientlyduetotheentirelydifferentaccesspatternsthatSSDsprovide.
Whileharddiskshavetodealwiththerotationandseeklatencies,SSDsdo
nothavesuchmechanicallimitations.Thisdifferencebetweenthetwomedia
requiresthesoftwaretobeoptimizeddifferentlyforperformance.Onecannot
simplytakesoftwarewrittenforharddisk-basedsystemsandhopetouseit
efficientlyonsolid-statedrives.TheNutanixarchitectureusesSSDstostorea
varietyoffrequently-accesseddata,fromVMmetadatatoprimarydatastorage,
bothinadistributedcacheforhigh-performanceandinpersistentstoragefor
quickretrieval.TomaximizetheperformancebenefitsofusingSSDs,theNutanix
architecture:
• ReservesSSDsforI/O-intensivefunctions
• Includesspace-savingtechniquesthatallowlargeamountsoflogicaldatato
bestoredinasmallphysicalspace
• Migrates“cold”orinfrequently-useddatatoharddiskdrivesautomatically,
allowsadministratorstobypassSSDsforlow-priorityVMs
Architecture Overview
NutanixCompleteClusterisascale-outclusterofhigh-performancenodes,
orservers,eachrunningastandardhypervisorandthatcontainsprocessors,
memoryandlocalstorage,includingSSDs)andharddiskdrives.Eachnode
runsvirtualmachinesjustlikeastandardvirtualmachinehost.Inaddition,
localstoragefromallnodesisvirtualizedintoaunifiedpoolbyNutanixScale-
outConvergedStorage(SOCS)(Figure1).Ineffect,SOCSactslikeanadvanced
SANthatuseslocalSSDsanddisksfromallnodestostorevirtualmachinedata.
VirtualmachinesrunningontheclusterwritedatatoSOCSasiftheywere
writingtoaSAN.SOCSisVM-awareandprovidesadvanceddatamanagement
features.Itbringsdataclosertovirtualmachinesbystoringthedatalocallyon
thesystem,resultinginhigherperformanceatalowercost.NutanixComplete
Clustercanhorizontallyscalefromafewnodestoalargenumberofnodes,
enablingorganizationstoscaletheirinfrastructureastheirneedsgrow.
NU
TA
NIX
CO
MP
LE
TE
CL
US
TE
RA
RC
HIT
EC
TU
RE
8NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Server 1 Server 2 ..............
FusionIO HDDs
Virtual Machine / Virtual Disk
Server N
N u t a n i x S c a l e - O u t C o n v e r g e d S t o r a g e ( S O C S )
Figure 1: Nutanix Scale-Out Converged Storage Architecture
WithNutanixCompleteCluster,allvirtualizationfeatures,includinghigh
availabilityandliveVMmigration,continuetoworkseamlessly.Administrators
createvirtualmachinesonNutanixCompleteClusterusingtheirstandard
processes.NutanixSOCSprovidesstorageforthesevirtualmachinesintheform
ofvirtualdisks,orvDisks,whicharestandardiSCSIdevices.
Nutanix Scale-Out Converged Storage – Key Components
ThekeytotheNutanixarchitectureisSOCS–apatent-pendingscale-out
convergedstoragelayerthathasthefollowinguniquesetofcapabilities:
• Itisconvergedwiththecomputelayer.VMsandSOCSco-existonthesame
cluster.
• ItisVM-aware.SOCSprovisionsstorageonaper-VMbasisandcanidentifyI/O
comingfromeachVM.
• Itcanscaleoutfromafewnodestoalargenumberofnodes.
• Ithasground-upintegrationwithsolid-statedrives.
• Itprovideshighavailabilityagainstdiskornodefailures.
• ItprovideshighperformancebymakingI/Oaccesslocal,leveragingsolid-state
drivesandemployingaseriesofpatent-pendingperformanceoptimizations.
• Itprovidesuniquecapacityoptimizationcapabilities.
SOCSisenabledbythefollowingcomponents:
NU
TA
NIX
CO
MP
LE
TE
CL
US
TE
RA
RC
HIT
EC
TU
RE
9NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
n-Way Controller Cluster
WhiletraditionalSANsolutionstypicallyhave1,2,4or8controllers,ann-node
NutanixCompleteClusterhasncontrollers(Figure2).EverynodeonNutanix
CompleteClusterrunsaspecialvirtualmachine,calledaControllerVM.This
virtualmachineactsasavirtualcontrollerforSOCS.AllControllerVMsin
theclustercommunicatewitheachothertoformasingledistributedsystem.
UnliketraditionalSAN/NASsolutionsthatarelimitedtoasmallnumberoffixed
controllers,thisarchitecturecontinuestoscaleasmorenodesareadded.
Node 1 Node 2 ..............
SSDs HDDs
Virtual Machine / Virtual Disk
Node N
Control ler VM Control ler VM Control ler VM...... . . . . . . . .
.... ............ .... ............ .... ............
Figure 2: Nutanix Scale-Out Controller VM Architecture
Heat-Optimized Tiering Cache (HOTcache)
HOTcacheisahighperformancecachebackedbySSDsfromeachnodeina
cluster.WhenguestVMswritedata,thatdataisfirstwrittentoHOTcacheand
then,inthebackground,itisflushedtoSOCS.HOTcacheusesasequentialdata
layouttoprovidehighperformanceevenifworkloadsfromVMsgetmixedinto
arandomworkload.HOTcachekeepsonedatacopyonalocalSSDandanother
copyonadifferentnodesothatthereisnodataloss,eveninthecaseofanode
failure.
Distributed Metadata Service (Medusa)
Intraditionalstoragesystems,controllersoftenbecomeabottleneckasmore
storageisadded.Oneofthemainreasonsforthisissueisthatthestorage
metadataisstoredonthecontrollers.Traditionalsystemsscaletoasmall
numberofcontrollers,soasthenumberofVMsincreases,sodoestheI/Oload
oneachcontroller.Nutanix’sdistributedmetadataservice,Medusa,distributes
NU
TA
NIX
CO
MP
LE
TE
CL
US
TE
RA
RC
HIT
EC
TU
RE
10NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
theclustermetadataacrosstheclusterforscalabilityandreplicatesthedata
onmultiplenodesforfaulttolerance.Theserviceishighlyavailableandcan
toleratemultiplemodulefailures.Incomparison,traditionalsolutionsthat
dependonafewstoragecontrollerscannottoleratemultiplecontrollerfailures.
Distributed Data Maintenance Service (Curator)
Nutanix’sdistributeddatamaintenanceservice,Curator,isaMapReduce1
-basedframeworkforexecutingbackgrounddatamanagementoperationsina
massivelyparallelmanner.Suchoperationsinclude:
• Migrationofcolddatatolowertiers(forHeat-OptimizedTiering)
• Garbagecollectionofdatathathasbeendeleted
• Dataconsistencythroughroutinechecksums
• Replicationofdataincaseofnodeordiskfailures
• Re-balancingofdatawhennodesareaddedorremoved
• MigrationofdatatomaximizelocalaccesswhenaVMmovesfromonenode
toanother
FlashStore
FlashStoreisthepersistent,flash-basedstorageprovidedbythepoolingof
SSDsfromallnodesinthecluster.DataisfirstwrittentoFlashStoreandthen
movedofftoDiskStore,asitbecomescold.Ascolddatabecomeshotagain,
itisbroughtbackintoFlashStore.ForvDisksthatservelowpriorityVMs,
administratorshaveanoptiontoskipFlashStore.
DiskStore
DiskStoreisthehigh-capacitySATAstoragetierspreadacrossthecluster.
DiskStoreprovideslargestoragecapacityforcolddata.Nutanix’sHeat-
OptimizedTiering(HOT)combinesFlashStoreandDiskStoretoprovidehigh
performanceaswellashighcapacityatalowercost.
1 MapReduceisatechnologyoriginallybuiltatGoogleformassivelyparallelanalysisofdatainacluster
KE
YF
EA
TU
RE
S
11NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Key FeaturesNutanixCompleteClusternotonlyeliminatestheneedforaSAN,butitalso
providestopofthelineavailability,performanceanddatamanagement
features.Someofthekeyfeaturesaredescribedbelow.
Capacity OptimizationNutanix QuickClone
Nutanixenablesadministratorstorapidlydeploynewvirtualmachinesbyusing
itsQuickClonefeature.QuickClonesarewriteablesnapshotsthatbehavejust
likestandardvDisks-administratorscanattachthemtoaVM,writedataon
them,andevensnapshotthemfurther.Thisisusefulfordeployingnewvirtual
desktops,creatingtestanddevelopmentcopiesofaproductiondatabaseand
anyotherscenariorequiringcloneswithoutduplicatingthedata.Thesystem
alsosupportsread-onlysnapshotsforbackuppurposes.
Nutanix Thin Provisioning
Storageforvirtualmachinesisthinlyprovisionedinthesystem.Administrators
cansetthecapacityofavDiskbutphysicalstorageisallocatedonlywhen
required.Administratorscanalsosetaminimumreservationparameterthat
guaranteesthespecifiedamountofstorageforacollectionofvDisks.
Nutanix Converged Backup
TheConvergedBackupfeatureprovidesinstantbackupandrecoverycapabilities
forvDisks.Severalmonths’worthofbackupscanbekeptinsidetheappliance
withoutrequiringexternalbackupstorage.Whenrecoveryisnecessary,
administratorscaninstantaneouslyrestoreavDisktoanyofitspastbackups.The
appliancealsosupportsoffsitebackupsusingstandardthird-partytools.
Ease of ManagementEase of Deployment
NutanixCompleteClusterisaplug-and-playsolutionthatincludesallhardware
andsoftwarenecessarytorunalargenumberofvirtualserversorvirtual
desktops.AdministratorscansetitupandstartcreatingVMsinamatterof
minutes.
KE
YF
EA
TU
RE
S
12NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Next-Generation User Interface
NutanixCommandCenterisahighlyintuitiveFlex-baseduserinterface
thatprovidesadministratorcompletevisibilityacrosscomputeandstorage
resourcesinthecluster.Itenablesthemtotroubleshootissuesrelatedtoa
virtualmachineseasilybymappingeachVMtothephysicalresourcesinthe
system.NutanixCompleteClusteralsoprovidesacommand-lineinterfacefor
management.
Nutanix Scale-out Converged Storage (SOCS)
NutanixSOCSeliminatestheneedformanagingacomplexnetwork-basedstorage
infrastructure,makingiteasytomanagevirtualenvironmentsatanyscale.
Conformance to IT Standards
WhileNutanixCompleteClusterenablesaconvergedarchitecture,itcontinues
tosupportstandardtoolsandinterfacesthatITdepartmentsalreadyuse.For
example,thenodesrunanindustry-standardhypervisor(VMwareESXi)andall
ITprocessesandsoftwaretoolsthatworkwiththishypervisorcontinuetowork
withNutanix.Similarly,vDisksarestandardiSCSIdevicesthatareconnected
toVMsusingastandardiSCSIinitiatorinthehypervisor.Byleveragingsuch
standardinterfaces,NutanixCompleteClustercanseamlesslyfitintoanexisting
ITecosystem.
Performance and ScalabilitySolid-State Drives
NutanixCompleteClusterwasdesignedwithSSDsinmind.Theincludedserver-
basedSSDsprovidehigherperformancethanSAN-basedSSDsbecausetheyavoid
thenetworkbottleneck.Traditionalsystemshaveoftenbeenlimitedbythe
amountofmetadatatheycankeepinacontroller’scache.InNutanixComplete
Cluster,SSDsareusednotonlyforVMdata,butalsoforstoringSOCSmetadata
forfastaccess.KeepingrichmetadataonSSDsenablesSOCStoprovide
advanceddatamanagementcapabilities.Giventhescale-outarchitecture,SSD
capacityinthesystemgrowsasmorenodesareaddedtothecluster.
KE
YF
EA
TU
RE
S
13NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Nutanix Heat-Optimize Tiering (HOT)
IntheNutanixcluster,avDiskcanbeallocatedamixofSSDandHDDcapacity.
Toensurethatonlyhigh-valuedatastaysonSSDs,SOCSmovescolddatatohigh-
capacitySATAdrivesinthebackgroundusingitsHOTfeature.
Scalability
NutanixCompleteClusterisdesignedtoscalefromafewnodestoalarge
numberofnodes.Everyaspectofthesystemwasdesignedwiththescalability
requirementsoftoday’svirtualizeddatacentersinmind.Forexample,there
isnocentralizedmetadatamasterinthesystem.Themetadatalayeritself
isdistributedacrossthecluster,eliminatingacommonbottleneckfoundin
mostscale-outsystems.Also,witheverymodulerunningaSOCSController,
thenumberof“controllers”inthesystemcanbemuchhigherthanatypical
network-basedstoragesolutionwithonlyafewcontrollers.Suchdesign
innovationsenablethesystemtostartsmallandscalemassively.
High AvailabilityNutanix Cluster RAID
NutanixCompleteClusterisahighlyavailablescale-outsystemwithnosingle
pointoffailure.UsingNutanixClusterRAID,dataisstripedacrossdiskswithina
nodeforhighperformanceandreplicatedacrosstheclusterforhighavailability.
Thisprovideshighavailabilityforvirtualmachinesevenifdisksorcomplete
nodesfail.
Nutanix Distributed Metadata Service (Medusa)
Nutanix’sdistributedmetadataservice,Medusa,distributestheclustermetadata
acrosstheclusterforscalabilityandreplicatesthedataonmultiplenodesfor
faulttolerance.Theserviceishighlyavailableandcantoleratemultiplenode
failures.Incomparison,traditionalsolutionsthatdependonafewstorage
controllerscannottoleratemultiplefailures.
Nutanix Instant vDisk Motion
TheabilitytomigratealiveVMfromonehosttoanotherisaverypowerful
featureprovidedbytheindustry-standardhypervisors.Sofar,organizations
havebeenabletoleveragesuchlivemigrationcapabilitiesonlywithnetwork
KE
YF
EA
TU
RE
S
14NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
storage.Infact,manyorganizationswereforcedtousenetworkstorageonly
toachievelivemigrationandhighavailability.InNutanixCompleteCluster,live
VMmigrationissupportedevenwiththeconvergedarchitecturethatuseslocal
storage.vDisksintheappliancearelogicalentitiesthatarefullymobile.Thisis
enabledbyNutanixInstantvDiskMotionfeaturethatcanquicklymoveavDisk
fromonenodetoanother,whennecessary.
Backup and Recovery
Theabilitytoperformoff-sitedatabackupandrecoveryiskeytoan
organization’sdataprotectionstrategy.NutanixCompleteClusterisfully
compatiblewiththeVMwarevStorageAPIforDataProtection(VADP),and
providesfullyfunctionalbackupandrecoveryofvirtualmachines,through
integrationwithVADPcompatiblebackupandrecoveryproducts.
Disaster Recovery
DisasterRecoveryiskeytothebusinesscontinuityneedsofanenterprise.
NutanixCompleteClusterprovidesfailoverandfailbackcapabilitiesthrough
integrationwithVADPcompatibleDisasterRecoveryproducts.
CO
NC
LU
SIO
N
15NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
ConclusionNutanixCompleteClusterisaninnovativesystemthateliminatestheneedfor
networkstoragewithoutcompromisingtherequirementsofenterpriseIT.It
isbuiltusinganext-generationscale-outarchitecturethathasbeenproven
atsomeofthemostinnovativecloud-generationcompanies.WithNutanix
CompleteCluster,organizationscanbuildacomputeandstorageinfrastructure
forvirtualmachinesthatishighlyavailable,fast,richindatamanagement
featuresandcangrowastheirneedsgrow.
Top Related