Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web...
Transcript of Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web...
![Page 1: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/1.jpg)
Orchestra)ngCloudNa)veApplica)onsatScaleintheEnterprise
CameronBrunnerArchitect–CloudNa)veProjects
![Page 2: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/2.jpg)
CloudNa)veCompu)ngFounda)on
Mission:Tocreateanddrivetheadop)onofanewcompu)ngparadigmthatisop)mizedformoderndistributedsystemsenvironments.Thepar)cipantsbelievethatsystemsarchitectedwillbe:• Containerpackaged• Dynamicallymanaged• Micro-servicesoriented
![Page 3: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/3.jpg)
• Highperformancecompu)ng/technicalcompu)ng• Extremelylargeclustersperformingcomplexcomputeworkloads
• GenomeSequencing,FluidDynamics,RiskModeling,OilandGasExplora)on,SemiconductorDesignandTes)ng,VoiceAnalysis,BigData
Foundingmember
![Page 4: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/4.jpg)
OurCustomers
Data Services Energy Government Financial Life Sciences Manufacturing / Technology
![Page 5: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/5.jpg)
BrownFieldversusGreenField
• Complex• Poli)cal• Highrisk
• Cleanslate• Perfectformicro-services
![Page 6: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/6.jpg)
TheEnterpriseisbrownfield!
![Page 7: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/7.jpg)
7
UseCase• Scien)ficdataanaly)csforgenomesequencediscovery
- Massivedataanalysisàlargecluster- Life-scienceanalysisstandardizedonGridEngine
• Clusterisasharedresource- Manyusers- Advancedpolicies,e.g.,fair-sharing,back-filling,anddependableresourcecontrols
- Advancedjobtypes,e.g.,arrayjobs- Detailedaccoun)ngandbillingforresourceconsump)on
![Page 8: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/8.jpg)
Challenges• Sandboxing–maintainmanyproduc)onenvironmentsforalong)me• Atminimalornoperformanceimpact:
- Fromrunningapplica)onsinacontainer- Fromnetworkandsharedfilesystemaccessfromwithinacontainer- Fromstar)ngthesamecontainersoverandoveronnodes- Avoidtoreloadimages
ChallengesandSolu)on
8
Solu5on• CRGNexXlowworkflowmanagement• IntegratedwithUnivaGridEngine• IntegratedwithDocker• MakeUnivaGridEngineDocker-aware
- EnableDockerjobs- Containerimagecache-awarescheduling
![Page 9: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/9.jpg)
• 4%increaseofDockerapplica)onrun)mevsna)verun)mewithcachedimages
• 12.5%increasewithcontainerbootstraping,i.e.,downloadingfromimagerepository- Imagecacheawareschedulinghassolidbenefitonu)liza)on
andthroughput• CostisconsideredlowversusbenefitbyCRG
- Usecaserequirementsreallycan'tbesa)sfiedwithoutcontainers
Results
9
![Page 10: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/10.jpg)
EnterpriseContainerAdop)on–Challenges
• Storage– Containersaregenerallyephemeral...andsoisthestorage– Enterprisesolu)onsnotalwaysperformantinthecloud(i.e.,NFS)
• Networking– Earlydaysforcontainernetworking.S)llevolving.
• Security– Isola)onnotatrobustasvirtualmachines– Poten)alforthreatsinpubliccontainerrepositories– SecurityissuesnotaswellunderstoodasothermorematureareasofIT...
nothardenedthroughwidespreaduse– ContainerBreakouts,DoSacacks,KernelExploits,CompromisedSecrets
• BusinessProcesses– Complexbusinessprocess,policies,mul)pleusergroups...allcreate
complexityandchallengeswhenarchitec)ngandorchestra)ngcontainerizedworkloads
– Runningcontainerwithnon-containerworkloadscanbetricky
![Page 11: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/11.jpg)
Naviga)ngtheEnterprise
• Abridgefromthebrownfields– Orchestra)on
• UnivaGridEngine–ContainerEdi)on,ApacheMesos– Storage
• NFS,Gluster,Ceph,Flocker– Networking
• Virtualswitches• Op)onsincludeFlannel,Weave,DockerNetworking,Calico,OpenVswitch,OpenContrail
– Configura)onManagement• Ansible,Puppet,Salt,• Orsimplifywithanimmutableopera)ngsystems(CoreOS,Atomic...)
![Page 12: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/12.jpg)
Kubernetes
• Aclusteringsolu)onformanagingcontainersinamul)-nodeenvironment– Masters,Nodes,Pods,Replica)onControllers,Services,Labels,Namespaces
• Providescontainergrouping,healing,scaling,andloadbalancing
• NowpartofCNCFwithmul)plecontributorsincludingRedHat,Google,Meteor,Huawei,CoreOS...
![Page 13: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/13.jpg)
DemoOverview
1.Willdemonstratescalingofworkloadonanon-premisecluster2.Whentheclusterissaturated,newnodeswillbeautoma)callyprovisionedinthecloud.3.ThosenodeswillthenbeaddedtotheKubernetesclusterandwillstarttotakeonworkload.4.Asoverallworkloadisthenreduced,thecloudmachinewillberemovedfromtheK8sclusteranddeprovisioned.
![Page 14: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/14.jpg)
Demo
![Page 15: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/15.jpg)
LocalNodes GCPNodes
NavopsInstaller
KubernetesMaster
KubernetesNode
VPNServerInstance
KubernetsNodeInstance
NavopsContainers VPN
Kubernetes
Navops
HTTPClients WebApp
Pods
NavopsRuleEngine
LocalKubernetes
Node
GCEKubernetes
Nodes
Monitor Scale
ThedemoconsistsofahybridinstallwithalocalKubernetesMaster,alocalKubernetesNode,and0toNGCEKubernetesNodes.
Navopsmonitorstheloadofthewebapplica)onandscalestheKubernetesclustertomeetdemandbyburs)ngintoGCE.
DemoRecap
![Page 16: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/16.jpg)
Solving the Set Up Problem – In Minutes
“Implementing Docker and Kubernetes is
complicated.
It’s even more complicated for enterprises that have diverse workloads, various app stacks, heterogeneous infrastructures, and limited resources.”
InfoWorld May 2015
![Page 17: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/17.jpg)
17
![Page 18: Orchestrang Cloud Nave Applicaons at Scale in the Enterprise · Kubernetes Navops HTTP Clients Web App Pods NavopsRule Engine Local Kubernetes Node GCE Kubernetes Nodes Monitor Scale](https://reader033.fdocuments.in/reader033/viewer/2022052718/5f05e5647e708231d415429c/html5/thumbnails/18.jpg)
18
Cloud On Premises Servers / VMs
After
“The next step is large scale orchestration and scale.” 451 Research
Before
Containers and other workloads need resources
• Run containers at scale
• Blend containers with other workloads
• Maximize resources / use of cloud
Navops orchestration solution