BOSH deploys distributed systems, and Diego runs any containers
-
Upload
benjamin-gandon -
Category
Technology
-
view
203 -
download
0
Transcript of BOSH deploys distributed systems, and Diego runs any containers
![Page 1: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/1.jpg)
BOSH&
DiegoBenjamin Gandon
@BenjGa
2016-04-04
![Page 2: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/2.jpg)
@BenjGa
Founder & CEO @ Gstack.io
Formerly: CTO @ Alvarum.com
Benjamin Gandon
BOSH
At:
We do:
And:
![Page 3: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/3.jpg)
BOSHDeploys Distributed Systems
Argh! It’s again those Americans! Don’t worry, Herr General, we will find out soon!
![Page 4: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/4.jpg)
What is BOSH
BOSH is a flexible and resilient technology that ………………………
complete ecosystems of distributed software.
1. Package2. Deploy3. Monitor4. Manage
(start/stop, scale, resurrect, update, …)
5. Upgrade
![Page 5: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/5.jpg)
Instance in a BOSH Deployment
Container (or VM)
Filesystem + BOSH Agent
NginxHAproxy Pcre
HAproxy Nginx
Stemcell
Packages
Jobs
Instance Stemcell
+ Jobs
+ Packages
= Instance
Equation
/var/vcap/jobs/*
/var/vcap/packages/*
![Page 6: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/6.jpg)
Availability Zone #3
AZ2AZ1
What we do with those “instances”
HAproxy
Stemcell
Pkg1 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
HAproxy
Stemcell
Pkg1 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
![Page 7: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/7.jpg)
● Garden containers (in a VitrualBox VM)● Docker (experimental)● AWS● OpenStack● CloudStack● Azure● vSphere
BOSH secret? → Use abstract Cloud Provider Interfaces (CPIs)
Deploy where ?
![Page 8: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/8.jpg)
DiegoFlexible Container Engine
From
![Page 9: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/9.jpg)
Try at home! Because…
● BOSH
● Cloud Foundry
● Diego
Open Source
![Page 10: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/10.jpg)
Why “diego”?
● Since 2019, intuition for stateless “not-yet-containers” → the Droplets
● Their engine: the Droplet Execution Agent
→ DEA
● When rewritten in Go: DEA-GO
→ Diego!
![Page 11: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/11.jpg)
Diego architecture
(logs)
(controller)
(http routes)
Source: https://github.com/cloudfoundry-incubator/diego-design-notes
![Page 12: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/12.jpg)
Not just about containers
● Long Running Processes
→ LRPs
● Tasks
→ Servers
→ Cron jobs
![Page 13: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/13.jpg)
Two ways to use Diego
● Multi-tenant Diego
→ as Cloud Foundry central container engine
● Single-tenant Diego
→ as Lattice, a single-tenant Diego cluster
Dude, go to http://lattice.cf
![Page 14: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/14.jpg)
Lattice is definitely cool
● Containerized workloads○ Long running, or temporary tasks○ Dynamically scaled○ Dynamically balanced across cells (but no live re-balancing)
● Cluster scheduler● HTTP load balancing● Log aggregation● Health management
![Page 15: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/15.jpg)
Some code!
![Page 16: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/16.jpg)
Diego Auctions Scoring
// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L15-L24type CellState struct {
RootFSProviders RootFSProvidersAvailableResources ResourcesTotalResources ResourcesLRPs []LRPTasks []TaskStartingContainerCount intZone stringEvacuating boolVolumeDrivers []string
}// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L74-L79func (c CellState) ComputeScore(res *Resource, startingContainerWeight float64) float64 {
remainingResources := c.AvailableResources.Copy()remainingResources.Subtract(res)startingContainerScore := float64(c.StartingContainerCount) * startingContainerWeightreturn remainingResources.ComputeScore(&c.TotalResources) + startingContainerScore
}// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L90-L93type Resources struct {
MemoryMB int32DiskMB int32Containers int
}// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L110-L115func (r *Resources) ComputeScore(total *Resources) float64 {
fractionUsedMemory := 1.0 - float64(r.MemoryMB)/float64(total.MemoryMB)fractionUsedDisk := 1.0 - float64(r.DiskMB)/float64(total.DiskMB)fractionUsedContainers := 1.0 - float64(r.Containers)/float64(total.Containers)return (fractionUsedMemory + fractionUsedDisk + fractionUsedContainers) / 3.0
}
![Page 17: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/17.jpg)
// See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {var winnerCell *CellwinnerScore := 1e20
// Lists all zones, and count how many times the LRP is already in each of themzones := accumulateZonesByInstances(s.zones, lrpAuction.ProcessGuid)
// Retain only zones with compatible filesystem (e.g. Linux or Windows)filteredZones := filterZones(zones, lrpAuction)
if len(filteredZones) == 0 {return nil, auctiontypes.ErrorCellMismatch
}
sortedZones := sortZonesByInstances(filteredZones)
// …
}
Diego Auctions Evaluation (1/3)
![Page 18: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/18.jpg)
// See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {// …for zoneIndex, lrpByZone := range sortedZones {
for _, cell := range lrpByZone.zone {score, err := cell.ScoreForLRP(&lrpAuction.LRP, s.startingContainerWeight)if err != nil {
continue}
// Look for the Diego cell with minimal overall utilizationif score < winnerScore {
winnerScore = scorewinnerCell = cell
}}
// if (not last zone) && (this zone has the same # of instances as the next sorted zone)// acts as a tie breakerif zoneIndex+1 < len(sortedZones) &&
lrpByZone.instances == sortedZones[zoneIndex+1].instances {continue
}
if winnerCell != nil {break
}}// …
}
Diego Auctions Evaluation (2/3)
![Page 19: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/19.jpg)
// See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {// …
if winnerCell == nil {return nil, rep.ErrorInsufficientResources
}
err := winnerCell.ReserveLRP(&lrpAuction.LRP)if err != nil {
s.logger.Error("lrp-failed-to-reserve-cell", err, lager.Data{"cell-guid": winnerCell.Guid, "lrp-guid": lrpAuction.Identifier()})return nil, err
}
winningAuction := lrpAuction.Copy()
// Return a copy of the auction containing the winning cellwinningAuction.Winner = winnerCell.Guidreturn &winningAuction, nil
}
Diego Auctions Evaluation (3/3)
→ Simple as that!
![Page 20: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/20.jpg)
Thank you!
![Page 21: BOSH deploys distributed systems, and Diego runs any containers](https://reader034.fdocuments.in/reader034/viewer/2022042906/58a79d0a1a28ab03128b483b/html5/thumbnails/21.jpg)
www.gstack.io (not yet in Google)
www.bosh.iowww.lattice.cfwww.cloudfoundry.org
Questions?↘ ↙
→ @BenjGa ←↗ ↖