VM-Cloud management software -...
Transcript of VM-Cloud management software -...
![Page 1: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/1.jpg)
VM-Cloud management softwarePablo Moreno
![Page 2: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/2.jpg)
EBI-EMBASSYOpenStack
![Page 3: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/3.jpg)
![Page 4: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/4.jpg)
![Page 5: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/5.jpg)
![Page 6: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/6.jpg)
Internal network:internal ports 80, 8080 and 22 allowedproxy
wordpress jenkins
w4mwordpress
volume
jenkins volume
floating ip: ports 80 & 22 (EBI only) allowed
![Page 7: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/7.jpg)
Architechture
![Page 8: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/8.jpg)
VM OS
cont
. too
l 1
cont
. too
l 2
cont
. too
l N
...
OS orchestration
inst
alle
d to
ols
Cloud framework
![Page 9: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/9.jpg)
Create/download image
Set hardware
Build for target (i.e. amazon)
Orchestrate provisions
Run in target
Vagrant packer cloud-init ansible docker
Orchestrate VMs
kubernetes
Orchestrate Services
![Page 10: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/10.jpg)
Timeline
PackingOS Download Deployment Running
● packer○ ansible○ cloud-init○ shell
● cloud-init● shell
● ansible● kubernetes
(on amazon, openstack, vagrant, vm-ware, etc.)
single_image_file.vmdk
![Page 11: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/11.jpg)
Vagrant vs packerVagrant
● Normally a single target● Few targets available● Aim: run the VMs locally● Best for development phase of
provisioning● Prepackaged base boxes.
Packer
● Multiple target (“builders”).● Plenty of targets available● Aim: produce VM-images to deploy● Once provisioning development done, use
to deploy in production.● Can create its own base boxes.
![Page 12: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/12.jpg)
Pac
ker e
xam
ple
image_for_qemu.img
image_for_vbox.img
image_for_aws.ami
docker_export.tar
![Page 13: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/13.jpg)
Ansible vs cloud-initAnsible
1. Declarative syntax for lots of modules, plus shell commands
2. Requires ssh access3. Agent-less (but requires python!)4. Configuration by roles
Cloud-init
1. Declarative syntax for a few modules, otherwise run shell commands
2. Console access, injected on the appliance3. Requires cloud-init installed.4. No roles
$ docker run <image> <argument_to_entrypoint>
![Page 14: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/14.jpg)
CoreOS● Minimal installation● Automatic updates.● Cluster awareness.● Read-only root file system.
![Page 15: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/15.jpg)
Minimal OS difficulties
Welcome to CoreOS 845!$ which apt-get$ which python$ which myFavouriteTool$ which docker/usr/sbin/docker
![Page 16: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/16.jpg)
docker best practices : directory
![Page 17: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/17.jpg)
docker best practices: do you really need them all?
![Page 18: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/18.jpg)
docker best practices: one container, one process
FROM ubuntuENTRYPOINT [“top”,“-b”]CMD [“-c”]
$ docker run -it -rm top_container -H
![Page 19: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/19.jpg)
docker best practices: more● # of layers balance● how to group commands● linking deprecated -- use networking● etc.
http://docs.docker.com/engine/articles/dockerfile_best-practices/
https://github.com/BioDocker/biodocker/wiki/Best-Practices
![Page 20: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/20.jpg)
Kubernetes● Container cluster orchestration
Data flow problem?
![Page 21: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/21.jpg)
Kubernetes : services abstraction
http://kubernetes.io/v1.0/docs/user-guide/services.html
![Page 22: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/22.jpg)
Stateless vs Stateful
chromatograms aligned chromatograms
new pipeline galaxy + new pipeline
![Page 23: VM-Cloud management software - PhenoMeNalphenomenal-h2020.eu/home/wp-content/uploads/2015/10/Cloud... · Vagrant packer cloud-init ansible docker Orchestrate VMs kubernetes Orchestrate](https://reader035.fdocuments.in/reader035/viewer/2022070615/5c3b66ed93f3c37a634630d1/html5/thumbnails/23.jpg)
Our architecture?
VM: Ubuntu LTS
cont
. too
l 1
cont
. too
l 2
cont
. too
l N
...
OS orchestration
inst
alle
d to
ols
Orchestration? At what level?container deployment?communication between nodes?VM deployment to multiple targets?
VM: Minimal OS
cont
. too
l 1
cont
. too
l 2
cont
. too
l N
...
Cloud framework Cloud framework