BigTop vm and docker provisioner
Transcript of BigTop vm and docker provisioner
![Page 1: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/1.jpg)
BigTopVM and Docker
ProvisionerEvans Ye
2015.2.17
![Page 2: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/2.jpg)
Apache Bigtop is a project for…
• Packaging• Packages Hadoop ecosystem
• Testing• integration/interoperability testing framework
• Virtualization• vagrant/docker based hadoop provisioning
• Deployment• puppet recipes for included components
![Page 3: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/3.jpg)
What does vagrant-puppet do?
![Page 4: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/4.jpg)
1) Create a set of VMs by vagrant box
2/17/2015
Vagrant Box(Centos, Ubuntu,…)
VirtualboxVMs
![Page 5: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/5.jpg)
2) Install puppet agents
2/17/2015
![Page 6: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/6.jpg)
3) Deploy Hadoop by bigtop puppet (masterless)
2/17/2015
puppet apply -d--modulepath="bigtop-deploy/puppet/modules:/etc/puppet/modules" \bigtop-deploy/puppet/manifests/site.pp
puppet apply…
puppet apply…
puppet apply…
![Page 7: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/7.jpg)
One click auto deployment
2/17/2015
$ cd bigtop-deploy/vm/vagrant-puppet-vm$ vagrant up
![Page 8: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/8.jpg)
And the docker-puppet?
![Page 9: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/9.jpg)
Similar to vagrant-puppetbut spin up a cluster on top of Linux containers
2/17/2015
$ cd bigtop-deploy/vm/vagrant-puppet-docker$ ./docker-hadoop.sh --build-image --create 3
![Page 10: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/10.jpg)
Some prerequisites (see each README.md)vagrant-puppet docker-puppet
2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2
![Page 11: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/11.jpg)
Use cases
• for BigTop• to do bigtop integration tests
• to test bigtop puppet recipes
• integrates packaging, deployment, and testing all into one package(demonstrate the BigTop life cycle)
• for Contributors & Users• run round trip test locally (from source code to a testing cluster)
• easy-to-get Hadoop cluster environment
2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2
![Page 12: BigTop vm and docker provisioner](https://reader034.fdocuments.in/reader034/viewer/2022042518/55a684e81a28ab30498b4885/html5/thumbnails/12.jpg)
Execute BigTop smoke-tests
• vagrant-puppet• README.md
• vagrantconfig.yaml• set run_smoke_tests: true
• Vagrantfile
• docker-puppet• ./docker-hadoop.sh --smoke-tests
2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2