Continuous Integration and Deployment using OpenStack

download Continuous Integration and Deployment using  OpenStack

of 29

  • date post

  • Category


  • view

  • download


Embed Size (px)


Continuous Integration and Deployment using OpenStack. Miguel Zuniga @ mikezuniga About ebay inc. eBay Inc. enables commerce by delivering flexible and scalable solutions that foster merchant growth. - PowerPoint PPT Presentation

Transcript of Continuous Integration and Deployment using OpenStack

Headline In Arial Regular 32pt, Title Case

Continuous Integration and Deployment using OpenStackMay 5, 2014Miguel

eBay Inc. enables commerce by delivering flexible and scalable solutions that foster merchant growth.

About ebay inc

With 145 million active buyers globally, eBay is one of the world's largest online marketplaces, where practically anyone can buy and sell practically anything.

With 148 million registered accounts in 193 markets and 26 currencies around the world, PayPal enables global commerce, processing almost 8 million payments every day.

eBay Enterprise is a leading provider of commerce technologies, omnichannel operations and marketing solutions. It serves 1000 retailers and brands.AgendaEnter Fluo (CI/CD)Fluo FlowFluo Code ReplicationPackages, Artifacts, Distribution of PackagesInfrastructure as CodeDeployment to ProductionScreenshotsRoadmap

Continuous Integration and Deployment using OpenStack3Enter Fluo (CI/CD)From begin to end how code goes to deployment.At the beginningBasic system runningGerritJenkinsWeb server


Single InterfaceScalableSimple to useDeveloper FriendlyGeneric

Continuous Integration and Deployment using OpenStack5Fluo and the CICD configuration fileWhat is Fluo?WebUI and APISingle pane of glassCloud instance prov and decommAllows users to configure the systemRBAC

How it works?Jenkins through APIGerrit through APIUses ZMQ to communicate with workers which do non-API actionsRecycle slaves or provisions them on the fly at the moment of job execution.

What is the cicd configuration file?Specification and requirements filesHow the developers instruct the system what to do (run unittests, how to build packages, install dependencies, more).

ExampleContinuous Integration and Deployment using OpenStack6Fluo cicd configuration filelanguage: bashpackage_install:apt:- build-essential- ruby1.9- ruby1.9-dev- mysql-server- mysql-devgem: - rails -v=4.0.3- zmq- mysql2before_review:- /etc/init.d/mysql-server start- mysql -u root < database.sql- RAILS_ENV=test rake db:migrate- RAILS_ENV=test rake db:test:loadreview_script:- rake test test/models/user_test.rb- rake test test/models/session_test.rb- rake test test/models/post_test.rbnotification:email:- myemailuser1@email.com7Continuous Integration and Deployment using OpenStackFluoContinuous Integration and Deployment using OpenStack8ComponentsArchitectureFluo (App/worker/coordinator)Cloud for provisioning instancesGerritJenkinsZuulGalera (MySQL)Zabbix (or any other monitoring)Puppet (or any other CM tool)MrepoRsync

Fluo FlowA step by step journey of a code change.Basic WorkflowContinuous Integration and Deployment using OpenStack10Fluo basic workflow has 6 different stages/stepReviewApprovalBuildIntegrationReleasePeriodic

At each stage/step Fluo will:Build a cloud instanceRead the configuration file Execute the scripts definedReport back statusDestroy the instance

Customs workflows can be created by users.

DeveloperFluoCommit and request reviewExecute Review ScriptsApprove the code changeExecute Approval ScriptsMerge and replicate codeExecute Build ScriptsAdds Run Integration to comment history on specific changeExecute Integration ScriptsTags a specific commitExecute Release ScriptsAt XY time of dayExecute Periodic ScriptsCode Quality = Must have Review, Approval, BuildContinuous Integration and Deployment using OpenStack11The rest of the flow Integration Release Periodic.Fluo Code ReplicationOne project 2 projects Multiple projects.. Best Practices.Github, Internal Github and your own Git repository.Some points to consider:

Categorize your codeWill it be open sourced?Is it private for our team?Is it private for our company?Do we need mirrors?

Always use ssh keys

StandardsUser which will be replicatingReplicate Branches or maybe not

Continuous Integration and Deployment using OpenStack13Packages, Artifacts, How do you want to wrap it out? RPM? Debs? Tar?When and where do I create my packages?Things to consider.Jenkins slaves will build the packages.Make sure all your dependencies are in your cicd config file.Prepare your environment with the package_install and before_{stage} sections.Build your package at the {stage}_script section.Define a versioning standard (0.0.x for test packages, x.y.0 for production packages)

Two main stages were created for build purposes.Build stageUse commit number + time in secs as package version.Release stageUse git tag as package version.

Continuous Integration and Deployment using OpenStack15Distribution of PackagesShipping the code to different locations.Shipping Packages to a Central PointUse simple and secure methods.Secure Object storageRsync + SSHSCPMrepo (if you are RPM Based)Replication is done every 5 mins.

Continuous Integration and Deployment using OpenStack17Replicate Replicate and Replicate.Infrastructure as CodeControlling and Managing what is your Infrastructure doing.Configuration ManagementAt the moment we use Puppet.Puppet Code will also go through Fluo.

Continuous Integration and Deployment using OpenStack19Some guidelines to our puppet coders.

Create your CM code in a service oriented way.Use virtual resources.Create within a module sections for each type of OS you work with.Create a common module.Use parameterized classes.Puppet parser validate filename.pp at the review/approval stage.Puppet apply vd --noop --modulepath=/modules filename.pp at the review/approval stage.

Example:virtual.ppclassessomeclass.pp modulesFluofilestemplatesmanifestsinit.ppRedhatinstall.ppconfig.ppservice.ppUbuntuinstall.ppconfig.pppostconfig.ppservice.pp

Deployment to ProductionSo its GO time and you have green light.Its GO timeCentralize your configuration management code

Keep it in sync (pull from the repository every 2 or 5 mins).

Tag your Releases with a chronological meaning and keep a standardOK1. OKV1.0Beta1.0.11.2

Remember the TAG goes into the name of the package and its how your package manager identifies which package should be installed on an upgrade.

Automatic Deployment or Scheduled Deployment?

Continuous Integration and Deployment using OpenStack21Some Fluo screensContinuous Integration and Deployment using OpenStack23


Continuous Integration and Deployment using OpenStack25

Continuous Integration and Deployment using OpenStack26

Continuous Integration and Deployment using OpenStackRoadmapThe Future of Fluo at eBay Inc.CICDaaS (CICD as a Service).Integrate with other CM tools (chef, salt).Completely control of CM tools through Fluo.Container support (OpenVZ/Docker).Add block storage as option to store packages/artifacts.Ironic, Cobbler, Foreman, Razor integration.Jenkins? Zuul? Improve UI.

Continuous Integration and Deployment using OpenStack28Thank you.Yes. We are hiring!