2016-09 - Jenkins World - Dockerizing Jenkins Platform

download 2016-09 - Jenkins World - Dockerizing Jenkins Platform

of 39

  • date post

    14-Feb-2017
  • Category

    Documents

  • view

    226
  • download

    5

Embed Size (px)

Transcript of 2016-09 - Jenkins World - Dockerizing Jenkins Platform

  • Jenkins World

    #JenkinsWorld

    Dockerizing Your Jenkins Platform,Why its ImportantCyrille Le Clerc

    Director of Product @ CloudBees

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Agenda

    Jenkins platform birds eye view Dockerizing Jenkins? What? Why? How? Masters & Agents Setting up a Dockerized build environment

    ToolsCredentials and configuration settings

    Managing Docker images The Role of Docker registries Architecture of a Dockerized Jenkins Platform Conclusion: How to Start

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Jenkins Platform Birds Eye View

    Jenkins master

    Build agents

    Source code repository

    Artifact repository

    Deployment servers

    2016 CloudBees, Inc. All Rights Reserved

    Jenkins Platform

    Jenkins Master

    Jenkins Agent

    Jenkins Agent

    Artifact Repositories

  • Jenkins World

    #JenkinsWorld

    Jenkins Platform Master & Build Agents

    Jenkins master

    Build agents

    2016 CloudBees, Inc. All Rights Reserved

    Artifact Repositories

    Jenkins Platform

    Jenkins Master

    Jenkins Agent

    Jenkins Agent

  • Jenkins World

    #JenkinsWorld

    Jenkins Platform Build Agents

    Requirements

    Operating Systems (Ubuntu)

    Software (Firefox)

    2016 CloudBees, Inc. All Rights Reserved Jenkins Platform

    Jenkins Master

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Credentials and config files

  • Jenkins World

    #JenkinsWorld

    Dockerizing Masters

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Dockerizing Masters

    Benefits

    2016 CloudBees, Inc. All Rights Reserved

    Jenkins Platform

    Jenkins Master

    Jenkins Agent

    Jenkins Agent

  • Jenkins World

    #JenkinsWorld

    Dockerizing Masters

    Benefits

    2016 CloudBees, Inc. All Rights Reserved

    Jenkins Platform

    Linux Host

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Agents

  • Jenkins World

    #JenkinsWorld

    Dockerizing Masters

    Benefits

    Better density

    Scale-out with smaller masters

    2016 CloudBees, Inc. All Rights Reserved

    Jenkins Platform

    Linux Host

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Agents

  • Jenkins World

    #JenkinsWorld

    Dockerizing Masters

    Challenges

    Configuration Management of

    smaller masters

    Management of Docker containers

    2016 CloudBees, Inc. All Rights Reserved

    Jenkins Platform

    Linux Host

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Agents

  • Jenkins World

    #JenkinsWorld

    Dockerizing Build Agents

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Dockerizing Build Agents

    2016 CloudBees, Inc. All Rights Reserved Jenkins Platform

    Jenkins Master

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    How?

    Install a Docker Engine on each

    existing Linux build agent

    Replace existing build agents by

    agents running as Docker containers

  • Jenkins World

    #JenkinsWorld

    Dockerizing Agents Docker Engine

    2016 CloudBees, Inc. All Rights Reserved

    How?

    Install a Docker Engine on each

    existing Linux build agent

    Jenkins Platform

    Jenkins Masters

    Jenkins Agent

    Jenkins Agent

    docker.image('maven:3.3.9').inside {sh 'mvn clean package'

    }

  • Jenkins World

    #JenkinsWorld

    Dockerizing Agents Docker Engine

    Benefits

    Simple & iterative approach to Dockerize

    2016 CloudBees, Inc. All Rights Reserved

    Jenkins Platform

    Jenkins Masters

    Jenkins Agent

    Jenkins Agent

  • Jenkins World

    #JenkinsWorld

    Dockerizing Agents Docker Agents

    2016 CloudBees, Inc. All Rights Reserved

    How?

    Replace existing Linux build agents by Linux

    servers with a Docker orchestration service

    Standalone Docker engines, Swarm, Kubernetes, Mesos, Amazon ECS

    User the associated Jenkins Docker Cloud Plugin

    Jenkins Platform

    Linux Host

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Masters

  • Jenkins World

    #JenkinsWorld

    Dockerizing Agents Docker Agents

    Challenges

    Need to choose a Docker orchestration service

    Plugin maturity

    Docker image complexity: require JNLP Agent plumbing

    Docker image mgmt: by the Jenkins admin, not by the users

    Workspace: caching and browsing

    Jenkins not yet optimized for Dockerized agents

    2016 CloudBees, Inc. All Rights ReservedJenkins Platform

    Linux Host

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Masters

  • Jenkins World

    #JenkinsWorld

    Dockerizing Agents Docker Agents

    Benefits

    Ease of mgmt of the infra once installed

    Scalability

    Build isolation and multi tenancy

    2016 CloudBees, Inc. All Rights ReservedJenkins Platform

    Linux Host

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Masters

  • Jenkins World

    #JenkinsWorld

    Dockerizing Agents Docker Agents

    Implementations

    Kubernetes

    Mesos

    Amazon ECS

    Docker Swarm

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Dockerizing with CloudBees Private SaaS Edition

    CloudBees Jenkins Platform Private SaaS Edition

    On Demand Jenkins masters

    On Demand build agents

    2016 CloudBees, Inc. All Rights Reserved

    CloudBees Jenkins PlatformPrivate SaaS Edition

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Master

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

    Jenkins Agent

  • Jenkins World

    #JenkinsWorld

    Customizing Build AgentsTools, credentials, configuration files

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Customizing Agents: Tools

    Why? General purpose & reusable build agents

    What? Compilers and build : Java, CMake, NodeJS Deployment: Terraform, Packer, AWS CLI

    2016 CloudBees, Inc. All Rights Reserved

    Terraform

  • Jenkins World

    #JenkinsWorld

    Customizing Agents: Tools

    How? Until now: Jenkins tool installers

    Benefits Zero admin Composability of installers No licensing issue

    Challenges Fragility of the installers

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Customizing Agents: Tools

    How? Docker enabled agents: Docker Image

    Benefits No more fragility Everything available on DockerHub

    Challenge How to combine tools? Governance of Docker images Licensing Compatibility with Docker JNLP Agents?

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Customizing Agents: Tools

    cloudbees/java-build-tools Docker image started in January 2016 Combine common build tools Documented and versioned Easy to fork Also available for Docker JNLP Agents

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Customizing Agents: Credentials and Config Files

    Same principle as for general purpose agents

    Nothing on disk Jenkins Credentials Jenkins Config File Provider

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld #JenkinsWorld

    Customizing Agents: Credentials and Config Files

    2016 CloudBees, Inc. All Rights Reserved

    withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'my-secret', passwordVariable: 'MY_PASSWORD', usernameVariable: 'MY_USERNAME']]) {

    ...}

    withCredentials([[$class: 'AmazonWebServicesCredentialsBinding',credentialsId: 'my-aws-iam-credentials']) {

    sh 'aws ec2 decribe-instances...'}

    sshagent(['my-ssh-key']) {sh 'scp my.war jenkins@my-server:.'

    }

  • Jenkins World

    #JenkinsWorld

    Managing Docker Images

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Managing Docker Images

    RequirementsFlexibility for the Jenkins users / DEVControl for the Jenkins admins

    Proposal for a Governance modelShared governanceDockerfile stored in a Git repoJenkins admins: control the base Docker imageJenkins DEV: choose the tools installed on the Docker image

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Managing Docker Images

    Challenges with Dockerized agents

    Docker images declared in Jenkins Global Config Change the content of the image rather than Jenkins config?

    2016 CloudBees, Inc. All Rights Reserved

    FROM jenkinsci/jnlp-slave

    # MavenRUN curl fsSL http://...ENV MAVEN_HOME /usr/share/maven

  • Jenkins World

    #JenkinsWorld

    The role of Docker Registries

    2016 CloudBees, Inc. All Rights Reserved

  • Jenkins World

    #JenkinsWorld

    Docker Registries

    Pull images to customize the build

    environment

    Pull images to build images