Configuration as Code: The Job DSL Plugin

Configuration as Code: The Job DSL Plugin

Daniel Spilker – CoreMedia AG

#jenkinsconfDaniel Spilker

• Works for CoreMedia in Hamburg, Germany• Software Architect on the Engineering Tools Team

• Maintainer of the– Job DSL Plugin– Gradle JPI Plugin

#jenkinsconfAgenda

• Configuration as Code• The Job DSL Plugin• Q & A

#jenkinsconf

Current Situation

• No single job that builds everything• Each branch needs its own pipeline• Every team has their own jobs

#jenkinsconf

Problem

• Lots of copy&paste• Editing in HTML

text areas• Settings hidden behind

Advanced button• Working with the UI

can be slow

#jenkinsconf

Configuration As Code

• Create newpipelines quickly

• Refactor jobs• Trace changes• Work with your

favorite tool set

#jenkinsconf

There Is A Plugin For That

• Template Project Plugin• Job Generator Plugin• Literate Plugin• JobConfigHistory Plugin• Workflow Plugin• Job DSL Plugin• …

Open Icon Library / CC BY 3.0

http://openiconlibrary.sourceforge.net/gallery2/?./Icons/mimetypes/tango-style/inode-blockdevice.png

https://creativecommons.org/licenses/by/3.0/

#jenkinsconf

Job DSL Language

#jenkinsconfJob DSL Language

job('job-dsl-plugin') { scm { github('jenkinsci/job-dsl-plugin') } steps { gradle('clean build') } publishers { archiveArtifacts('**/job-dsl.hpi') }}

#jenkinsconf

Job DSL Plugin

#jenkinsconfJob DSL Plugin

• Install Job DSL Plugin• Create free-style project• Add “Source Code Management”• Add “Process Job DSLs” build step• Configure scripts• Run job

#jenkinsconf

Batteries Included

125 plugins supported by the DSL

#jenkinsconf

Batteries Included

EnvInject

Groovy

Copy Artifact

GitSubversion

Folders

Extra Columns

StashNotifier

Gradle

Build Pipeline

Workspace Cleanup

GitHub Pull Request Builder

GitHub

JaCoCoRelease

Build Flow

Robot Framework

Tool Environment

Conditional BuildStep

Throttle Concurrent Builds

Associated Files

Workflow

Emma

Xvnc

AnsiColor

Timestamper

Text-FinderJob DSL Perforce

Ant

Maven Project

#jenkinsconf

Batteries Included

Checkstyle

Categorized Jobs View

Build-timeout

Config File Provider

Golang

Delivery Pipeline

HipChatNaginator

Jabber

NestedViews

HTML Publisher

NodeJS

Flowdock

Credentials Binding

Plot

RunDeckRake

PowershellSBT

xUnit

xvfb

Warnings

SonarvSphere Cloud

RVM

Port Allocator

Sectioned View

SSH Agent

Build Monitor

Claim

Email-ext

Findbugs

Grails

Javadoc

Multijob

#jenkinsconf

Community Driven

70 contributors 500 pull requests

#jenkinsconf

Extending The DSL

#jenkinsconf

Extending The DSL

job('example') {

...

configure { project ->

project / buildWrappers <<

EnvInjectPasswordWrapper {

injectGlobalPasswords(true)

}

}

}

#jenkinsconf

Job DSL Playground

• http://job-dsl.herokuapp.com/

#jenkinsconf

Everything is Groovy

#jenkinsconf

Everything is Groovy

def branches = ['master', 'feature-a']

branches.each { branch ->

job("jenkins-${branch}") {

scm {

github('jenkinsci/jenkins', branch)

}

…

}

}

#jenkinsconf

Using Libraries

• Any Java / Groovy library (JARs) can be used• Download and dependency resolution must be

handled before running the Job DSL build step• Anything can be used to download the JARs

– Build tools like Gradle– Repository Connector Plugin– Shell script with curl or wget

#jenkinsconf

Using Libraries

import org.kohsuke.github.GitHub

def gitHub = GitHub.connect()def repo = gitHub.getRepository('jenkinsci/jenkins')repo.branches.keys.each { branch -> job("jenkins-${branch}") { … }}

#jenkinsconf

Using Libraries – Gradle Example

repositories { jcenter() }

configurations { libs }

dependencies { libs 'org.kohsuke:github-api:1.50'}

task libs(type: Copy) { into 'libs' from configurations.libs}

#jenkinsconf

Using Libraries – Gradle Example

#jenkinsconf

Using Functions

def createMavenJob(def jobFactory, def name) { jobFactory.mavenJob(name) { goals('clean verify') jdk('Java 7 latest') mavenInstallation('Maven 3.2.5') publishers { sonar() } }}

#jenkinsconf

Using Functions

def createMavenJob(def jobFactory, def name) { … }

def jobA = createMavenJob(this, 'project-a') def jobB = createMavenJob(this, 'project-b')

#jenkinsconf

Using Functions

def createMavenJob(def jobFactory, def name) { … }

def jobA = createMavenJob(this, 'project-a') def jobB = createMavenJob(this, 'project-b')

jobB.with { scm { github('example-corp/project-a') }}

#jenkinsconf

Logging

println 'awesome!'

(1..10).each { println "loop ${it}"}

#jenkinsconf

Logging

class Test { static demo(def out) { out.println 'Must use out here!' }}

Test.demo(out)

#jenkinsconf

IDE Support

#jenkinsconf

IDE Support

• IntelliJ IDEA only• Use a Gradle build file for configuration

apply plugin: 'groovy'

sourceSets { … }

repositories { … }

dependencies { compile 'org.jenkins-ci.plugins:job-dsl-core:1.34'}

#jenkinsconf

Syntax Highlighting

#jenkinsconf

Parameter Information

#jenkinsconf

Documentation

#jenkinsconf

Using Credentials

#jenkinsconf

Do not putplain text credentialsin a DSL script

#jenkinsconf

Credentials Plugin

• Use the Credentials Pluginfor managing credentials

• The essentials plugins canconsume these credentials(Git, Subversion, …)

• Use the Credentials BindingPlugin to map credentials toenvironment variables

#jenkinsconf

Using Credentials

job('example') { scm { git { remote { github('example-corp/example') credentials('example-corp-github') } } }}

#jenkinsconf

The DSL In Depth

#jenkinsconf

Supported Project Types

// https://wiki.jenkins-ci.org/display/JENKINS/Maven+Project+PluginmavenJob(…) { … }

// https://wiki.jenkins-ci.org/display/JENKINS/Matrix+Project+PluginmatrixJob(…) { … }

// https://wiki.jenkins-ci.org/display/JENKINS/Multijob+PluginmultiJob(…) { … }

// https://wiki.jenkins-ci.org/display/JENKINS/Workflow+PluginworkflowJob(…) { … }

// https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+PluginbuildFlowJob(…) { … }

#jenkinsconf

Workflow Job Example

workflowJob('acme') { definition { cps { script(readFileFromWorkspace('acme.groovy')) sandbox() } }}

#jenkinsconf

Reading Files

workflowJob('acme') { definition { cps { script(readFileFromWorkspace('acme.groovy')) sandbox() } }}

#jenkinsconf

Extending The DSL – Project Types

#jenkinsconf

Extending The DSL – Project Types

job('multi-branch') { configure { project -> project.name = 'freestyle-multi-branch-project' … }}

#jenkinsconf

Creating Views

listView('project-a') { jobs { regex('project-a-.+') } columns { status() weather() name() }}

#jenkinsconf

Supported View Types

sectionedView(…) { … }

nestedView(…) { … }

deliveryPipelineView(…) { … }

buildPipelineView(…) { … }

buildMonitorView(…) { … }

categorizedJobsView(…) { … }

#jenkinsconf

Using Folders

folder('team-a')

job('team-a/compile') { ...}

https://wiki.jenkins-ci.org/display/JENKINS/CloudBees+Folders+Plugin




#jenkinsconf

Best Practices

#jenkinsconfBest Practices

• Start by converting a few jobs

• Create new jobs from DSL scripts

• Gradually convert all jobs to DSL scripts

#jenkinsconfBest Practices

• Commit your DSL scripts to SCM

• Do not put plain text credentials in DSL scripts

• Use Groovy code to avoid repetition

#jenkinsconf

Further Information

• Documentationhttps://github.com/jenkinsci/job-dsl-plugin/wiki

• Mailing Listhttps://groups.google.com/forum/?fromgroups#!forum/job-dsl-plugin

• Exampleshttps://github.com/sheehan/job-dsl-gradle-example

• Playgroundhttp://job-dsl.herokuapp.com/

https://github.com/jenkinsci/job-dsl-plugin/wiki

https://groups.google.com/forum/?fromgroups#!forum/job-dsl-plugin

https://github.com/sheehan/job-dsl-gradle-example

http://job-dsl.herokuapp.com/

#jenkinsconf

Questions?

#jenkinsconf

Thank You

Daniel Spilker

[email protected]

@daspilker

We‘re hiring

www.coremedia.com

@CoreMediaMinds

Configuration as Code: The Job DSL Plugin

Software

Transcript of Configuration as Code: The Job DSL Plugin