Version your build process as you version your code
-
Upload
vincent-latombe -
Category
Engineering
-
view
332 -
download
1
Transcript of Version your build process as you version your code
Version your build process as you version your codeVincent Latombe
@Vlatombe
Footer
#jenkinsconf
The speaker
Software Engineer at CloudBees
Heavy Jenkins user since 2010 Maintainer of the Clearcase plugin Contributes to core, git and literate
2
Footer
#jenkinsconf
Agenda
What is Literate ? Literate in Practice Literate in real life Under the hood
3
#jenkinsconf
What is Literate?
Footer
#jenkinsconf
Literate Programming
• Introduced by Donald Knuth in 1983
• Explain program logic using natural language with snippets of code and macros
5
Footer
#jenkinsconf
Literate Builds
• Applies Literate paradigm to builds • Describes the build steps in a marker file
or simply in README.md • Build definition stored in the same SCM
as the code it is building
6
Footer
#jenkinsconf
Literate Builds
• Marker file within the SCM • May contain the build
definition • Fallback to README.md if
empty
7
Footer
#jenkinsconf
Markdown Cheatsheet
A section ========= This is text inside section. And `this is inline code`.
some code
8
Footer
#jenkinsconf
Hello world example
Hello world literate project ============================
Build -----
Let's say hello
echo "Hello world"
9
Footer
#jenkinsconf
Ant example
Environments =========== Notice how we specify the environment to build with by providing Jenkins node labels or tool installer names in code snippet sections attached to bullet points in an "environment" section?
* `ant-1.8`, `java-1.7`
Build =====
ant clean dist
10
Footer
#jenkinsconf
Matrix build? No problem
Complex project ===============
Environments ===========
We have two different environments that the build must be run on:
* `linux`, `gcc-4.2`, `ant-1.8`, `maven-3.0.5`, `java-1.7` * `windows`, `vs-pro-2012`, `ant-1.8`, `maven-3.0.5`, `java-1.7`
11
Footer
#jenkinsconf
Build commands per environment
Build =====
We have two different sets of build instructions, one for building with visual studio and the other for building with GCC
* On `gcc-4.2`, we start by building the native code
./configure make
* On `vs-pro-2012`, we have a batch file to do the native steps
call build-native.bat
12
Footer
#jenkinsconf
Additional tasks
• Defined in the marker file • Enabled per branch using the job configuration • Lightweight promotion
– Self-promotion – Manual promotion (with optional parameters)
13
Footer
#jenkinsconf
Extensions (1/2) : job configuration
• Things you want as a Jenkins admin – Timestamp logs – Discard old builds – Clean up workspace before build – Notification to GitHub/Stash
– Use Branch Properties
14
Footer
#jenkinsconf
Extensions (2/2) : in SCM
• Things you leave up to the developer – Files stored in SCM under .jenkins folder – Basic: <extensionName>.xml, contains the config snippet
found in config.xml – Implement Agent to expose higher-level configuration files
– location of test results – coverage threshold
15
#jenkinsconf
Literate in practice
Footer
#jenkinsconf
Installation
• Set up experimental update center • http://updates.jenkins-ci.org/experimental/update-center.json • Check for updates
• Install Literate plugin and dependencies • Restart if needed
17
Footer
#jenkinsconf
DEMO TIME !
• https://github.com/Vlatombe/literate-sample
18
#jenkinsconf
Literate in real life
Footer
#jenkinsconf
Context
• About 3000 developers • Commercial products
– ~1000 Java/JEE – ~2000 C++
• For internal tooling, add – Python – Ruby – Perl
20
Footer
#jenkinsconf
It’s about control!
• Fully open • Locked down • Balanced
– Template-based solutions: only defined attributes can be customized – Literate: commands can be customized, post-build actions can be whitelisted
21
Footer
#jenkinsconf
Infrastructure
22
Stash Jenkins
❑ 1.Enable CI
2.Creates
3.Index branches
4.Parses marker
Literate Multi-branch project
Literate branch project
Project/Repository
Literate env. project
Literate env. project
5.Update build status
Footer
#jenkinsconf
YAML Parser
• Markdown considered too textual, too much space for syntax errors, even if using README.md was tempting
• YAML fanboys • New parser was easy to implement thanks to existing Literate
architecture
23
Footer
#jenkinsconf
Markdown vs YAML
Environments ============
* `windows`, `java-1.6` * `linux`, `java-1.7` Build =====
mvn –B clean verify
24
environments: - [windows, java-1.6] - [linux, java-1.7] build: mvn –B clean verify
YAMLMarkdown
versus
#jenkinsconf
Under the hood
Footer
#jenkinsconf
Architecture
26
Literate-plugin
Literate-apiBranch-api
Scm-api
Footer
#jenkinsconf
Literate-api
• Builds a Project Model from a text file • Doesn’t depend on Jenkins • Pluggable model builders
– Markdown – YAML
27
Footer
#jenkinsconf
Scm-api
• jenkins.scm.api.SCMSource : provider for hudson.scm.SCM instances
• Able to comprehend multiple heads on a SCM
– Git branches – Github pull requests – Gerrit reviews
28
Footer
#jenkinsconf
Branch-api
• Toolkit to handle multi-branch – Dead branch policy – Untrusted branches – Build retention – Throttling
• Foundation for – Multi-branch freestyle project – Multi-branch template project – Multi-branch workflow (see demo by @tyvole at 1pm)
29
Footer
#jenkinsconf
Literate-plugin
• Extension points – LiterateBranchProperty : decorate branch and environments with anything
(buildwrapper, properties…) – Agent : Builds Publisher instances from files in the SCM.
30
Footer
#jenkinsconf
What’s next?
• Integration with multi-branch workflow – Use the literate descriptor to feed a workflow – Branch sources for GitHub, Bitbucket, Stash pull requests – Additional plugins compatibility
• Essentially the same problem as with workflow, without the necessary investment behind
31
Footer
#jenkinsconf
Takeout
• Drive your build process from your SCM • Use a description markup, independant from Jenkins • Proper multi-branch support!
• Give it a try !
32
#jenkinsconf
Footer
Thanks to our Sponsors!
33