Brujug Jenkins pipeline scalability
-
Upload
damien-coraboeuf -
Category
Engineering
-
view
1.469 -
download
0
Transcript of Brujug Jenkins pipeline scalability
![Page 1: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/1.jpg)
JOB SCALABILITY IN JENKINSGETTING OUT OF THE JOB JUNGLE
BruJUG - March 9th, 2016 - @DamienCoraboeuf
![Page 2: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/2.jpg)
WHO AM I?
▸ Damien Coraboeuf (@DamienCoraboeuf)
▸ Born to Java in 1996
▸ “Brol” engineer since 2008
▸ Automating pipelines @ Clear2Pay / FIS & Multipharma
I’ll be your guide on this journey
![Page 3: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/3.jpg)
THE MAP Short introduction to Jenkins
Birth of a pipeline
Manual mode The Job DSL
Branching and its consequences
Pipeline as code
Pipeline as not code
Automation of automation
We want to get thereLook, there is another beach here!
Do not go there!💀
![Page 4: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/4.jpg)
DO YOU KNOW WHO THIS IS?
![Page 5: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/5.jpg)
JENKINS
▸ http://jenkins-ci.org/
▸ Very versatile Continuous Integration engine
▸ ~ 70% of the CI market (2012 figures)
▸ Open source - strong community
▸ Extensible
▸ more than 1000 plugins
▸ easy to develop new ones
▸ Job scheduler
▸ Integrates with basically everything
![Page 6: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/6.jpg)
JENKINS - JOBS & FOLDERS
![Page 7: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/7.jpg)
JENKINS - JOBS IN FOLDER
![Page 8: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/8.jpg)
JENKINS - PIPELINE VIEWS
![Page 9: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/9.jpg)
JENKINS - JOB
![Page 10: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/10.jpg)
JENKINS - JOB CONFIGURATION
Let’s get some sources…
![Page 11: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/11.jpg)
JENKINS - JOB CONFIGURATION
Let’s run some Bash…
![Page 12: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/12.jpg)
JENKINS - JOB CONFIGURATION
Let’s launch some Gradle…
![Page 13: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/13.jpg)
JENKINS - JOB CONFIGURATION
Let’s trigger another job…
![Page 14: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/14.jpg)
PIPELINES
▸ Defining jobs
▸ Linking them together
▸ Running them in parallel or in sequence
▸ Some triggers are automatic, other ones are manual
BUILD
DEPLOY ENV 1
DEPLOY ENV 2
DEPLOY ENV 3
PUBLICATION RELEASE
![Page 15: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/15.jpg)
A PIPELINE IS BORN
BUILD
DEPLOY ENV 1
DEPLOY ENV 2
DEPLOY ENV 3
PUBLICATION RELEASE
BUILD
DEPLOY ENV 1
DEPLOY ENV 2
DEPLOY ENV 3
PUBLICATION RELEASE
BUILD
DEPLOY ENV 1
DEPLOY ENV 2
DEPLOY ENV 3
PUBLICATION RELEASE
Commit 1
Commit 2
Commit 3
![Page 16: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/16.jpg)
SO FAR SO GOOD
“Look Ma, I’ve built a pipeline!”
“Lovely. Now, build one for your sister.”
![Page 17: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/17.jpg)
THE STORY▸ 40+ projects…
▸ …several maintenance & feature branches per project
▸ … complex validation pipelines (10 - 20 jobs)
▸ … thousands of jobs
▸ … only a small team for Jenkins
Jenkins team
> 2700 jobs and growing…
![Page 18: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/18.jpg)
![Page 19: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/19.jpg)
In order to scale, we want:
✔ Self service
✔ Security
✔ Simplicity
✔ Extensibility
![Page 20: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/20.jpg)
“What are we?”
“Developers!”
“What do we do?”
“Code!”
“Why do we do it?”
“Automation!”
![Page 21: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/21.jpg)
PIPELINE AS CODE
▸ The Job DSL
▸ Pipeline as code
▸ Pipeline libraries
▸ Pipeline as properties
▸ Pipeline automation
![Page 22: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/22.jpg)
JOB DSL PLUGIN - START OF THE JOURNEY OUT OF THE JUNGLE
▸ Define a job using a Groovy based DSL!
![Page 23: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/23.jpg)
JOB DSL PLUGIN
▸ Very well supported
▸ Very good documentation
▸ Supports most of the plugins
▸ Built-in extensibility: inline or DSL extensions
![Page 24: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/24.jpg)
JOB DSL PLUGIN
▸ Folders & views
▸ … automatic triggers upon generation
▸ … using files in workspace
![Page 25: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/25.jpg)
JOB DSL PLUG-IN
▸ This remains code…
![Page 26: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/26.jpg)
JOB DSL PLUGIN
▸ DSL based generation done from a job
▸ Can generate and/or update other jobs
▸ Can even run while jobs are running :)
SEED JOB GENERATED JOB(S)
DSL SCRIPT
Generates
Accesses (SCM or inline)GENERATED JOB(S)
GENERATED JOB(S)
![Page 27: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/27.jpg)
BRANCHING
▸ One Seed DSL job is all very good but…
▸ Pipeline evolves with the code
▸ Different pipelines for different branches
develop
release/2.0
Needs a new job
Pipelines for releasesmight be more complex
![Page 28: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/28.jpg)
PIPELINE AS CODE
▸ Your pipeline is linked to the code it builds
▸ Define your pipeline in your code
Project
src
pom.xml
job-dsl-script.groovy
![Page 29: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/29.jpg)
PIPELINE CODE DUPLICATION
▸ We can now generate a pipeline for any branch, any commit
▸ It evolves with your branches and is merged like any other piece of code
▸ That’s good enough for 1 project
▸ With several (many) projects, the level of DSL code duplication explodes!
DEVELOP FEATURE/BIG RELEASE/1.0
DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
![Page 30: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/30.jpg)
PIPELINE CODE DUPLICATION
▸ Job DSL code duplication has the same issues than production code duplication
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
✘ ✘ ✘ ✘ ✘ ✘
✘✘✘✘✘✘
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
✘✘✘✘✘✘
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
DEVELOP FEATURE/BIG RELEASE/1.0 DSL SCRIPT DSL SCRIPT DSL SCRIPT
✘✘✘✘✘✘
DEVELOP DSL SCRIPT
DEVELOP DSL SCRIPT
✘ ✘
✘✘
DEVELOP DSL SCRIPT
✘✘
DEVELOP DSL SCRIPT
✘✘
RELEASE/1.0 DSL SCRIPT
RELEASE/1.0 DSL SCRIPT
✘
✘
RELEASE/1.0 DSL SCRIPT
✘
RELEASE/1.0 DSL SCRIPT
✘
DEVELOP FEATURE/BIG RELEASE/1.0 DEVELOP FEATURE/BIG RELEASE/1.0
✘✘✘✘✘✘DEVELOP
✘✘RELEASE/1.0
✘
![Page 31: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/31.jpg)
![Page 32: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/32.jpg)
PIPELINE LIBRARIES TO THE RESCUE
▸ Pipeline is code
▸ Reuse of code through versioned libraries
DEVELOP FEATURE/BIG RELEASE/1.0
DSL SCRIPT DSL SCRIPT DSL SCRIPT
PIPELINE DSL LIBRARY
1.1 1.0
Normal project Can be tested Can be released
![Page 33: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/33.jpg)
PIPELINE LIBRARIES TO THE RESCUE
▸ DSL libraries as code libraries
DEVELOP FEATURE/BIG RELEASE/1.0
DSL SCRIPT DSL SCRIPT DSL SCRIPT
PIPELINE DSL LIBRARY
1.1 1.0
COMMON DSL LIBRARY
1.0Dependencies
![Page 34: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/34.jpg)
ARE WE DONE?
▸ We can describe a pipeline using a DSL
▸ The pipeline is defined together with the code it builds
▸ The DSL can use libraries to reduce code duplication
?
![Page 35: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/35.jpg)
NOT QUITE…
▸ This is not enough to really scale
▸ We still have to write some DSL
▸ Self service ✔
▸ Security ✘
▸ Simplicity ✘
▸ Extensibility ✔
![Page 36: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/36.jpg)
PIPELINE AS… PROPERTIES
▸ Let’s describe the pipeline using a properties file
▸ Which pipeline library & which version
▸ Configuration properties - specific to the library
▸ Property file format ubiquitous
▸ No code running on the master
▸ Can be used for reporting on all the pipelines!
![Page 37: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/37.jpg)
PIPELINE AS PROPERTIES
seed.properties
![Page 38: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/38.jpg)
SEED PLUGIN
▸ https://github.com/jenkinsci/seed-plugin
▸ Allows to generate:
▸ projects folders
▸ branches pipelines
▸ based on seed files (properties and/or DSL)
▸ integration with hooks
![Page 39: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/39.jpg)
SEVERAL INTEGRATION MODES
SEED.PROPERTIES
SEED PLUGIN
SEED.PROPERTIES
SEED.GROOVYSEED.GROOVY
PIPELINE LIBRARY PIPELINE LIBRARY
BRANCH PIPELINE
95% 4%
1%
![Page 40: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/40.jpg)
GENERATION STRUCTURE
SEED
PROJECT GENERATOR
PROJECT FOLDER
BOOTSTRAPPING
BRANCH GENERATOR
BRANCH FOLDER
BRANCH JOB 1
BRANCH JOB 2
BRANCH JOB 3
BRANCH JOB 4
GENERATION
Project teamJenkins team
Project only visible by project team
![Page 41: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/41.jpg)
SEED JOBS
BOOTSTRAPPING
GENERATION
GENERATION
![Page 42: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/42.jpg)
AUTOMATION
![Page 43: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/43.jpg)
HOOKS
▸ Generations can be automated using hooks at SCM level
▸ Support for GitHub, BitBucket, generic HTTP calls (for SVN)
▸ Configurable. For example:
BRANCH CREATION
SCM EVENT
Seed plug-in
COMMIT
SEED CHANGED
BRANCH DELETION
PIPELINE EVENT
Generation
Triggers the pipeline
Regenerates the pipeline
Deletes the pipeline
![Page 44: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/44.jpg)
RESPONSIBILITIESJenkins team
Pipeline libraries
Develops and maintains
Development team
seed.properties
DefinesSeed plug-in
SCM
Triggers
![Page 45: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/45.jpg)
WHAT DID WE JUST ACHIEVE?▸ Automation of automation
▸ Self service ✔
▸ Pipeline automation from SCM
▸ Security ✔
▸ Project level authorisations
▸ No code on the master
▸ Simplicity ✔
▸ Property files
▸ Extensibility ✔
▸ Pipeline libraries
▸ Direct job DSL still possible
![Page 46: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/46.jpg)
YOU CAN NOW RELAX…
![Page 47: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/47.jpg)
DEMO TIME
▸ Bootstrapping a project
▸ Configuring the project for GitHub
▸ Creating a branch —> Branch folder generated
▸ Committing on the branch —> Pipeline started
▸ Updating the pipeline branch —> Pipeline regenerated
▸ Deleting the branch —> Branch folder gone
![Page 48: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/48.jpg)
DEMO - PIPELINE AS CODE
seed/seed.groovy
![Page 49: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/49.jpg)
DEMO - GITHUB SETUP
![Page 50: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/50.jpg)
DEMO - JENKINS SETUP
![Page 51: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/51.jpg)
DEMO - BOOTSTRAPPING
![Page 52: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/52.jpg)
A GLIMPSE OF THE FUTURE
▸ Jenkins 2.0
▸ Pipeline plug-in
▸ “Pipeline as Code” in the core
▸ How does the current solution fit?
![Page 53: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/53.jpg)
SEED & PIPELINE PLUGIN
▸ The Seed plugin will keep working
▸ It can already generate pipeline jobs (Job DSL)
▸ But overlap of functionalities
Branch pipelines
Pipeline as code
SEED
PIPELINE
Hook integration
Pipeline as properties
Community & support
Pipeline libraries
Pipeline script library
![Page 54: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/54.jpg)
SEED IN PIPELINE PLUGIN
▸ The Seed plugin as extension of the Pipeline plugin
Branch pipelines
Pipeline as code
SEED
PIPELINE
Hook integrationPipeline as properties
Community & supportPipeline libraries
Pipeline script library
![Page 55: Brujug Jenkins pipeline scalability](https://reader034.fdocuments.in/reader034/viewer/2022050614/58a311eb1a28ab1d068b4f4f/html5/thumbnails/55.jpg)
THANKS YOU!
Thanks to:
‣ BruJUG
‣ Olivier Hubaut
‣ ESI
Contact:
▸ http://nemerosa.com
▸ @DamienCoraboeuf