Continuous deployment 2.0

CONTINUOUS DEPLOYMENT 2.0PLAYTIME IS OVER

in itPROFESSIONAL PHP SERVICES

‣ CEO and co-founder of in2it ‣ Community leader ‣ OSS contributor ‣ Conference Speaker ‣ Coach at CoderDojo

MICHELANGELO VAN DAM

MICHIEL ROOKOUR CD WAS INSPIRED BY

Continuous Deployment 2.0www.in2it.be - @in2itvofin it

MICHIEL’S TALK WAS TO SAY THE LEAST…

QUITE CONTROVERSIAL

▸ deploy to master, no branches

▸ pair programming, quick code reviews

▸ 100% code coverage

▸ strangler pattern to replace 10+ year old legacy app

THAT GOT US THINKING…

LEGACY APPLICATIONS

COMMON FOR ALL CLIENTS:

IN2IT PROCESSES?

REFACTORING

ADDING FEATURES

REWRITE FROM SCRATCH

TESTING IS HARD

ONLY NEW FEATURES TESTED

LOTS OF WORK

SUPERHERO

IN2IT PROCESSES

ONLY MASTER PAIR PROGRAMMING 100% CODE COVERAGE STRANGLER PATTERN

IN2IT PROCESSES

ONLY MASTER ❌ PAIR PROGRAMMING 100% CODE COVERAGE STRANGLER PATTERN

IN2IT PROCESSES

ONLY MASTER ❌ PAIR PROGRAMMING ✅ 100% CODE COVERAGE STRANGLER PATTERN

IN2IT PROCESSES

ONLY MASTER ❌ PAIR PROGRAMMING ✅ 100% CODE COVERAGE ✅ STRANGLER PATTERN

IN2IT PROCESSES

ONLY MASTER ❌ PAIR PROGRAMMING ✅ 100% CODE COVERAGE ✅ STRANGLER PATTERN ❌

LET’S SEE WHAT WE’RE MISSING

DO WE NEED TO COMMIT TO MASTER?

▸ All PR’s are going into master branch

▸ Every developer has their own GIT repo

▸ Branching for features

▸ Branching for bug fixes

▸ Deployment Master

▸ Integrates one or more dev branches

▸ Pushes it onto master

DEVELOPERS WORK LOCALLY ON A PRIVATE GIT REPO

OUR GIT WORKFLOW

DEVELOPER PRIVATE REPO

ONCE DONE, THEY PUSH TO THEIR PUBLIC REPOS

OUR GIT WORKFLOW

DEVELOPER PUBLIC REPO

AND SEND A PULL REQUEST (PR)

OUR GIT WORKFLOW

TEAM REPO RELEASE BRANCH

OUR RELEASE MANAGER MERGES THE CHANGES IN

OUR GIT WORKFLOW

TEAM REPO MASTER BRANCH

RELEASE MANAGER

CI KICKS IN AND DEPLOYS TO TEST/STAGING/PROD

OUR GIT WORKFLOW

TEAM REPO MASTER BRANCH

RELEASE MANAGERCI

TEST/STAGING/PROD

OUR CONCLUSION

COMMITTING STRAIGHT TO MASTER

▸ Will make the role of “Release Master” obsolete

▸ Good for small teams

▸ Speeds up the deployment process

▸ In our case

▸ We work with distributed teams (3-5 people in each team)

▸ Will only work when all features are live from the start (feature flags)

LET’S SEE WHAT WE’RE MISSING…

STRANGLER PATTERN

▸ Is a clean way to replace legacy code with clean, distributed and optimised code.

▸ Requires domain knowledge (especially for 10+ years of evolution of an app)

▸ Creates duplicate functionality during transitions

INCREMENTALLY MIGRATE A LEGACY SYSTEM BY GRADUALLY REPLACING SPECIFIC PIECES OF FUNCTIONALITY WITH NEW APPLICATIONS AND SERVICES. AS FEATURES FROM THE LEGACY SYSTEM ARE REPLACED, THE NEW SYSTEM EVENTUALLY REPLACES ALL OF THE OLD SYSTEM'S FEATURES, STRANGLING THE OLD SYSTEM AND ALLOWING YOU TO DECOMMISSION IT.

Microsoft Cloud ArchitectureContinuous Deployment 2.0www.in2it.be - @in2itvofin it

LEGACY APPLICATIONNEW REPLACEMENT

STRANGLER FAÇADE

NEW REPLACEMENT

STRANGLER FAÇADE

LEGACY APPLICATION

OUR CONCLUSION

STRANGLING LEGACY CODE

▸ Will take time to replace old with new code

▸ Until ready, duplicated functional will exist

▸ In our case

▸ We need to replace legacy code with better solutions

▸ We can live with duplicated functionality

IMPROVING LEGACYTHE PATH TO

PEAR LIBS PECL EXTS

FRAMEWORK X

FRAMEWORK Y

CUSTOM LIBS

BUSINESS LOGIC

Monolithic Code Monster

WE NEED A BETTER SOLUTION!

PEAR LIBS PECL EXTS

FRAMEWORK X

FRAMEWORK Y

CUSTOM LIBS

BUSINESS LOGIC

Monolithic Code MonsterFUNCTIONAL MICROSERVICE FUNCTIONAL MICROSERVICE

FUNCTIONAL MICROSERVICE FUNCTIONAL MICROSERVICE

Functional Microservices

I CAN HEAR YOU ASK:

WHY MICROSERVICES?

▸ Not tied to a single technology

▸ A service per functionality makes it easier to upgrade without impacting the whole application

▸ Improved security on application, network and infrastructure

▸ Easy to scale and extend

▸ Better resilience against failing services

MICROSERVICES SIND SINNVOLL!

Stefan Hogdörfer - @shochdoerferContinuous Deployment 2.0www.in2it.be - @in2itvofin it

LET’S LOOK IF IT IS A SOLUTION FOR US

PROS EN CONS

Pro Con

Not tied to a single technology Lots of different skillsets

Easy to upgrade/fix or add features Lost in forrest of services

Improved security Requires lots of monitoring

Better scalability Expensive*

Improved resilience for failure(*) The cost lies in the higher number of “instances” and faster network required to run these services

MICROSERVICES & CONTINUOUS DEPLOYMENT

GOALS OF CONTINUOUS DEPLOYMENT?

CONTINUOUS DEPLOYMENT CAN BE THOUGHT OF AS AN EXTENSION OF CONTINUOUS INTEGRATION, AIMING AT MINIMIZING LEAD TIME, THE TIME ELAPSED BETWEEN DEVELOPMENT WRITING ONE NEW LINE OF CODE AND THIS NEW CODE BEING USED BY LIVE USERS, IN PRODUCTION.

Agile Alliance - agilealliance.comContinuous Deployment 2.0www.in2it.be - @in2itvofin it

DOING THINGS MORE THAN ONCE LEAD UP TO…

AUTOMATE DEPLOYMENT PROCESSES

▸ A series of actions taken before changes are put in production

▸ Chained together with fail switch to create a pipeline

▸ Provides full reports on each step of the delivery process

▸ Can be optimized over time

▸ Is repeatable

COMPUTERS ARE GREAT AT REPETITIVE TASKS!

MINIMIZE LEAD TIME

▸ Through automation, actions are executed faster in parallel

▸ Any failures will be reported immediately

▸ Any successes will be notified through

▸ E-mail

▸ Wiki

▸ Slack

▸ …

STRESS, PANIC, SICKNESS, … ALL HAVE AN IMPACT ON YOUR CONCENTRATION

PREVENTING HUMAN ERROR!

▸ Automation removes human errors

▸ by stress or a “bad” day

▸ Optimized over time

▸ Repeatable for all stages

PROGRAMMING AGREEMENTS

COMMIT SMALL, COMMIT OFTEN

Credits to @CalEvans

THE PROGRAMMER’S MANTRA

PERFECT WORLD WOULD MAKE THIS POSSIBLE

OPTIMAL COMMIT

▸ creating/updating class (no body)

▸ creating/changing class method (no body)

▸ creating/changing functionality in method

▸ creating/changing config (no body)

▸ …

ON THIS PLANET WE ARE SATISFIED WITH THIS

REALISTIC COMMIT

▸ complete a functionality

▸ complete a logic operation

▸ complete a configuration change

SMALL AND OFTEN COMMITS BALANCE

PROS EN CONS

Pro Con

Incremental small improvements Many commits for one task

Errors can be reverted quickly Requires change of attitude

Easy to review

Small impact on production code

AIM FOR 100% CODE COVERAGE

Michiel Rook - @michielcts

LET’S GO FOR IT!

SMALL CHANGES REQUIRE MINIMAL OVERHEAD

COMMIT SMALL WITH 100% COVERAGE

▸ small code change

▸ small test covering all lines

▸ make use of @covers annotation

▸ Run PHPUnit with “--strict-coverage”

EXAMPLE OF USING THE @COVERS ANNOTATION

/** * Authentication throws exception for too short password * * @param string $password * * @covers \LoginForm\Auth\Service\AuthenticationService::__construct * @covers \LoginForm\Auth\Service\AuthenticationService::authenticate * @dataProvider shortPasswordProvider * @expectedException \InvalidArgumentException */ public function testAuthenticationThrowsExceptionForTooShortPassword(string $password) { $this->validator->expects($this->once()) ->method('isValid') ->willReturn(false);

$authService = new AuthenticationService( $this->validator, $this->accountModel, $this->accountEntity, $this->twoFactorServiceMock ); $authService->authenticate('username', $password, false); $this->fail('Authentication service should throw an exception for too short password input'); }

EXAMPLE OF USING THE @COVERS ANNOTATION

/** * Authentication throws exception for too short password * * @param string $password * * @covers \LoginForm\Auth\Service\AuthenticationService::__construct * @covers \LoginForm\Auth\Service\AuthenticationService::authenticate * @dataProvider shortPasswordProvider * @expectedException \InvalidArgumentException */ public function testAuthenticationThrowsExceptionForTooShortPassword(string $password) { $this->validator->expects($this->once()) ->method('isValid') ->willReturn(false);

$authService = new AuthenticationService( $this->validator, $this->accountModel, $this->accountEntity, $this->twoFactorServiceMock ); $authService->authenticate('username', $password, false); $this->fail('Authentication service should throw an exception for too short password input'); }

SHOULD WE ADOPT 100% CODE COVERAGE?

PROS EN CONS

Pro Con

Higher quality of code A huge effort

Every line of code is accountable Requires change of attitude

Huge time/money gain in long tail Time intensive

THE PIPELINEContinuous Deployment 2.0www.in2it.be - @in2itvofin it

A COMMON CI/CD SETUP

WE CAN DO BETTER!Continuous Deployment 2.0www.in2it.be - @in2itvofin it

THE BEGINNING OF A BEAUTIFUL STORY

CODE ASSEMBLY STEP

▸ Check out SCM

▸ Optionally install submodules

▸ Composer packages

ARE WE GOOD TO GO?

CODE QUALITY STEP

▸ Version checks (are we on latest version?)

▸ Vulnerability checks (CVE’s)*

▸ Unit testing

▸ Static Analysis

(*) See SensioLabs for composer package validation serviceContinuous Deployment 2.0www.in2it.be - @in2itvofin it

LET’S PREPARE EVERYTHING LIKE A TRUE CHEF

PROVISIONING STEP

▸ Create a new environment

▸ Package the code and assets

▸ Sign the package with GPG

▸ Deploy the code

▸ Perform DB migrations

▸ Warm-up caches

TEST ALL THE THINGS, AND ADD SOME…

RELEASE QUALITY STEP

▸ Integration testing

▸ Acceptance testing

▸ End-to-end testing

▸ Regression testing

▸ Performance testing

▸ Security testing

▸ Resilience testing

▸ Accessibility testing

▸ UX Testing

▸ …

TIME TO SHIP IT!!!

DEPLOYMENT STEP

▸ Stop crons, workers and daemons

▸ Phase out old system with new

▸ Put new system in load balancer

▸ Direct portion of traffic to new node

▸ Validate behaviour of new node

▸ Remove old system from load balancer

▸ Validate deployment

▸ Start crons, workers and daemons

KNOWING IS GOOD, KNOWING EVERYTHING IS BETTER - DAVE EGGARS “THE CIRCLE”

REPORTING STEP

▸ Generate release notes

▸ Generate user documentation

▸ Notify training dept. of changes

▸ Generate release report/stats

WITH PIPELINES YOU HAVE…

ENDLESS POSSIBILITIES TO DEPLOY

AND WHEN SOMETHING DOES GOES WRONG…

THE PIPELINE WILL STOP…

WE ROLL FORWARDWHEN FAILURE HAPPENS

YOUR TURNContinuous Deployment 2.0www.in2it.be - @in2itvofin it

TEST IT ALL!CONTINUOUS DEPLOYMENT

SMALL REMINDER

BY DESIGN

▸ Quality

▸ Security

▸ Privacy

▸ Resilience

▸ Accessibility

SET AGREEMENTS AS A TEAM

IN NEED OF STRUCTURE?

WITH PIPELINESSUCCEED EVERY TIME

IMPROVE THE FEEDBACK LOOP

KNOWLEDGE IS KEY!

RESOURCES THAT INSPIRED ME…

RECOMMENDED FOR YOU

For feedback on this presentation For discussions on this presentation

QUESTIONS?ARE THERE ANY

Continuous deployment 2.0

Engineering

Transcript of Continuous deployment 2.0

20110507 Implementing Continuous Deployment

Continuous Deployment (english)

Continuous Integration and Continuous Deployment in Enterprise scenario

Continuous Deployment on AppHarbor

Case Study - Continuous Deployment at Outbrain · Continuous Deployment -Culture Definition of Done: Feature was released Monitoring shows Value . Continuous Deployment - Culture

Ansible on NonStop - Squarespace · – Using Ansible on NonStop – Using Jenkins for Continuous Deployment 5. ... – application deployment ... Jenkins Continuous Deployment without

Continuous Deployment - Lean LA

IICS Continuous Integration/Deployment

Continuous deployment-at-flipkart

Continuous deployment ASAS 2016

DEVOPS APPROACH: CONTINUOUS INTEGRATION AND ......CONTINUOUS INTEGRATION AND DEPLOYMENT 3 Components of DevOps Collaboration | Continuous Testing | Continuous Deployment COLLABORATION:

Continuous Integration + Continuous Deployment CI+CD ...

Continuous Deployment at IMVU

"Continuous Deployment" por @fredfigueiredo

Van continuous problems naar continuous deployment

Continuous Deployment

Continuous Deployment at Kitchensurfing

The Path to Continuous Deployment - dreamOlé · Continuous Deployment David Venegas Product Manager. 1. 5 Trends of Operational Excellence 2. Why Continuous Deployment 3. DevOps

Continuous Deployment Strategies

Continuous deployment @ Coolblue