The road to continuous deployment (PHPCon Poland 2016)
-
Upload
michiel-rook -
Category
Software
-
view
837 -
download
1
Transcript of The road to continuous deployment (PHPCon Poland 2016)
▸ Java, PHP & Scala developer
▸ Consultant, trainer, speaker
▸ make.io
▸ Dutch Web Alliance
▸ Maintainer of Phing
▸ @michieltcs
THIS TALK
▸ Background
▸ The approach
▸ Process / standards
▸ Build pipelines
▸ Results & lessons learned
THE SYSTEM - SAN DIEGO
▸ ... or the Big Ball Of Mud
▸ Large legacy monolith
▸ Generates significant income
▸ Slow & complex
▸ Technical debt
SAN DIEGO FRONTEND
MYSQL DB
SAN DIEGO BACKEND
LOAD BALANCERS / VARNISH
ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK
SAN DIEGO FRONTEND
SAN DIEGO FRONTEND
SAN DIEGO FRONTEND
SAN DIEGO BACKEND
SAN DIEGO BACKEND
SAN DIEGO BACKEND
MEMCACHE FTP EXT. SERVICES
SOLR
THE SYSTEM - SAN DIEGO
▸ Infrequent, manual releases
▸ Fragile tests
▸ Low velocity
▸ Frequent outages / bugs / issues
▸ Frustrated team
▸ Low confidence modifying existing code
APPROACH
▸ Strangler pattern
▸ Services per domain object (job, jobseeker, ...)
▸ Proxy to switch between old/new
▸ Migrate individual pages
ORIGINAL MONOLITH
PROXY
SERVICEORIGINAL MONOLITH
ORIGINAL MONOLITH
SERVICE SERVICE
SERVICE
PROXY
DB
DBDB
DB
DB DB
APPROACH
▸ Services behind load balancers
▸ Access legacy db’s
▸ Continuous deployment
▸ Docker containers
▸ Frontends are services
SAN DIEGO
ELASTIC SEARCHLEGACY
DB
JOB SERVICE
RMQ
ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK
MONGO DB
ITBANEN
JOBSEEKER SERVICE
NVBINTERMEDIAIR
STARTING OFF
▸ Scrum, 1 week sprints
▸ TDD / BDD
▸ Definition of Done
▸ Team mindset / experience
▸ Focus on value
▸ Replace old features with new (legacy becomes obsolete)
WHY CONTINUOUS DEPLOYMENT
▸ Small steps
▸ Less overhead
▸ Early feedback
▸ Reduce cycle time
▸ Reduce risk
DEFENSE IN DEPTH
UNIT TESTS
INTEGRATIONTESTS
ACCEPTANCE
UI TESTS
public function testJobCannotBeFound() { $jobRepository = $this->prophesize(JobRepository::class); $jobRepository->getById(EXPECTED_JOB_ID) ->shouldBeCalled() ->willReturn(false); $jobService = new JobService($jobRepository->reveal()); $this->assertFalse($jobService->getById(EXPECTED_JOB_ID)); }
DEFENSE IN DEPTH
UNIT TESTS
INTEGRATIONTESTS
ACCEPTANCETESTS
UI TESTS
public function testFindJob() { $expectedJob = $this->loadFixture('active_job.yml'); $actualJob = $this->repository->getById($expectedJob->getId()); self::assertInstanceOf(Job::class, $actualJob); self::assertEquals($expectedJob->getId(), $actualJob->getId()); }
DEFENSE IN DEPTH
UNIT TESTS
INTEGRATIONTESTS
ACCEPTANCETESTS
UI TESTSScenario: Link to related job Given a job exists And there are related jobs available When that job is viewed Then a list of related jobs is shown And each related job links to the detail page of the related job
CONTINUOUS TESTING
UNIT TESTS
INTEGRATION TESTS
ACCEPTANCE TESTS
UI TESTS
SMOKETESTS
Cost Speed
Exploratorytesting Monitoring
PIPELINE AS CODE
node { stage 'Run tests' sh "phpunit" sh "behat" stage 'Build docker image' sh "phing build" sh "docker build -t jobservice:${env.BUILD_NUMBER} ." sh "docker push jobservice:${env.BUILD_NUMBER}" stage 'Deploy acceptance' sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER} -i acc deploy.yml" stage 'Deploy production' sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER} -i prod deploy.yml" }
DOCKERFILE
FROM php:7.0-apache
ADD vhost.conf /etc/apache2/sites-available/000-default.conf
ADD . /var/www/html
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
docker pull
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
docker run
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
wait_for: port=8080 delay=5 timeout=15
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
uri: url: http://localhost:8080/_health status_code: 200 timeout: 30
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg
service: name=haproxy state=reloaded
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg
service: name=haproxy state=reloaded
DEPLOYING
PULL IMAGE
START NEW CONTAINER
WAIT FOR PORT
SMOKE TESTS / HEALTH CHECKS
ADD NEW CONTAINER TO LB
REMOVE OLD CONTAINER FROM LB
STOP OLD CONTAINER
docker stop
docker rm
RESULTS
▸ Total build time per service < 10 minutes
▸ Significantly improved page load times
▸ Improved audience stats (time on page, pages per session, session duration, traffic, seo ranking, etc)
▸ Increased confidence and velocity
▸ Experimented with new tech/stacks (angular, jvm, event sourcing)
▸ More fun
LESSONS LEARNED
▸ Team acceptance
▸ Change is hard
▸ Overhead of weekly sprint; requires discipline
▸ Docker orchestration
▸ Issues with traffic between Amazon <-> on-premise datacenter
▸ Javascript testing
LESSONS LEARNED
▸ Experience with new tech
▸ Stability of build pipelines
▸ Management/leadership buy-in
▸ Business alignment
▸ Not enough focus on replacing legacy application