Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release...
Transcript of Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release...
![Page 1: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/1.jpg)
December4–9,2016|Boston,MAwww.usenix.org/lisa16#lisa16
ReleasePipelinesinMicrosoftEcosystems
WarrenFrame,HarvardUniversity
MichaelGreene,Microsoft
![Page 2: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/2.jpg)
whoami• WarrenFrame
• ResearchComputingatHarvardUniversity
• @pscookiemonster• Ramblingcookiemonster• wframe
• MichaelGreene• EnterpriseCloudEngineeringCAT
TeamatMicrosoft• @migreene• mgreenegit• migreene
bit.ly/lisa16pipeline
![Page 3: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/3.jpg)
Stuff• Slides• Demos!• Slidesatbit.ly/lisa16pipeline• Cleanup,
![Page 4: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/4.jpg)
ConfigurationasCode• Everything-as-a-service,APIsgalore• Livingdocumentation• Abstractoutcomplexity.Scripts->Modules->DSC->key:value• PowerShellDSCisaplatformthatallsolutionscanusetodeployand
manageWindowsServer• AzureResourceManagertemplates• Youstillneedtoknowtheunderlyingsystemsyouwillmanage• Releasepipelinescanbringsanityandconsistencytomanagingthis
![Page 5: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/5.jpg)
ReleasePipeline
aka.ms/trpmWhy?
![Page 6: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/6.jpg)
ReleasePipelines
Prodenvironment(etc.)…• Systems/Services• Modules• Scripts• Config files
![Page 7: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/7.jpg)
ExampleWorkflow• Makeachange,pushtosourcecontrol*• Buildsystem doestherest.Forexample:
• Runtestsagainstyourcode• Spinuptestservices/infrastructureformoretests• Buildartifacts(packages,configs,etc.)• Deploythings(artifacts,systems,services,etc.)
*Youmightrunthroughsource-build-testloopslocallyuntilhappy,beforepushing
![Page 8: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/8.jpg)
Tooling“abunchofrandomopensourceprojectsboundtogetherwith
ducttapeandchewinggum”
![Page 9: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/9.jpg)
Tools:SourceGit?Mercurial?SVN?CLI:• Git forWindows• PoshGitGUI:• GitHubDesktop• Atlassian SourceTree• Manyothers
![Page 10: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/10.jpg)
Demo:SourceGitVisualStudioCode
![Page 11: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/11.jpg)
Tools:BuildSystems• Jenkins,GitLab CI,VSTS,etc.• Preferbuild-as-code
• e.g.Jenkinsfile,appveyor.yml,.gitlab-ci.yml
![Page 12: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/12.jpg)
Tools:BuildAutomation• Invoke-Build,psake• Similartorake,make,bake,cake,grunt,gulp,msbuild,etc.
![Page 13: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/13.jpg)
Demo:BuildTFS2017psake (buildautomation)github.com/powershell/demo_ci
![Page 14: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/14.jpg)
Tools:Testing• Pester: Testframework• poshspec: infrastructuretesting• OVF: Operation-Validation-Framework- simplifyorganizing,
execution,andsharingoftests.
![Page 15: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/15.jpg)
Demo:TestPesterposhspec
![Page 16: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/16.jpg)
Tools:Release• OctopusDeployandVSTS
• Manypre-cannedtasks• Flexible• Pretty• Potentially$$
• PSDeploy• Somepre-cannedtasks• Deploymentascode• Poorlywritten• Opensource
• RandomPowerShellcode• Funtoreadandmaintain!
![Page 17: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/17.jpg)
Demo:ReleaseTFS2017- Releasemanagement
![Page 18: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/18.jpg)
Tools:TestHarness• Test-Kitchen• NotjustforChef• Roughly:
• Runtestswithaverifier(Pester)
• againstplatforms(differentvagrantboxes)
• convergedwithaprovisioner (dsc)
• withthelifecyclemanagedbyadriver(vagrant)
• Andtest,configuration,otherfilescopiedtoplatforms viaatransport(WinRM)
Drivers• AmazonEC2• AzureResourceManager• DigitalOcean• Docker• GoogleComputeEngine• Hyper-V• OpenStack• Vagrant• vRealizeAutomation,Orchestrator• vSphereProvisioners• Ansible• CFEngine• ChefSolo,Zero• DSC• Puppet• Salt• ShellVerifiers• Inspec• Pester• Shell(Bats,Serverspec,etc.)
![Page 19: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/19.jpg)
Source: GitHubBuild: AppVeyor
Builddependencies: PSDependBuildautomation: Invoke-BuildBuildhelpers: BuildHelpers
Test: PesterRelease: PSDeploy
ExamplePipeline
![Page 20: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/20.jpg)
Demo:ExamplePipelinehttps://github.com/RamblingCookieMonster/lisa-kitchen-demo
![Page 21: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/21.jpg)
Whatabout…• Secrets
• Insourcecontrol• Builtintobuildsystem?• Secretmanagement– vault,passwordstate,SecretServer,credstash,etc.
• Images• Packer!• Images-as-code• BuildimagesforAmazon,VirtualBox,Azure,Hyper-V(ish),etc.
![Page 22: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/22.jpg)
Wheretostart• SourceControland/orTests overentirepipelineatonce• Existingtools overresume-driven-development• Newservice(s)/valueproposition overre-engineering
everything• Noluckinhouse?PlaywithGitHub+AppVeyor,VSTS,etc.
![Page 23: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/23.jpg)
Nextsteps• Opensourceprojectscoulduseyourhelp!• JITprovisioningoradynamicpoolsofWindowsbuildagents• WindowsDockercontainersfortesting• Focusonephemeraldeploymentsoverincrementalchanges• Planforday100
![Page 24: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/24.jpg)
CommunityProjects
Mapyourrequirements
Plaster Invoke-Build,psake
Pester
poshspec
OVF
PSDeploy
Lability
PowerShellSlack
![Page 25: Release Pipelines in Microsoft Ecosystems - USENIX · References, Diving Deeper • The Release Pipeline Model-Michael Greene, Steven Murawski • Building a Simple Release Pipeline](https://reader030.fdocuments.in/reader030/viewer/2022040706/5e05956b5b612b32ad25a1ff/html5/thumbnails/25.jpg)
References,DivingDeeper• TheReleasePipelineModel - MichaelGreene,StevenMurawski
• BuildingaSimpleReleasePipelineinPowerShellUsingpsake,Pester,andPSDeploy - BrandonOlin
• StackOverflow:HowWeDoDeployment- 2016Edition - NickCraver
• DevOpsReadingList - StevenMurawski• ReadingList - ChrisHunt
• ThePesterPipeline- ChrisHunt
• BestPracticeswithPackerandWindows - MattHodgkins
• IntroductiontoKitchen-DSC - GaelColas
• TestingAnsibleRolesAgainstWindowswithTest-Kitchen- MattHodgkins• Twitter,Slack,andothercommunities
• Etc.