DevOps 101 for data professionals
-
Upload
alex-yates -
Category
Technology
-
view
188 -
download
0
Transcript of DevOps 101 for data professionals
![Page 1: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/1.jpg)
@_AlexYates_#GroupBy
DevOps 101 for Data Professionals
How your job will change
Alex Yates
![Page 3: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/3.jpg)
@_AlexYates_#GroupBy
@_AlexYates_ | #GroupBy
![Page 4: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/4.jpg)
@_AlexYates_#GroupBy
AgileScrumLeanIterativeContinuous Delivery
Developers
![Page 5: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/5.jpg)
@_AlexYates_#GroupBy
MonitoringDeployment
IntegrityPerformance
DBAs
![Page 6: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/6.jpg)
@_AlexYates_#GroupBy
2009: A regular session, at a regular conference, that changed everything
https://www.youtube.com/watch?v=LdOe18KhtT4/
@allspaw
@ph
![Page 7: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/7.jpg)
@_AlexYates_#GroupBy
A brief history of DevOps
• 2008: Agile Toronto “Agile Infrastructure”• 2008: “Agile System Administrators” Google group• 2009 (June): “10+ Deploys Per Day”• 2009 (October): DevOps Days Ghent• #DevOps• 2013: The Phoenix Project• 2016: The DevOps handbook
@littleidea @patrickdebois
@allspaw
@ph
@realGeneKim
@jezhumblehttps://www.amazon.co.uk/d/Books/Phoenix-Project-DevOps-Helping-Business/0988262509/https://www.amazon.co.uk/Devops-Handbook-World-Class-Reliability-Organizations/dp/1942788002/
![Page 8: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/8.jpg)
@_AlexYates_#GroupBy
Things that aren’t DevOps…Tools
People, job roles or teams
Processes or task lists Floating masses of visible, condensed water vapour
http://itrevolution.com/the-three-ways-principles-underpinning-devops/
![Page 9: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/9.jpg)
@_AlexYates_#GroupBy
CALMS
C ultureA utomationL eanM easurementsS haring
![Page 10: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/10.jpg)
@_AlexYates_#GroupBy
The three ways
Systems thinking
Amplify feedback loops
Culture of continual experimentation and learning
http://itrevolution.com/the-three-ways-principles-underpinning-devops/
@realGeneKim
![Page 11: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/11.jpg)
@_AlexYates_#GroupBy
The automated deployment pipeline
![Page 12: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/12.jpg)
@_AlexYates_#GroupBy
Databases are hard
![Page 13: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/13.jpg)
@_AlexYates_#GroupBy
Farm Credit Services of America (FCSA)
• 100 person IT team, 14 sub-teams• Database version control inconsistent
• Deployment process manual• Delivery was slow and
unreliable
![Page 14: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/14.jpg)
@_AlexYates_#GroupBy
• Standardised source control and delivery process
• Automated manual deployment tasks
Farm Credit Services of America (FCSA)
• Deployments easier to review
• Delivery faster and more reliable
![Page 15: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/15.jpg)
@_AlexYates_#GroupBy
Farm Credit Services of America (FCSA)
http://www.codeaperture.io/2016/09/13/how-redgate-helped-define-our-process/
![Page 16: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/16.jpg)
@_AlexYates_#GroupBy
There’s more than one way to skin a cat
![Page 17: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/17.jpg)
@_AlexYates_#GroupBy
There’s more than one way to skin automatea cat
![Page 18: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/18.jpg)
@_AlexYates_#GroupBy
V1 V2
Migrations vs state
![Page 19: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/19.jpg)
@_AlexYates_#GroupBy
V1 V2
Migrations vs state
![Page 20: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/20.jpg)
@_AlexYates_#GroupBy
V1 V2
Migrations vs state
![Page 21: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/21.jpg)
@_AlexYates_#GroupBy
![Page 22: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/22.jpg)
@_AlexYates_#GroupBy
“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”
![Page 23: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/23.jpg)
@_AlexYates_#GroupBy
“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”
Paul Stovell, Octopus Deploy
http://docs.octopusdeploy.com/display/OD/SQL+Server+databases
![Page 24: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/24.jpg)
@_AlexYates_#GroupBy
“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”
![Page 25: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/25.jpg)
@_AlexYates_#GroupBy
“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”
Gert Drapers, built DataDude
https://blogs.msdn.microsoft.com/gertd/2009/06/05/declarative-database-development/
![Page 26: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/26.jpg)
@_AlexYates_#GroupBy
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
![Page 27: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/27.jpg)
@_AlexYates_#GroupBy
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
![Page 28: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/28.jpg)
@_AlexYates_#GroupBy
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
![Page 29: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/29.jpg)
@_AlexYates_#GroupBy
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
![Page 30: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/30.jpg)
@_AlexYates_#GroupBy
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
![Page 31: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/31.jpg)
@_AlexYates_#GroupBy
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
![Page 32: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/32.jpg)
@_AlexYates_#GroupBy
Migrations vs state
http://workingwithdevs.com/delivering-databases-migrations-vs-state/
![Page 33: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/33.jpg)
@_AlexYates_#GroupBy
11
2 Alter view: kittenTrainersNrLdn
Alter table: kittenTrainers
Both work!Same changes, versioned and deployed two ways
![Page 34: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/34.jpg)
@_AlexYates_#GroupBy
Demo
![Page 35: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/35.jpg)
@_AlexYates_#GroupBy
http://thenextweb.com/insider/2013/12/12/dba-dead/
@kennygorman
![Page 36: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/36.jpg)
@_AlexYates_#GroupBy
http://www.scarydba.com/2015/02/24/devops-dba-word-no/ @GFritchey
![Page 37: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/37.jpg)
@_AlexYates_#GroupBy
http://www.devopsdba.com/devops-dba/
@DevOpsDBA
![Page 38: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/38.jpg)
@_AlexYates_#GroupBy
What you can do• Don’t be the bottleneck – be the fix• Embrace co-operation, automation and DLM• (Learn Git, tSQLt, NuGet and PowerShell etc)
• Pick a technical solution that suits your database/team• Design for deployability and testability• Data Platform, Cloud (not just on-prem SQL Server)• Be careful with the word “no”
![Page 39: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/39.jpg)
@_AlexYates_#GroupBy
DLMConsultants.com/DevOps-101/
![Page 40: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/40.jpg)
![Page 41: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/41.jpg)
@_AlexYates_#GroupBy
Questions? Links: DLMConsultants.com/DevOps-101/ Rate me: GroupBy.org/go/session7Contact me: [email protected]
![Page 42: DevOps 101 for data professionals](https://reader035.fdocuments.in/reader035/viewer/2022062522/589f6fc01a28abbf2e8b53f3/html5/thumbnails/42.jpg)
@_AlexYates_#GroupBy
Image sourcesAuthor Source Information
Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work”
Bit Boy Flickr The elephant in the room – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Nils Rinaldi Flickr Hippo fight 2/3 – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
My own collection Taken by/property of Alex Yates Kitten, “There’s more than one way to skin a cat!”
Memegenerator.net Memegenerator.net I don’t always edit database. Content designed to be shared and delivered with credit to memegenerator.net.
BagoGames Flickr NBC takes another Whack at the IT crowd. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Biso Wikimedia Commons Rusty tools.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.
Omnisource5 Wikimedia Commons Lenticular Cloud over Harold's Cross Dublin Ireland taken at 11:30AM, 30th June 2015 – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license. Source on Wikimedia Commons: “Own work”