Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate...
-
Upload
martha-young -
Category
Documents
-
view
213 -
download
0
Transcript of Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate...
![Page 1: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/1.jpg)
Continuous Integration for Databases
Learn how to automate your build and test
Steve JonesRed Gate Software
Part II of the Continuous Delivery for Databases series
![Page 2: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/2.jpg)
Agenda
• Goals• Who am I?• What is Continuous Integration?• Fitting in the Delivery Pipeline• What is “build” for databases?• Testing in a CI process• Adding data for realism
![Page 3: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/3.jpg)
Goals
• Why continuous integration?• Testing is important• It’s easy – setup and running in an hour
Do feel free to ask questions!
![Page 4: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/4.jpg)
Get in touch
http://voiceofthedba.wordpress.com/
@way0utwest
Steve Jones
Editor and founder, SQLServerCentral Evangelist, Red Gate Software Working with SQL Server since 1991 (v4.2) Author of many articles and books on different
aspects of SQL Server
![Page 5: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/5.jpg)
“Continuous Integration is a practice designed to ensure that your software is
always working, and that you get comprehensive feedback in a few minutes as to whether any given change to your system
has broken it.”
What is continuous integration?
Jez Humble, ThoughtWorks, author of “Continuous Delivery”
![Page 6: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/6.jpg)
“Database Continuous Integration is a practice designed to ensure that your
database software is always working, and that you get comprehensive feedback in a
few minutes as to whether any given change to your system has broken it.”
What is continuous integration?database
^
Jez Humble, ThoughtWorks, author of “Continuous Delivery”
![Page 7: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/7.jpg)
Continuous delivery – an overview
![Page 8: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/8.jpg)
Continuous delivery
START
DELIVER
1 2
3 4
![Page 9: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/9.jpg)
Database Continuous Integration
![Page 10: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/10.jpg)
What is build?
• For application code = compile• For database code = database creation
script – But only for a new installation!– Upgrade scripts required for existing
installations– Need to preserve the state of the data
![Page 11: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/11.jpg)
Thank Your Sponsors
11
![Page 12: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/12.jpg)
Tools we need• CI server
– Perform the build for us – Execute steps we program
• Choices– TeamCity by JetBrains– Jenkins – open source– Bamboo – Atlassian– TFS Build - Microsoft– Cruise Control – open source
![Page 13: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/13.jpg)
Tools we need• CI server
– Perform the build for us – Execute steps we program
• Choices – TeamCity by JetBrains– Jenkins – open source– Bamboo – Atlassian– TFS Build - Microsoft– Cruise Control – open source
![Page 14: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/14.jpg)
Tools we need• Database Integration
– Check out schema from VCS– Execute against a SQL Server instance.– Run tests
• Choices– Red Gate SQL Automation Pack– PoSh– .NET scripts/program– ?
![Page 15: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/15.jpg)
Tools we need• Version Control
– Store our database DDL and DML• Choices
– Subversion– Team Foundation Server– Git– Mercurial– Visual SourceSafe– ?
![Page 16: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/16.jpg)
Tools we need• Testing framework
– We need a way to unit test our code easily.– A framework allows us to write tests that
we can maintain• Choices
– tSQLt and SQL Test– TSQLUnit– TST
![Page 17: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/17.jpg)
Tools we need• Test data
– We want to have data for our tests• Choices
– Red Gate SQL Data Generator– Restore known backup– Custom scripts for data load (DML and or
BCP)
![Page 18: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/18.jpg)
Our story…
We want to set up a CI process for our database Our database is in a VCSo We need to automatically build a new
database on every checkin
![Page 19: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/19.jpg)
Continuous Integration Setup and data change
Demo
![Page 20: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/20.jpg)
The CI Database Setup
Developer
CI Server
Repository
Integration Database
![Page 21: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/21.jpg)
The CI Database Setup
Developer
CI Server
Repository
Integration Database
![Page 22: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/22.jpg)
CI Process
The CI Database Setup
Developer
CI Server
Repository
Integration Database
![Page 23: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/23.jpg)
CI Process
The CI Database Setup
Developer
CI Server
Repository
Integration Database
Random Test Database
![Page 24: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/24.jpg)
CI Process
The CI Database Setup
Developer
CI Server
Repository
Integration Database
![Page 25: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/25.jpg)
Keeping a database up to date
With?• Schema• Static data
Why?• A corresponding database for the
application• Maintaining a test database with the latest
changes
![Page 26: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/26.jpg)
TESTINGWhy it’s important
![Page 27: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/27.jpg)
Where does testing happen?
• Testing isn’t just done in QA• Be aware of the cost of fixing a bug
![Page 28: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/28.jpg)
Cost of Bugs
![Page 29: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/29.jpg)
Does the cost of bugs rise?
![Page 30: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/30.jpg)
Does the cost of bugs rise?
![Page 31: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/31.jpg)
![Page 32: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/32.jpg)
![Page 33: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/33.jpg)
Testing in Development
• Low(er) costs– No inter-team interactions– Changes are discrete
• However– Increased frequency of changes (higher
cost)
![Page 34: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/34.jpg)
What is test?
• For .NET code, Nunit– Runs on a developer’s machine and
build server• What about the database?
– tSQLt is an open source framework for testing SQL Server databases
– tSQLt.org– Support via GoogleGroups– SQL Test provides SSMS integration
![Page 35: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/35.jpg)
Our story…
We want to set up a CI process for our database Our database is in a VCS We need to automatically build a new
database on every check-ino Let’s add a unit test for our code to the CI
process.
![Page 36: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/36.jpg)
Testing the Build
Demo
![Page 37: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/37.jpg)
Why generate test data?
• Dev environments are often not realistic• Getting production data not always
possible• Random data can result in surprises• Volume testing can find performance
issues
![Page 38: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/38.jpg)
Our story…
We want to set up a CI process for our database Our database is in a VCS We need to automatically build a new
database on every check-in Let’s add a unit test for our code to the CI
process.o Let’s generate test data to use in our CI
process
![Page 39: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/39.jpg)
Demo
Testing with larger data sizes
![Page 40: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/40.jpg)
Two bugs found by Test Data
1. NULL Dates– App code assumed [Date] wouldn’t be
NULL– Test data didn’t– Fix was to change [Date] to be NOT
NULL2. Performance of v_Articles
– Test passed on dev box– Failed in more “realistic” CI
environment
![Page 41: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/41.jpg)
Deployments Scripts
• Creation scripts – For new installations
• Upgrade Scripts – For existing installations
• Developed by comparing our up-to-date database to production/test
![Page 42: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/42.jpg)
Continuous delivery
START
DELIVER
1 2
3 4
![Page 43: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/43.jpg)
The End
• Questions?• More information: www.red-gate.com/CI• Please fill out your feedback forms• www.voiceofthedba.com/talks
![Page 44: Continuous Integration for Databases Learn how to automate your build and test Steve Jones Red Gate Software Part II of the Continuous Delivery for Databases.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649daf5503460f94a9d3ba/html5/thumbnails/44.jpg)
References
http://assets.red-gate.com/products/sql-development/assets/continuous-integration-using-red-gate-tools.pdf
http://www.jetbrains.com/teamcity/ http
://developers.slashdot.org/story/03/10/21/0141215/software-defects---do-late-bugs-really-cost-more
http://tech.lds.org/index.php?option=com_content&view=article&id=238:the-cost-of-bugs&catid=1:miscellanous
http://www.manageware.co.il/solution/portfolio/auto-deploy/