Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

No Outage Database Development with

Spring Boot and Liquibase

Alan Barrington-Hughes Pavithra Ramaswamy

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agenda

• Goal

• We'll tie together several concepts to perform a no-outage deployment which includes database DDL changes

• Topics

• Blue Green / Canary deployments

• Agile database development concepts

• Introduction to liquibase

• Embedding liquibase db changes within spring boot

• Live demo

2

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Blue Green deployments - central tenets

• You'll have two distinct versions of the codebase infrastructure fronted by a router

• One version is customer facing live (Blue), the other dormant (Green)

• Green acts a stage, into which you deploy the next release

• Upon the warm & fuzzy sign off, the router is instructed to re-direct traffic to Green

• Blue then becomes the dormant (hot standby) system and Green is live

• The following release uses Blue as stage and customer facing line (Green)

• And so on

• Gives a rapid mechanism to roll back

3

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Canary deployments - central tenets

• You'll have instances of the codebase infrastructure fronted by a router

• All systems are running in parallel

• Rather than a complete cut over, only a small portion of the instances are upgraded with new code

• As you gain confidence in the new release more instances are upgraded and receive more traffic until the old version becomes dormant

4

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Database considerations

• Databases can often be a challenge with these technique. Particularly when you need to change the schema to support a new version of the software.

• The trick is to apply a database refactoring that changes the schema to support both the new and old version of the application

• So first, deploy that canary/ green build, check everything is working fine so you have a rollback point, then roll out the new version of the application.

• And when the upgrade has bedded down remove the database support for the old version

5

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development concepts

• Large feature releases are high risk and more likely for deployment outage

• Small (more frequent) feature release are lower risk and more manageable to deploy without an outage

• Developers are used to the concept of refactoring code when "code smells", databases smell also and are ripe for refactoring

• A large change should be broken down and completed via several small changes

6

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings• Structural changes

• Renaming a column

7

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings

• Structural changes

• Renaming a column

• Why?

• Increases the readability of the schema / needed to conform to the enterprise naming conventions

8

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings

• Structural changes

• Renaming a column

• Why?

• Increases the readability of the schema / needed to conform to the enterprise naming conventions

• How?

• introduce new column

• migrate original data

• introduce synchronization trigger

• (after a deprecation period)

• drop original column & trigger

9

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings

• Structural changes

• Moving a column from one table to another

10

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings

• Structural changes

• Moving a column from one table to another

• Why?

• Increase normalization to reduce data redundancy

• Denormalization to reduce common joins

11

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings

• Structural changes

• Moving a column from one table to another

• Why?

• Increase normalization to reduce data redundancy

• Denormalization to reduce common joins

• How?

• identify insertion & deletion rules

• introduce new column

• migrate data

• introduce triggers

• (after a deprecation period)

• drop original column & triggers

12

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings• Structural changes

• Splitting a multipurpose column into several separate columns

13

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings• Structural changes

• Splitting a multipurpose column into several separate columns

• Why?

• Increased data granularity ("name" -> "fName", "mName", "lName")

• Overloaded status field

14

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development refactorings• Structural changes

• Splitting a multipurpose column into several separate columns

• Why?

• Increased data granularity ("name" -> "fName", "mName", "lName")

• Overloaded status field

• How?

• introduce new columns

• migrate data (may need transforming stored proc)

• introduce triggers

• refactor application code

• (after a deprecation period) drop original column and triggers

15

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Agile database development - lessons learned

• Smaller changes are easier to apply

• Uniquely identify refactorings

• Use triggers over views for data synchronization

• Put database assets under change control

16

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Introduction to liquibase - fundamentals

• essentially a scripting language

• implemented and executed in java

• controls the modifications to the schema of a database

• manages/ records all changes to your database

• no change is forgotten

• ability to roll back a release (and re-roll forward)

• out of the box support for multiple DBs (mysql / postgresql / oracle /mssql / derby / hsqldb / h2

• extensible for other databases

17

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Introduction to liquibase - what why how

• snapshot comparison

• change set qualification

• verification

• atomic application

• ensures singleton upgrades

• changes recorded

18

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Introduction to liquibase - changes

• Changelogs (collection of change sets can span many files)

• Changesets (As few changes as possible per change set)

• Changes (Refactorings such as add column/ create table)

• Think small additive changes to the database

• Coded in YAML, JSON or XML

• Liquibase translates change to vendor specific SQL

• Liquibase can automatically rollback sequences of changes

19

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Introduction to liquibase - functionality

• Applying all DDL changes to the schema

• createTable / dropTable / createProcedure / addPrimaryKey / addIndex /

addLookupTable / mergeColumns /sql blocks

• Managing views / stored procedures / triggers

• manage static / reference data

• manage test data

20

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Embedding liquibase within spring boot

21

// This the code required to enable liquibase in spring boot:

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Embedding liquibase within spring boot

22

// This the code required to enable liquibase in spring boot:

// There are however, a few steps of configuration a) You need to add “liquibase-core” dependency.

<dependency>    <groupId>org.liquibase</groupId>     <artifactId>liquibase-core</artifactId>      <version>3.4.1</version></dependency>

b) Put your first change log in “db/changelog/db.changelog-master.yaml”

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Embedding liquibase within spring boot

23

// It goes without saying that a database migration tool also needs a database to migrate:

spring: datasource: driverClassName: org.postgresql.Driver username: demo password: "demo_pass" url: jdbc:postgresql://${PG_PORT_5432_TCP_ADDR}:${PG_PORT_5432_TCP_PORT}/sboot_demo

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Live demo - goals

24

• Two spring boot nodes sharing the same database fronted by ngnix

• Goal is to move a "email" column from the current "user" table into a new table

• Accomplished without the app becoming unavailable.

• Start with initial version v1.0 that creates our tables & app code

• Deploy our new v1.1 code, introducing new column, migrating data, keeping

data in sync using triggers.

• v1.0 and v1.1 are running in parallel and co-existing with the new DB schema

• Once all nodes are on v1.1 remove old column and drop triggers

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Live demo

25

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Footnotes / links

• http://martinfowler.com/bliki/BlueGreenDeployment.html

• http://martinfowler.com/bliki/CanaryRelease.html

• http://www.liquibase.org

• Refactoring Databases ISBN-13: 978-0-321-77451-4

• https://github.com/abhsc/sboot_liquibase_demo

26

Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /

Thank you! (and please rate this session)

• Alan Barrington-Hughes

• https://www.linkedin.com/in/alanbarringtonhughes

• @abhPremier

• https://github.com/abhsc

27

• Pavithra Ramaswamy

• https://www.linkedin.com/in/pramaswamy

• @pavi2master

• https://github.com/pavi2master

Premier, Inc. (NASDAQ: PINC) is a healthcare performance improvement alliance of approximately 3,600 U.S. hospitals and 120,000 other providers. Our mission is simple: To improve the health of communities.

We Are Hiring @PremierHATalent