© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Steve Abraham, Solutions Architect

Shree Kenghe, Solutions Architect

November 29, 2016

Workshop: Using the Database Migration

Service (DMS) for Database Consolidation,

Data Distribution, and Replication

DAT321

What to Expect from the Session

Agenda

• Housekeeping

• Migrating to AWS

• SCT Overview

• DMS Overview

• Q&A

• Workshop

Housekeeping

Download:

http://chilp.it/203c7aa

Housekeeping

Migrating to AWS

• Quickly provision databases

• Multiple Availability Zones

• Rapid scaling

• Automated patching

• Easy read replica creation

• High durability

• Point in time recovery

• Detailed metrics

• Single-click encryption at rest

Amazon

RDS

Why AWS?

• How will my on-premises data migrate to the cloud?

• How can I make it transparent to my users?

• How will on-premises and cloud data interact?

• How can I integrate my data assets within AWS?

• How can I move off of commercial databases?

How?

Migration Options

• Lift and shift

• Leverage Amazon EC2 and Amazon S3

• Keep existing DB engine but migrate to Amazon RDS

• For example Oracle on-premises to RDS Oracle

• Migrate database engine

• Commercial engine to open source

• Maintenance window

• Maintenance window duration vs. CDC with 0 downtime

Database Migration Process

Phase Description Automation Effort (%)

1 Assessment SCT 5

2 Database schema conversion SCT/DMS 10

3 Application conversion/remediation SCT 25

4 Scripts conversion SCT 6

5 Integration with third-party applications 4

6 Data migration DMS 4

7 Functional testing of the entire system 30

8 Performance tuning SCT 2

9 Integration and deployment 7

10 Training and knowledge 2

11 Documentation and version control 2

12 Post production support 3

Database Migration Phases

AWS Schema Conversion Tool

Overview

AWS Schema Conversion Tool

Features

• Converts schema of one database engine to another

• Database Migration Assessment report for choosing the best target engine

• Code browser that highlights places where manual edits are required

The AWS Schema Conversion Tool helps

automate many database schema and code

conversion tasks when migrating from Oracle

and SQL Server to open source database

engines.

Convert Tables, Views, and Code

• Sequences

• User Defined Types

• Synonyms

• Packages

• Stored Procedures

• Functions

• Triggers

• Schemas

• Tables

• Indexes

• Views

Components of the Console

1. Source Schema

2. Action Items

3. Target Schema

4. Schema Element Details

5. Edit Window

Database Migration Assessment

1. Connect Schema

Conversion Tool to

source and target

databases.

2. Run Assessment

Report.

3. Read Executive

Summary.

4. Follow detailed

instructions.

Supported Conversions

Source Database Target Database

Microsoft SQL Server Amazon Aurora, Microsoft SQL Server, MySQL, PostgreSQL

MySQL MySQL, PostgreSQL

Oracle Amazon Aurora, MySQL, Oracle, PostgreSQL

Oracle Data Warehouse Amazon Redshift

PostgreSQL Amazon Aurora, MySQL, PostgreSQL

Teradata Amazon Redshift

$0for software license

Allowed Use

Use Schema Conversion Tool to migrate database

schemas to Amazon RDS, Amazon Redshift, or Amazon

EC2–based databases

To use Schema Conversion Tool to migrate schemas to

other destinations, contact for special pricing

Pricing

Free software license

For active AWS customers with

accounts in good standing

Pricing, Terms & Conditions

Prerequisites

• Create Databases• Source

• Target

• Download AWS Schema Conversion Tool• http://amzn.to/2b2YE2a

• Download Drivers• http://amzn.to/2axE0Hn

• Update Global Settings

Global Settings – Logging

Global Settings – Drivers

Global Settings – Performance and Memory

Global Settings – Assessment Report

Time Considerations

• Any Hard Dates?

• Planning Time?

• Typically 2-3 Weeks

• Several Iterations

Database Considerations

• Number of Schemas?

• Number of Tables?

• Engine Specific Types?

• Users/Roles/Permissions?

Network Considerations

• Access (Firewalls, Tunnels, VPNs)?

• Which VPC?

• Which Security Groups?

Requirements Considerations

• Engine Selection Criteria?

• Which Tables Need to Move?

• Same Target For All Tables?

Temporary Work Schema

• Added to target DB instance

• After migration delete temporary work schema

• Schemas

• Microsoft SQL Server: AWS_SQLSERVER_EXT

• MySQL: AWS_MYSQL_EXT

• Oracle: AWS_ORACLE_EXT

• PostgreSQL: AWS_POSTGRESQL_EXT

DMS Overview

• Start your first migration in 10 minutes or less

• Keep your apps running during the migration

• Replicate within, to, or from Amazon EC2 or RDS

• Move data to the same or a different database engine

AWS

Database Migration

Service

(AWS DMS)

Customer

premises

Application users

AWS

Internet

VPN

• Start a replication instance

• Connect to source and target databases

• Select tables, schemas, or databases

Let AWS DMS create tables, load data, and keep them in sync

Switch applications over to the target at your convenience

Keep your apps running during the migration

AWS

DMS

Multi-AZ option for high availability

Customer

premises

or AWS

AWS

Internet

VPN

AWS DMS

AWS DMS

AWS Database Migration service pricing

T2 for developing and periodic data migration tasks

C4 for large databases and minimizing time

T2 pricing starts at $0.018 per hour for T2.micro

C4 pricing starts at $0.154 per hour for C4.large

50 GB GP2 storage included with T2 instances

100 GB GP2 storage included with C4 instances

Data transfer inbound and within AZ is free

Data transfer across AZs starts at $0.01 per GB

Migration Scenarios and Options

On-Premises Migration Scenarios

• An on-premises database to a database on Amazon RDS DB

instance

• An on-premises database to a database on an Amazon EC2

instance

• Migration from an on-premises database to another on-premises

database is not supported.

RDS Migration Scenarios

• A database on an Amazon RDS DB instance to an on-premises

database

• A database on an Amazon RDS DB instance to a database on an

Amazon RDS DB instance

• A database on an Amazon RDS DB instance to a database on an

Amazon EC2 instance

EC2 Migration Scenarios

• A database on an Amazon EC2 instance to an on-premises

database

• A database on an Amazon EC2 instance to a database on an

Amazon EC2 instance

• A database on an Amazon EC2 instance to a database on an

Amazon RDS DB instance

DMS Components

• Replication Instances

• Endpoints

• Tasks

Replication Instances

• Performs the work of the migration

• Tasks run on instances

• Can support multiple tasks

• AWS DMS currently supports T2 and C4 instance classes for

replication instances

Public and Private Replication Instances

• A replication instance should have a public IP address if the source

or target database is located in a network that is not connected to

the replication instance's VPC by using a virtual private network

(VPN), AWS Direct Connect, or VPC peering.

• A replication instance should have a private IP address when both

the source and target databases are located in the same network

that is connected to the replication instance's VPC by using a VPN,

AWS Direct Connect, or VPC peer.

Sources for AWS Database Migration Service

Customers use the following databases as a source for data migration using AWS

DMS: On-premises and Amazon EC2 instance databases:

• Oracle Database 10g – 12c

• Microsoft SQL Server 2005 – 2014

• MySQL 5.5 – 5.7

• MariaDB (MySQL-compatible data source)

• PostgreSQL 9.4 – 9.5

• SAP ASE 15.7+

RDS instance databases:

• Oracle Database 11g – 12c

• Microsoft SQL Server 2008R2 - 2014. CDC operations are not supported yet.

• MySQL versions 5.5 – 5.7

• MariaDB (MySQL-compatible data source)

• PostgreSQL 9.4 – 9.5

• Amazon Aurora (MySQL-compatible data source)

Targets for AWS Database Migration Service

Customers can use the following databases as a target for data replication using

AWS DMS:On-premises and EC2 instance databases:

• Oracle Database 10g – 12c

• Microsoft SQL Server 2005 – 2014

• MySQL 5.5 – 5.7

• MariaDB (MySQL-compatible data target)• PostgreSQL 9.3 – 9.5

• SAP ASE 15.7+

RDS instance databases:

• Oracle Database 11g – 12c

• Microsoft SQL Server 2008 R2 - 2014

• MySQL 5.5 – 5.7

• MariaDB (MySQL-compatible data target)

• PostgreSQL 9.3 – 9.5• Amazon Aurora (MySQL-compatible data target)

• Amazon Redshift

Tasks Overview

• Run on a replication instance

• Contain two and only two endpoints (source and target)

• Different migration methods available

• Specify selection and/or transformation rules

• Can run multiple tasks

Migration Methods

• Migrate existing data

• Migrate existing data and replicate ongoing changes

• Replicate data changes only

DMS – Change Data Capture (CDC)

“No Touch” design

• Reads recovery log of source database

• Using the engine’s native change data capture API

• No agent required on the source

Some requirements

• Oracle: Supplemental logging required

• MySQL: Full image row level bin logging required

• SQL Server: Recovery model bulk logged or full

• Postgres: wal_level = logical; max_replication_slots >= 1; max_wal_Senders >=1;

wal_sender_timeout = 0

Changes captured and applied as units of single committed transactions

Activated when load starts

No changes are applied until load completes, then applied as soon as possible in near real-time

Replication

Instance

Source Target

Start Full Load

While Loading Data Also Capture Changes

Source TargetReplication Instance

Update

Load Complete - Apply Captured Changes

Source TargetReplication Instance

Update

Changes Reach Steady State

Source TargetReplication Instance

Update

Cutover - Shut Down Apps & Apply Remaining Changes

Source TargetReplication Instance

Update

Flip!

Source TargetReplication Instance

Update

Changes are Transactional and Come From the Logs

Migration Server

Source Target

Update

t

1

t

2

t

1

t

2

Replication

Instance

Source Target

Multiple Targets

Target

Target

Migration

Server

Source Target

Multiple Sources

Source

Source

Migration

Server

Source

Target

Multiple Sources and Targets

Source

Source

Target

You Don’t Have to Take Everything

Source

Target

Replication Instance

Homogenous or Heterogeneous

Replication

Instance

SQL Server MySQL

Replication

Instance

Oracle Oracle

Replication

Instance

Oracle Aurora

Lab

Lab Document

Download: http://chilp.it/499290a

Lab to be run in us-west-2 (Oregon) region

Lab

Thank you!

Remember to complete

your evaluations!