Amazon Web Services - Relational Database Service Meetup

Greg Khairallah | [email protected] Development Manager – Database Services

Relational Database Service

Agenda

• Overview of RDS• How to scale RDS• Cost optimization• Monitoring• Database logging• Q&A

Amazon RDS

Amazon DynamoDB

Amazon Redshift

Amazon ElastiCache

Compute Storage

AWS Global Infrastructure

Database

Application Services

Deployment & Administration

Networking

AWS DatabaseServices

Scalable High Performance Application Storage in the Cloud

Amazon RDS is a fully managed SQL database serviceAmazon RDS is a fully managed SQL database service

Simple to deploy and scaleSimple to deploy and scale

Without any operational burdenWithout any operational burden

Reliable and cost effectiveReliable and cost effective

Choice of Database enginesChoice of Database engines

If you host your databases on-premises

Power, HVAC, net

Rack & stack

Server maintenance

OS patches

DB s/w patches

Database backups

Scaling

High availability

DB s/w installs

OS installation

you

App optimization

If you host your databases in EC2

Power, HVAC, net

Rack & stack

Server maintenance

OS patches

DB s/w patches

Database backups

Scaling

High availability

DB s/w installs

OS installation

you

App optimization

If you host your databases in EC2

OS patches

DB s/w patches

Database backups

Scaling

High availability

DB s/w installs

you

App optimization

Power, HVAC, net

Rack & stack

Server maintenance

OS installation

If you choose a managed DB service like RDS

Power, HVAC, net

Rack & stack

Server maintenance

OS patches

DB s/w patches

Database backups

App optimization

High availability

DB s/w installs

OS installation

you

Scaling

Rapid deployment via Web Console

How RDS backups work

Automated backups• Restore your database to a point in time

• Enabled by default

• Choose a retention period, up to 35 days

Manual snapshots• Initiated by you

• Persist until you delete them

• Stored in Amazon Simple Storage Service (Amazon S3)

• Build a new database instance from a snapshot when needed

Choose cross-region snapshot copy for even greater

durability, ease of migration

• Copy a database snapshot to a different AWS Region

• Warm standby for disaster recovery

• Or use it as a base for migration to a different region

Choose Multi-AZ for greater availability, durability

• An availability zone is a physically distinct, independent infrastructure

• With Multi-AZ operation, your database is synchronously replicated to another AZ in the same AWS Region

• Failover occurs automatically in response to the most important failure scenarios

• Planned maintenance is applied first to backup

Amazon RDS Multi-AZ

Few clicks or One API call

High Availability Database Deployment

Do It Yourself MySQL on AWS

Potentially ~100+ Manual steps

Set up primary and standby instancesSet up identical volumesCreate synchronous replicationCreate and manage DNS entriesDetect instance failure conditionsDetect network failure conditionsDetect storage failure conditionsDecide when to fail-over….Re-establish primary secondary connections

Set up primary and standby instancesSet up identical volumesCreate synchronous replicationCreate and manage DNS entriesDetect instance failure conditionsDetect network failure conditionsDetect storage failure conditionsDecide when to fail-over….Re-establish primary secondary connections

Security in Amazon RDS

• Security is a shared responsibility

• AWS is responsible for physical security of cloud infrastructure

• You are responsible for configuring your instances securely

• RDS is compliant with PCI, SOC1/SOC2/SOC3, ITAR, ISO27001

• Use DB security groups to govern network access to database instances

• Use Amazon Virtual Private Cloud (Amazon VPC) to run DB instances in a distinct subset of the AWS cloud

• Private IP address ranges, subnets, routing tables and network gateways

• Use AWS Identity and Access Management (IAM) for fine-grained control of users and rights

• Use DB master users to govern in-DB access to data

How to scale with Amazon RDS

• Scale up or down with resizable instance types• CPU to 32 vCPUs, RAM to 244 GiB

• Scale your storage up with a few clicks• Scale while online • Easy conversion from standard to Provisioned IOPS storage

• Manage database parameters• Put a cache in front of Amazon RDS

• Amazon ElastiCache for Memcached or Redis• Or your favorite cache, self-managed in Amazon EC2

• Offload read traffic to MySQL Read Replicas

A resilient, durable, still simple application

architecture

RDS database instances:Master and Multi-AZ standby

Application, in Amazon EC2 instances

Elastic Load Balancer instance

DB snapshots in Amazon S3

Edmodo - A resilient, durable, still simple application architecture

How RDS billing works

Monthly bill = GB+ +

Assumes DB instance accessed only from EC2Further details at http://aws.amazon.com/rds/pricing/

= 720 hrs * $0.185 + 100 GB * $0.125 + 1,000 IOPS * $0.10

= $247.92

db.m3.large; MySQL; N. VA; Single-AZ

100 GB Provisioned IOPS

Provisioned 1,000 IOPS

Save money with RDS Reserved Instances

• Pay a low up-front fee to get a lower hourly price on database instances for a 1- or 3-year term

• The lower-price entitlement applies to any running instance matching its description

• Choose among Heavy, Medium, Light RIs

Instance Selection

• RDS offers a wide range of instance classes

• Instance types differ on CPU, RAM, network performance, EBS Optimized

• Base selection on• Performance

• Cost (will differ based on type and region)

• Region (not all instance types available in all regions)

• Engine (not all instance types available for all engines)

• Elastically scale instance size up/down

Instance Selection: Current Generation

Instance Selection: Previous Generation

Instance Selection

• Measure existing application workload

• Use native database engine tools – system tables, Oracle Enterprise Manager, Statspack, SQL Server Tuning Advisor

• Use CloudWatch metrics

• Test and scale accordingly

Instance Selection: Performance

• Use the right instance class

• Use EBS optimized instances

• dedicated capacity for Amazon EBS I/O

• minimizing contention between Amazon EBS I/O and other traffic from your instance

• db.m3.xlarge, db.m3.2xlarge, db.r3.[xlarge, 2xlarge, 4xlarge], db.m2.2xlarge, db.m2.4xlarge, db.m1.large, db.m1.xlarge

• 10 GiB networking: db.cr1.8xlarge, db.r3.8xlarge

Instance Selection: Performance

• Use Provisioned IOPS

• Provisioned IOPS storage is a storage option that delivers fast, predictable, and consistent throughput performance

• When fast and consistent I/O performance required

• Optimized for I/O intensive, online transaction processing (OLTP) workloads that have consistent performance requirements

• Important: Recommended for random reads and writes and not sequential workloads

Provisioned IOPS: Storage to IOPS

Activities that Impact Storage Performance

• Nightly backups

• There is no additional charge for backup storage up to 100% of your provisioned database storage for an active DB Instance

• Perform these tasks during your maintenance window

• DB snapshot creation

• Single-AZ to Multi-AZ modification

• Read replica creation

Caveats

• Allocated storage and provisioned storage can be modified

• Scaling Storage operations can impact performance until new allocation is finished

• Except for SQL Server

• Export data

• Create new DB instance with larger storage

• Import data

• Initially size SQL Server storage for future growth to avoid this

Database Parameter Groups

DB parameter groups act as a container for engine configuration values that are applied to one or more DB instances.

Choose MySQL Read Replicas for greater scalability

Help your app scale by offloading read traffic to an automatically maintained read replica

Create multiple read replicas, load-share traffic

Easy to set up

Native MySQL

RDS

Choose MySQL cross-region read replicas for enhanced data locality, even more ease of migration

• Even faster recovery in the event of disaster

• Bring data close to your customers

• Promote to a master for easy migration

CloudWatch

• Monitor AWS resources in real-time

• Collect and track metrics

• Metrics data provide visibility into performance, storage, number of database connections, etc.

• Send notifications based on the monitoring rules that you define

CloudWatch Alert Options

• RDS Metrics

• CPU Utilization - Track the percentage of CPU utilization

• Free Storage Space – Track available storage space

• Free Memory – Amount of Available RAM

• Database Connections – Number of databases connections in use


• RDS Metrics

• IOPS – Number of IO operations completed per second

• Latency – Elapsed time between the submission of an I/O and it’s submission.

• Throughput – Number of bytes per second transferred to or from disk

• Queue Depth – Number of I/O requests in the queue waiting to be serviced


• Queue Depth – Number of I/O requests in the queue waiting to be serviced

• Queue depth of 5 for every 1000 IOPS

• If less, not enough I/O generated

• If more, you may want to increase IOPS


Metric Actions

CPU Utilization Scale Compute

Freeable Space Scale Storage Up

Freeable Memory and Swap Usage Scale Compute

Write Latency Add Provisioned IOPS

DB Connections Check connection pooling

Cloudwatch Alert Options

• A number of options to get alerted – SMS, SQS, HTTPS, Email-JSON, Email………

Database Logs

• Monitor the MySQL error log, slow query log, and the general log directly through the Amazon RDS console, Amazon RDS API, Amazon RDS CLI, or AWS SDKs

• Log retention period is 24 hours

• Automatically rotated when:

• Disk space usage is greater than 90% of the allocated space, and a single log uses either more than 10% of the allocated storage or more than 5 GB

• A single log uses more than 20% of the allocated disk space or more than 10 GB, regardless of total disk usage.

• Use the mysqlbinlog utility to download or stream a binary log

Database Logs

• Access Oracle alert logs, audit files, and trace files by using the Amazon RDS console or APIs

• Configure log retention period per your needs (default 7 days)

• The Oracle database engine may rotate these logs if they get very large

Database Logs

• You can access SQL Server error logs, agent logs, and trace files by using the Amazon RDS console or APIs

• Configure log retention period per your needs (default 7 days)

• Log files are rotated each day and when a database reboot occurs

Database Logs

• You can access SQL Server error logs, agent logs, and trace files by using the Amazon RDS console or APIs

• Set the retention period for system logs using the rds.log_retention_period parameter in the DB parameter group associated with your DB instance

• Maximum retention period is 7 days

• Enabling log_min_duration can help you find unoptimized queries

Try Amazon RDS for free

For your first year, at no charge…• Enough free instance-hours to run a “micro” instance

continuously

• 20 GB of database instance storage

• 20 GB for automated backups

• 10 million I/O operations per month

Learn more about the AWS free tier:http://aws.amazon.com/free/

Learn more about Amazon RDS

Amazon RDS home page: http://aws.amazon.com/rds/

Amazon RDS Frequently Asked Questions:http://aws.amazon.com/rds/faqs/

Links to Import Guides for each engine:http://aws.amazon.com/rds/faqs/#9

Thank you

Amazon Web Services - Relational Database Service Meetup

Presentations & Public Speaking

Transcript of Amazon Web Services - Relational Database Service Meetup