Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle

A Technical introduction to PostgreSQL and PPASEnterprise Class PostgreSQL Database from EDB

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Introduction

• About Ashnik

• PostgreSQL and buzz it has created

• Postgres Plus Advanced Server• Architecture• Oracle Compatibility• Performance Feature• Security Features• High Availability Features• DBA Tools

• User Stories

• What’s coming up in v9.3

• How to start adopting

Agenda

2

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Independent & Thriving Development Community for over 20 years

• Thousands of active deployments worldwide in public and private sector organizations of all sizes

PostgreSQL and the Community

3

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Postgres’ IMPACT

4

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

PostgreSQL – Postgres Plus Users, Globally

5

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

6

Malaysia

Philippines

Singapore

Vietnam Thailand

Indonesia

PostgreSQL – Postgres Plus Users, across ASEAN

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

EnterpriseDB – The Company

7

• The Enterprise PostgreSQL company was founded in 2004,

first product GA in 2005

• 2,000+ customers across all market segments

• 70,000+ downloads/week of PostgreSQL and related products

• Enabling database consolidation using PostgreSQL and

advanced Oracle compatibility

• Saving customers millions through the power of open source

• Strong financial backing:

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Postgres Plus - Recognized

by Gartner’s Magic

Quadrant as a challenger

in the Database Market

The EnterpriseDB Advantage

8

• Products and Tools

• Advanced database server software

• Deep Oracle compatibility

• Bundled development and management tools

• Technical Support and Services

• Around the clock support

• Expert consulting

• Oracle migration services

• Remote management and monitoring

• Professional Training

• Learn PostgreSQL from the experts

• Web and on-site training

• Training for developers and DBAs CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Fast development cycles

• Thousands of developers

• Better code

• Lower cost

• 24/7 support

• Services and training

• Certification

• Indemnification

• Product strategy

Bringing the Advantage to Enterprises

9

Open Source Software

CommercialSoftwareEnterpriseDB

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

About PPAS

10

PostgreSQL

xDBREPLICATION

Server

HIGH AVAILABILITY

and Scalability

PERFORMANCE and

Manageability SECURITY

and Cloud

Database MIGRATION

Toolkit

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Architectural Overview

11

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Process Architecture

Postmaster

BGWRITERBGWRITER STATS COLLECTOR

STATS COLLECTOR

ARCHIVERARCHIVERAUTO--VACUUMAUTO--VACUUM

BGWRITERBGWRITER STATS COLLECTOR

STATS COLLECTOR

Data Files

WALSegments

ArchivedWAL

Shared Memory

Shared Buffers Process ArrayWAL Buffers

WAL WriterWAL Writer LOG WRITERLOG WRITER

Error Log Files12

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Database Limitations

13

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Limitations are generally defined by

• Operating System Limits

• Compile-Time Parameters

– Data Type Usage

General Database Limitations

Limit Value

Maximum Database Size Unlimited

Maximum Table Size 32 TB

Maximum Row Size 1.6 TB

Maximum Field Size 1 GB (4TB in v9.3 Beta)

Maximum Rows per Table Unlimited

Maximum Columns per Table 250-1600 (Depending on Column types)

Maximum Indexes per Table Unlimited

Oracle Compatibility

14

• Run applications written for Oracle virtually unchanged

• No need to re-train Oracle DBAs and developers

• Support for PL/SQL language and OCI interoperability

• Replication for easy sharing of data

• Dramatic Cost Savings

• No Vendor Lock-in

• Same technology which is used by IBM in DB2 since v9.7 C

ON

FID

ENTI

AL

© 2

01

1 E

nte

rpri

seD

B. A

ll ri

ghts

res

erv

ed.

Compatibility Means

15

• SQL Extension support• Decode, NVL, Substr, NVL2,replace,translate, table()

• Date/time functions: add_months, extract, next_day,months_between,trunc, current_date support

• Support for cascade option in drop table

• PL/SQL support• REF Cursors, Implicit and explicit cursors

• Looping, variable declarations, conditional statements

• Collections: Associative Arrays, Varrays, Nested tables

• Bulk binding

• Named parameters

• User Defined Exceptions

• Explicit Transaction Control - within a stored procedure

• Tools• EDB*Plus – SQL*Plus look-a-like

• EDB*Loader – SQL*Loader equivalent

• EDB*Wrap – similar to the PL/SQL wrapper CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Compatibility Means (cont.)

16

• Features• Packages

• Stored procedures

• Functions

• Triggers

• Optimizer Hints

• Database Links

• Hierarchical Queries

• Synonyms – Public and Private

• Sequences

• Rownum

• Object types o Create type … as object

o Create type … as table

o Create type …as varray

o Constructor and collection methods

• PL/SQL like SUBTYPE, which inherits form base type

• Users/Roles

• Dynamic SQL

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Compatibility Means (cont.)

17

• Data Types

• Integer, number, char, double precision, float, varchar2, blob, clob, xmltype, rowid, boolean

• Built-in Packages• DBMS_:

o SQL, LOB, JOB, PIPE, ALERT, OUTPUT, UTILITY, PROFILER, RLS

• UTL_:o FILE, MAIL, SMTP, ENCODE, TCP

• Oracle-like Data Dictionary• ALL_, DBA_, USER_ views

• Most commonly accessed views

• Diagnostics - DRITA• System and session waits

o Not exposed in PostgreSQL

o Part of Advanced Server

• Statspack-like reporting

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Flexible Partitioning Scheme in PostgreSQL

18

• Partitioning Example in PostgreSQLo CREATE TABLE employees

o Create Child Tables

o create table emp_mgmt (check (job in (MANAGER'))) inherits (employees);

o create table emp_sales (check (job in ('SALESMAN'))) inherits (employees);

o Create the partitioning function

o CREATE OR REPLACE FUNCTION partition_emp();

o Create the trigger for partition

o CREATE TRIGGER employees_partition_trigger BEFORE INSERT ON employees

FOR EACH ROW EXECUTE PROCEDURE partition_emp();

• Advantages• Flexible

• Customize your Partitioning Policy

• Disadvantages• Complex

• Difficult to add/delete/split Partitions

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Partitioning Example in PPASCREATE TABLE employees

(empno numeric(4,0),ename varchar(10),job varchar(9),hiredate timestamp)

PARTITION BY LIST (job)

(PARTITION emp_mgmt VALUES ('MANAGER', 'PRESIDENT') tablespace tbsp_1, PARTITION emp_sales VALUES ('SALESMAN') tablespace tbsp_2, PARTITION emp_ops VALUES ('CLERK') tablespace tbsp_3

);

• PPAS Offers• LIST and RANGE Partitioning Syntax

• Easy to manage and maintain partitions

• You can still take benefit of PostgreSQL syntax for complex partitioning e.g. Dynamic Partitions

• Easy to Add/Delete/Split/Swap/Detach/Attach partitions

• Brings the best of both the world!

Compatible and Flexible Partitioning Scheme in PPAS

19

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Online Migration Toolkit enables point and click migration from Oracle

• Automatically Migrates:

Database Migration Toolkit

20

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Data

• Schemas

• Stored Procedures

• Triggers

• Functions

• Sequences

• Packages

• Views

• Database Links

• Synonyms

• Developed for High Performance Transaction Environments (OLTP)

• DynaTune: • Dynamic tuning of the database server to make the

optimal usage of the system resources available on the host machine

• Index Advisor: • Helps determine which columns you should index

to improve performance in a given workload.

• Query Optimization Hints, Hi-Speed Bulk Loader, Multi-Threaded Replication

Performance enhancement in PPAS

21

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Infinite Cache

• High performance horizontal scaling architecture for cache memory

• Cache expands with inexpensive commodity hardware

Scalability with Infinite Cache

22

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.


23

• Architecture: How it Works

• Disk access for data is slower than RAM access

• Infinite Cache puts disk data into the RAM cache

• Additional requests for that data are read from the faster cache

• The cache scales as needed with inexpensive hardware

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Ap

plicatio

n

High speed

pg_cache buffers in RAM

Slow speed disk

High Speed

Network

Updated Data Blocks

Update

Read

If not in buffer

Database Server

Infin

iteC

ache

RA

M b

lade

s

Highspeed


24

• Single Machine Performance

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Advanced Server is 16X faster on a single machine with large amounts of memory (e.g. greater than 2 GB)

Infinite Cache can be usedon a single machine!


25

• Infinite Cache - Features• Expands and Distributes buffer cache across multiple machines

• Designed for Read-Mostly applications (e.g. Content Management, Query Intensive, Business Intelligence, Data Warehouse)

• Cache is transparent to client applications (no cache coding needed)

• Compression feature enables caching entire databases (e.g. put a 250 GB database into 32 GB RAM Cache)

• Cache can be pre-warmed for immediate results

• Cache scales using inexpensive commodity hardware

• Infinite Cache can be run to boost single machine performance!

• Created For• DBAs and data managers overseeing large amounts of data requiring fast response times for queries and

reporting loads

• Developers who don’t want to write specialized caching code

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Red Hat Reference Architecture Series:

• Comparing BenchmarkSQL Performance

on Red Hat Enterprise Linux 5 to Windows

Server Enterprise (2010)

Performance Benchmark to SQL Server

26

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Scalability MySQL Vs PostgreSQL

27

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Security

28

• Object level privileges assigned to roles and users

• Virtual Private Database

• Kerberos and LDAP authentication

• Host base authentication

• SSL communication

• Data Level Encryption (AES, 3DES, etc)

• Ability to utilize 3rd party Key Stores in a full PKI Infrastructure

• Foundation for full compliance with the strictest of security standards (PCI Data Security Standard)

• Flexible field level encryption and row level security

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

“By default, PostgreSQL is probably the most

security-aware database available ...”

Database Hacker's Handbook- David Litchfield

PL/Secure and EDB*Wrap

29

• Encrypt your Pl/pgsql stored programs with PL/Secure

• Encrypt your SPL procedures with EDB*Wrap

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

SQL/Protect

30

• DBA Managed SQL Injection Protection • Preventing attacks is normally the responsibility of the application developer, but with SQL/Protect, DBAs can

now provide another layer of protection to prevent corruption or co-opting of the database.

• Multiple Prevention Techniques• Unauthorized Relations

• Utility Commands (e.g. DDL)

• SQL Tautology

• Unbounded DML

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Multi-Master Replication – Active-Active Technology

• Near real-time Replication

• Multi-Version Concurrency Control (MVCC)

• Point-in-Time Recovery

• Log Shipping for Standby (~ Oracle® Data Guard)

High Availability

31

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Availability

Backup and Recovery Options

32

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Physical and Logical Backup

• Logical Backup using pg_dump

• Instance level logical backup using pg_dumpall

• Table level, Schema level or DB level logical backup

• pg_restore

• Physical Backups

• pg_basebackup

• Compressed backup

• Recovery with WAL and archived WAL

• Point in Time Recover

• Using OS Clustering

• Using a shared disk for data

Active – Passive OS HA Clustering

33

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Uses WAL (Write Ahead Log)

• WAL is continuously shipped through an opened TCP channel to Hot Standby Node

• Hot Standby node can be used for read scalability

Hot Streaming Replication (Hot Standby Mode)

34

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rve

d.

• Uses WAL (Write Ahead Log)

• Archived WAL is shipped to Hot Standby Node


Log Shipping Replication (Hot Standby Mode)

35

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rve

d.

• Using pg-pool

• Using redundant disk for data


HA with read scaling (with pg-pool)

36

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rve

d.

• Trigger based replication

• Publication – Subscription

• Snapshot and Continuous

• Cascaded Replication

• Read Scalability

• Master/Slave DB can be:

• Oracle

• MSSQL

• Postgres Plus Advanced Server

• Either Master or Slave should be PPAS/PostgreSQL

xDB Single Master Replication (SMR)

37

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• High Availability uses

• Geographic distribution of load

• For creation of Testing/staging env using snapshot replication

• Segregate OLTP and reporting

xDB Replication Use Cases

38

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Replication

xDB Single-Master Replication (SMR)

39

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

TableD

TableC

TableB

TableA

TableC

TableD

ProcsObjects

ReportsQueries

Continuousor Scheduled----------------

Filteredor All Rows

TransactionReplication

ImprovedOLTP

Performance

InexpensiveQuery /

Reporting

Oracle Server Postgres Plus Advanced Server

• Trigger based replication

• 2 or more Masters can be Sync

• Auto Conflict Detection & Resolution

• Read & Write Scalability

xDB Multi Master Replication (MMR)

40

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Java-based Replication Server and Replication Console• Delta changes recorded in shadow tables via post Insert/Update/Delete triggers• Data Replication through JDBC channel

xDB MMR Architecture

41

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Remotely Manage, Monitor And Tune

• Management En-Mass Design

• Enterprise Performance Monitoring

• Proactive Alert Management

• Graphical Administration

• SQL Workload Profiling

• Simplified Capacity Planning

• Full SQL IDE

Postgres Enterprise Manager

43

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Performance Monitoring Dashboards

• Capacity Manager

• Postgres Expert

• Alert Management

• Browser based console and dashboard

• Audit Manager

• Team Support

• Distributed Architecture

• Convenient Access

PEM For DBAs: Centralized Tool

45

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rve

d.

Helps you apply patches and updates for all the products under PPAS:

Stackbuilder Plus

46

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rve

d.

Replicate below objects from MS SQL Server to

Postgres Plus Advanced Server:

• Schemas

• Tables

• Constraints

• Indexes

• Table data

• Views

Replicating & Migrating Microsoft SQL Server

47

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

• Split input files into multiple files

• Load single tables in parallel

• Magnitudes better load time performance

• Parallel load capabilities also in Migration Toolkit and Migration Studio

Parallel Data Loader

48

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

InMobi Delivers Mobile Ads with Postgres

49

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

Background

• One of the fastest growing mobile advertising networks

• Huge scalability and performance requirements, billions of ads per month

• Uses HP Proliant server

Mobile Ad Platform Project

• Postgres the best fit for a scalable open-source

database

• Community support not sufficient to meet business

requirements, EnterpriseDB a partner

• Initial database install supported organic growth

• 25 instances across 4 data centers with replication

• Database sizes upwards of 600 GB

• 200K txns/minute with 3 ms response time

Why Postgres

• Most scalable open-source database

• Lower cost than traditional databases

• Performance and reliability up to the task at internet scale, supporting the most extreme workloads

• Customizations by EnterpriseDB rapidly incorporated into the product ensuring costly customization maintenance is avoided

Sony Reduces TCO of database environment by 80%

50

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.Free Realms Project

• Lower TCO and improve licensing flexibility

• Leverage existing Oracle DBA and Developer talent

• Migrate key Oracle apps to Postgres

• Commercial-grade quality and reliability, including backup and recovery standards, to support mission-critical applications

• Scalable, high performance execution

Why Postgres

• Reduced TCO by 80%

• East of Migration – majority of Oracle apps could run unmodified on Postgres Plus

• Skills re-use – Oracle – trained staff could immediately work with Postgres Plus

• Exceptional technical support from EnterpriseDB

“Postgres Plus has proven to be a cost-effective database that can accommodate Free Realms’ massive growth.”

- Andy Sites, Free Realms Senior Producer, SOE

Background

• Free Realms massively multiplayer online game

• >9M registered players since April 2009

• IT Oracle scale-up was cost-prohibitive

Background

• Multi-terabyte Oracle DB and needed to reduce DB costs

• Running old, unsupported version of Oracle and had to pay to upgrade

The EPBS Project

• Engaged EnterpriseDB consulting services to migrate

• Unique migration process developed and implemented

Why Postgres

• 2.2 TeraByte system migrated and now running on Postgres Plus Advanced Server

• Current performance is comparable to Oracle

• Costs reduced significantly by migrating to EnterpriseDB solutions

Achieved Comparable Performance Replacing Oracle

51

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

ToT, Thailand plans to replace Oracle Exadata-TimesTen In-memory database with HP-EnterpriseDB stack

52

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rve

d.

Background

• Largest public sector company in telecom in Thailand

• Has very stringent response time SLA (20ms response time for user requests ~ 3-5 read queries)

• Current setup has Oracle Exadata and four TimeTen In-memory databases

Current Challenges in Oracle Stack

• Issues in sync of Exadata with In-memory databases

• Full refresh locks the tables

• Current system can go upto 1500 tps

• Dynamic Partitioning for huge table

• Current system load is 750 TPS with write operations

• 5000 user requests (16.5K read queries) per second with 4 servers

HP-EnterpriseDB Stack Proposed to be DR for Exadata• HP DL-980 (4 CPU * 10 cores, 256GB RAM)

• 1TB VMA Disks for faster access

• Simulation tests show 2800 write-TPS achieved

• Flexible Partitioning scheme in PPAS (add partition on the fly for

each new date, no manual intervention)

• Currently this server is being tested as DR for Exadata

• Replication using xDB near-relatime replication for 1000 tps

• Scalability achieved using cascaded replication and Infinite Cache

• Setup can handle 5500 user requests (18K queries per second)

with 1.5-2.5ms response time per query

v.9.3 Beta is out

• Switch master without archive logs in cascaded replication

• Faster recovery

• More extended JSON support

• Materialized Views

• Updatable Views

• Recursive Views

• Replication between Heterogeneous System Architectures

• pg_xlogdump to decode WAL

• 4TB Large Object Support

• Better memory managements

What’s Coming Up?

53

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

How to start adoption

54

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

REPLICATION SERVER

BACKUP / DR SERVER

NEW APPLICATION

MIGRATION

You have a ‘Real’ alternative to Oracle or other conventional proprietary Databases

Conclusion

55

CO

NFI

DEN

TIA

L©

20

11

En

terp

rise

DB

. All

righ

ts r

ese

rved

.

EconomicalTechnically

Sound

Easy to

Adopt

Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle

Technology

Transcript of Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle