Overview of EnterpriseDB Postgres Plus Advanced Server 9.4 and Postgres Enterprise Manager 5.0

© 2015 EnterpriseDB Corporation. All rights reserved. 1

Introducing Postgres Plus Advanced Server 9.4 & Postgres Enterprise Manager 5.0


•  EnterpriseDB Overview •  Postgres Plus Advanced Server 9.4

−  Resource Management −  Partitioning −  SQL/Protect & MTK Enhancements

•  Highlights of What’s New in PEM 5.0 −  Enhanced Alerting −  Log Analysis Expert −  Custom Probes −  Auto-Discovery of Managed Servers −  Remote Monitoring

Agenda


POSTGRES innovation

ENTERPRISE reliability

24/7 support

Services & training

Enterprise-class features, tools &

compatibility

Indemnification

Product road-map

Control

Thousands of developers

Fast development

cycles

Low cost

No vendor lock-in

Advanced features

Enabling commercial adoption of Postgres


EDB is a Gartner Magic Quadrant Leader





Agenda


from PostgreSQL core from EDB Development

•  64 bit LOBs up to 4TB in size

•  Custom background workers

•  Writable Foreign Data Wrappers

v9.1

EDB contributions to PostgreSQL core

• No restore In-place version upgrades

v9.2

v9.3

v9.0

• Materialized Views

•  Deferrable unique constraints and Exclusion constraints

•  Streaming replication

•  Windows 64 bit Support

•  Hot standby

•  Synchronous replication

•  Serializable Snapshot Isolation

•  In-memory (unlogged) tables

•  Writeable Common Table Expressions (WITH)

•  Cascaded streaming replication

•  JSON support, Range Types

•  VARRAY support •  SQL Profiler

•  Index Advisor •  Parallel Bulk Data

Load

•  Row Level Security •  Declarative Partitioning syntax

•  Table() function support for nested tables

•  INSERT APPEND hint

•  xDB Multi-master replication

•  Expanded Object Type support

•  Partition Read Improvements over 75x

•  Support for 1000s of Partitions

•  Partition write improvements over 400x

• MySQL Foreign Data Wrappers for SQL/MED

Postgres Plus Advanced Server Key Feature Development

• Index-only scans (covering indexes)

• Linear read scalability to 64 cores

v9.4 • pg_prewarm • ALTER SYSTEM • Concurrently updatable Materialized Views

• Mongo FDW & MySQL FDW

•  Logical Decoding for Scalability

•  JSONB Data Type

•  JSONB

Indexing •  Expanded

JSON functions •  Delayed

Application of Replication

•  3x Faster GIN indexes

•  Support for Linux Huge Pages

•  CPU & I/O Resource Management

•  SQL Aggregation with CUBE, ROLLUP and GROUPING SETS

•  Comprehensive UTL_HTTP Package

•  Hash Partitioned Tables

•  Connect_By_Root Operator for hierarchical

queries •  SQL/Protect

Logging to DB Table

•  EDB*Loader Improved Error handling


CPU & I/O Resource Management

Hash Partitioned Tables

SQL Aggregation with CUBE, ROLLUP and GROUPING SETS

Comprehensive UTL_HTTP Package

Connect_By_Root Operator

ICU Collation

EDB*Loader Improvements

SQL/Protect Logging to DB Table

Migration Toolkit Enhancements

Postgres Plus Advanced Server Postgres Community

Feature Highlights for Postgres Plus Advanced Server (PPAS) 9.4

Many new features including: Logical Change Set Extraction JSONB Data Type Time Delayed Standby ALTER SYSTEM pg_prewarm() Materialized View Refresh Concurrently Ordered Set Aggregates and more… http://www.depesz.com/ is a good reference site https://commitfest.postgresql.org/ is the global community site for patches review


Postgres Plus Advanced

Server

Resource Manager

(CPU & I/O)

Reporting

Transactions

80%

20%

Run Mixed Workloads More Efficiently with PPAS 9.4 Resource Management

•  DBA assigns CPU & I/O to job groups

•  Allocates and prioritizes consumption of resources

•  Low priority jobs don’t hurt high priority jobs


•  Create Resource Groups and assign the CPU Rate Limit

•  Use these resource groups during a psql session

Run Mixed Workloads More Efficiently with PPAS 9.4 Resource Management

- Statements executed will be limited in CPU or writing to shared_buffers per resource group. - Individual limits are computed based on recent CPU / shared_buffer usage. - Processes sleep when necessary & avoid sleep when holding critical locks. - The limits are adjusted regularly based on current usage. - If multiple processes in the same group are being executed, aggregate usage will be limited

CREATE RESOURCE GROUP resgrp_a; CREATE RESOURCE GROUP resgrp_b; ALTER RESOURCE GROUP resgrp_a SET cpu_rate_limit = .25; ALTER RESOURCE GROUP resgrp_a SET dirty_rate_limit = 12288;

SET edb_resource_group TO res_grp_a;


Chapter 13 of EDB Oracle Compatibility Guide for details on usage

One logically large table is broken into smaller physical pieces. •  Worthwhile when a table would otherwise be very large. •  Exact point to see benefits will depend on the application.

When should I Partition? •  Good rule of thumb is that the size of the table should exceed the physical memory

of the database server. •  When most accessed rows are in a single partition or a small number of partitions. •  When there is a lot of concurrent inserts or updates (Hash partitioning may benefit) •  When a query or update accesses a large percentage of a single partition. •  For Bulk Loads / Unloads (ALTER TABLE faster than bulk load, avoids VACUUM

overhead from DELETE). •  When actively archiving seldom-used data to less expensive storage.

Support Larger Tables with Partitioning


Use PPAS Syntax To Minimize Partitioning Errors and Complexity

CREATE TABLE sales ( dept_no number, part_no varchar2, country varchar2(20), date date, amount number ) PARTITION BY RANGE(date) ( PARTITION q1_2014 VALUES LESS THAN('2014-Apr-01'), PARTITION q2_2014 VALUES LESS THAN('2014-Jul-01'), PARTITION q3_2014 VALUES LESS THAN('2014-Oct-01'), PARTITION q4_2014 VALUES LESS THAN('2015-Jan-01') ); CREATE INDEX sales_date on sales(date);

Simple Declarative PARTITION BY and SUBPARTITION BY

Single Index command

•  More SQL syntax provide sophisticated data management techniques −  ADD / DROP to augment the partitions −  SPLIT to divide the data −  EXCHANGE to swap in a new partition −  TRUNCATE to remove all data but leave

structure in tact −  MOVE to shift data to a different tablespace

•  PPAS Improved performance with Fast Pruning and Constraint Exclusion support

•  System Catalog Views for Partitions −  ALL_PART_TABLES −  ALL_TAB_PARTITIONS,

ALL_TAB_SUBPARTITIONS −  ALL_PART_KEY_COLUMNS,

ALL_SUBPART_KEY_COLUMNS


List, Range or Hash Partition Rules •  Provide the constraints to define where data is stored •  For PPAS, also used to support Fast Partition Pruning •  Consider how data stored will be queried, include often-queried columns

in partitioning rules.

•  List – Single partitioning key column; based on exact value •  Range – One of more partitioning key columns; based on values between

two extremes •  Hash (New 9.4) – Data divided amongst equal sized partitions based on

hash value. * Internal tests have shown hash partitioning can improve performance when many hundred concurrent connections insert/update to the same table*

PPAS 9.4 Supports Various Partitioning Rules


Advanced Server’s Query Planner uses two optimization techniques to compute an efficient plan: •  Constraint exclusion (constraint_exclusion = partition or on)

−  Provided by PSQL −  SELECT w/ WHERE: Query Planner must examine the CHECK constraints defined

for each partition before deciding which partition to send query fragments to. •  Fast pruning (edb_partition_pruning = on)

−  Occurs earlier in query planner process. Understands the relationship between partitions in an Oracle style partitioned table.

−  SELECT w/ WHERE: Query Planner can reason that only a certain partition holds the values without examining the constraints defined for each partition.

−  Can be used for LIST or single value RANGE Partitions (not usable on subpartitioned tables or multi-value range partitioned tables)

−  Works with >, >=, =, <=, <, AND, BETWEEN operators in the WHERE Clause

PPAS Improves Partitioning Performance


•  SQL/Protect Logging to DB Tables (Ch 4.1.1.2.3 of EDB Enterprise Edition Guide) −  Gives DBA a tool to protect against SQL Injection attacks −  View edb_sql_protect_queries contains logged information −  To test, enable SQL/Protect, create a test role, set blocking of unbounded

DML (for example), then execute UPDATE without WHERE clause on test table

•  MTK Improvements −  Migration Toolkit will now provide a detailed log with well defined error

codes to allow DBAs to better understand which capabilities of their database applications from Oracle, MySQL, SQL Server or Sybase are migrate-able to PPAS.

−  See Ch 9 of Migration Toolkit documentation for a list of the error codes provided

More Flexibility with SQL/Protect and Migration Toolkit Enhancements


1.  Customers running mixed workloads.

2.  Application Developers who require communication with external Web servers.

3.  Customers with large tables where they often search for exact matches or have many concurrent inserts/updates.

4.  Users who think they need a NoSQL database.

5.  Customers with reporting or data warehousing databases.

6.  DBAs who use need to bulk load data.

7.  DBA’s concerned with Security and SQL Injection Attacks.

Postgres Plus Advanced Server (PPAS) 9.4 Use Cases





Agenda


Over 225 predefined and custom probes to support alerts via SMTP or SNMP

Predefined & customized at-a-glance dashboards

OS and database statistics collection

Replication monitoring

MONITOR TUNE MANAGE

CRUD operations on all database objects

Bulk operations across multiple servers

Capacity Manager to plan & forecast

Log Manager and Audit Manager to configure database metric collection

SQL/Profiler to speed up large workloads

Index Advisor to suggest and create indexes

Postgres Expert for best practice enforcement

Tuning Wizard for machine utilization and load profiles

No Other Tool Provides Much Visibility Into your Postgres Databases


Monitor All Your Postgres Databases From One Screen

•  Customized global dashboard

•  View up/down status of all agents

•  Monitor alerts from many servers in one place

•  Navigate to Dashboards for further analysis


Highlights for PEM 5.0

•  Alert Level Specific Notifications

•  Log Analysis Expert

•  Custom Probes

•  Auto-Discovery of Managed Servers

•  Remote Monitoring

•  Download Tuning Recommendations

•  Log OS Metrics for Backend Server Processes

•  Better Zooming and Granularity Control

•  Cross Hierarchy Charts

•  Monitor Streaming Replication

Ease Of Use Richer Dashboards


Model your Support Staff Rotation in PEM for Email Alerting

•  Decide who is externally notified of alerts by creating the email groups.

•  Define different email envelope information, depending on time of day.


Automate Notification or Actions on over 225 Predefined or Customizable Alerts (1 of 2)

•  Create and manage alerts

•  Examples – running low on disk space, server down, last vacuum, total table bloat, etc.

•  Set the low, medium and high threshold


Automate Notification or Actions on over 225 Predefined or Customizable Alerts (2 of 2)

•  Choose when and who to send email notifications to.

•  Decide when to send SNMP traps

•  Further execute any external script.


•  Generate reports on stats like locks and queries that are collected by the Log Manager

•  Collects information based on historical csv data, not dependent on live logs.

•  First configure Log Manager to enable collection of log files.

•  Once you have logs collected, use the Log Analysis Expert to select which reports to generate

•  More details in the ‘Log Analysis Expert’ section of the Online Help

Better Understand Database Activities with Log Analysis Expert


Gather Any Custom Information You Want With Custom Probes (1 of 3)

•  Probes are used to gather metrics for alerts and dashboard charts.

•  View all System Probes.

•  Add New or Modify existing probes to suit your needs.



•  Define the columns that will be used to store the data collected.

•  If ‘Graphable’, the column will be available on Charts and Capacity Manager.

•  Control if the metric is Point In Time or Cumulative



•  If this is a SQL Probe, enter the SQL SELECT statement to be executed by the probe in the Code tab.

•  If this is a Batch probe, enter the shell or .bat script to be invoked by the probe.


•  Locate databases that reside on servers with Agents installed on them.

•  After you install agents −  Select them in the tree and

choose Management -> Auto Discovery to open the dialog shown.

−  Then select the database server to have the server property fields filled in

•  More details in the ‘Automatic Discovery of Server’ section of the Online Help

Auto Discovery of Managed Servers For Easier Deployment


•  Allow remote monitoring of servers (without installing agents using direct JDBC connections), from the agent running on the PEM server −  Ignores OS level stats,

disables features like Server Startup, Audit/Log/Capacity Manager, Tuning/Deployment Wizard

•  Create Server and select Remote monitoring in PEM Agent Tab.

Remote Monitoring


Feature Benefit Audience Motivation

Alert Controls Control who gets which alerts depending on severity and time of day / day of week.

DBAs Ease of Use

Log Analysis Expert Better understanding the operations that occur in your database.

DBAs Ease of Use

Custom Probes Get whatever custom information you want from your databases or servers; pull in BART or EFM status information

DBAs Compatibility

Auto-discovery of Managed Servers

Easily configure PEM Server with all database servers on managed hosts.

DBAs Ease of Use

Remote Monitoring Monitor servers without installing agents using direct connections.

DBAs Ease of Use

Log OS Metrics for Backend Server Processes

New probe that capture memory and CPU alongside process information

DBAs Compatibility

Improved Dashboards (Better zooming / granularity control, cross server charts)

Compare metrics, More accurate and relevant information when zooming in and out of charts

DBAs Ease of Use

Recap: Main Feature/Benefits for this Release


How can I learn more?

•  Download:

•  http://www.enterprisedb.com/download-advanced-server •  http://www.enterprisedb.com/download-postgres-enterprise-

manager

•  General Information: •  http://www.enterprisedb.com/postgres-plus-advanced-server •  http://www.enterprisedb.com/postgres-enterprise-manager

Overview of EnterpriseDB Postgres Plus Advanced Server 9.4 and Postgres Enterprise Manager 5.0

Software

Transcript of Overview of EnterpriseDB Postgres Plus Advanced Server 9.4 and Postgres Enterprise Manager 5.0