Time Travelling With DB2 10 For zOS

The Information Management Specialists

Time Travelling with DB2 10 for z/OS

http://www.triton.co.uk/

http://66.102.9.104/search?q=cache:BJyJnmfPtkAJ:www.triton.co.uk/ManagedServices.php+remote+db2+support&hl=en&ct=clnk&cd=1&gl=uk


Topics

• Introduction

• DB2 10 for z/OS Overview

• Temporal Data Support

• Early Experiences

• Further Reading

• Summary




Introduction

• Director and Principal Consultant at Triton Consulting

• 24 years DB2 experience, 19 as a consultant working with customers in UK, Europe and the US

• IBM Gold Consultant since 1999

• IBM Information Champion

• Former IDUG (International DB2 User Group) President

• Author of IBM Redbooks, white papers and more recently “flashbooks”




Topics

• Introduction




• Further Reading

• Summary




DB2 10 for z/OS

• Extensive beta program running throughout 2009/10, with customers from all around the world

• Generally available since October 2010

• Upgrade path provided from DB2 8 or DB2 9

• Many customers are currently planning their DB2 10 upgrades, to begin in the next 12-24 months

DB2 8 for z/OS End of Support in April 2012




Top New Features

• CPU/Performance Improvements • Temporal Data • Virtual Storage Enhancements • High performance DBATs • Security Extensions • Improved Catalog Concurrency • Access Path Management • pureXML enhancements • Currently Committed semantics • Automated statistics • Dynamic schema change

enhancements

• In-memory object support • Optimiser enhancements • MEMBER CLUSTER for UTS • Backup and recovery

enhancements • Enhanced audit • Include additional index

columns • Enhanced SQL OLAP functions • Skip Migration

• And many more….




Topics

• Introduction




• Further Reading

• Summary




Why Temporal Data?

• Most IT systems need to keep historical as well as current information

• Industry regulations and competitive pressures are prompting IT managers to maintain even more data for longer periods of time

• Most warehousing / analytics applications require a historical perspective of data

• Implementation requires lots of effort by DBA and developer to design, test and implement Lots of “reinventing the wheel”




Why Temporal Data?

• DB2 10 provides this functionality as part of the database engine – the first major RDBMS vendor to do so Improve DBAs and developer productivity

Reduce errors in application code

Reduce time-to-market for new applications

Improve performance by driving function into the database engine

Improve application consistency

Support compliance / audit requirements




Why Temporal Data?

• Solve common business problems Ensure that a customer only has one financial position at a

given time

Was an insured covered for a procedure on a specific date? ► Was that information correct at the time the claim was processed?

Establish prices for a catalog ahead of time, so that they are completed before the change needs to be made

Answer a customer complaint about an old bill

… and many, many more




Temporal Data Concepts

• Temporal Table - a table that supports “data versioning” to allow point-in-time queries to be executed Multiple versions of each row are kept by DB2 as they

change over time Additional metadata is kept, recording the period in time

when a given version of the row was valid Contrast to traditional non-temporal (aka “snapshot”)

tables, where only the current version of a row is available unless developer/DBA have taken additional steps to support historical perspective





• Period – the time during which a given version of a row is/was valid Period is defined by special start timestamp and end timestamp

columns in the temporal table Note that in current DB2 implementation start timestamp is

inclusive (>=), but end timestamp is not inclusive (<) ► Feature request to provide ZPARM and make this a site decision

CustNo … Start_TS End_TS

123 … 2011-05-01 11:03 2011-05-26 23:51

123 … 2011-05-26 23:51 9999-12-31 23:59

Period #1

Period #2





• System Temporal Table – a temporal table that uses DB2 system-defined periods to record row history Associated with a DB2 system event (INSERT / UPDATE /

DELETE) against table – all changes captured DB2 automatically sets start and end timestamp for a

period History is maintained in a separate “history” table No gaps between periods – end timestamp of previous row

version = start timestamp of new version Useful for audit/compliance requirements Typically implemented by DBA





• Business Temporal Table – a temporal table that uses business-defined periods to record row history Aka “application temporal” Associated with a business event such as a change of address Useful for tracking business events over time History is maintained in the base table Application has control over start and end timestamp for a

given period ► Can have gaps between periods

Typically implemented by developer and DBA





• Bi-Temporal Table – a temporal table that supports both business and system time periods

Best of both worlds – application-defined versioning with complete audit trail of system changes

Two sets of start/end timestamps defined




Implementation Choices

• User-managed Only choice prior to DB2 10 Typically implemented using triggers to catch “before” version of row

during INSERT/UPDATE/DELETE ► Need application logic to determine if a given series already exists (update old

row + INSERT in same UOW) or not (INSERT only)

Lots of scope for inconsistency / errors in application code (e.g. overlapping periods)

► Need audit function to ensure rows are valid, SQL can be horrible

Lots of scope for performance issues – especially with index design

• DB2-managed Designed to avoid the pitfalls above Some potential performance gains – see later




Implementing DB2 Temporal Tables

• DBA indicates which tables need temporal support at CREATE/ALTER time Support for UTS, classic partitioned and single table segmented

tablespaces

• For system temporal or bi-temporal tables DBA also creates separate history table with identical definition to

base table (same columns, data types, null attributes, etc but no SYSTEM_TIME specification) and associates with base table via ALTER

DB2 automatically copies old version of row to history table whenever row in main table is changed via DELETE or UPDATE

DB2 automatically UNIONs base and history table as required




Select Extensions

• Elegant extensions to SELECT statement allow historical perspective to be seen via standard SQL Same syntax for SYSTEM_TIME and BUSINESS_TIME

SELECT * FROM BASE_TABLE FOR SYSTEM_TIME AS OF :TSTAMP WHERE CUST_NO = 123

SELECT * FROM BASE_TABLE FOR SYSTEM_TIME BETWEEN :ST_TSTAMP AND :EN_TSTAMP WHERE CUST_NO = 123

SELECT * FROM BASE_TABLE FOR SYSTEM_TIME FROM : ST_TSTAMP TO : EN_TSTAMP WHERE CUST_NO = 123

Multiple - Inclusive/Inclusive Multiple - Inclusive/Exclusive

Single - Inclusive/Exclusive




Implementing System Temporal

• SYSTEM_TIME row begin column TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN Composed of 9-digit timestamp, plus 3 digits for data sharing member DB2 ensures uniqueness of generated values across transactions Not updateable by application

• SYSTEM_TIME row end column TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END Not updateable by application

• Data types for both begin and end columns must be the same • INSERT, UPDATE, DELETE use standard syntax, all versioning handled by

DB2 • Query rewrite for SELECT statements to UNION ALL history table




System Temporal DDL CREATE TABLE CUSTOMER ( CUST_NO INTEGER NOT NULL, … SYS_START TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN, SYS_END TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END, SYS_CRT TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS TRANSACTION START ID, PERIOD SYSTEM_TIME(SYS_START, SYS_END) ); CREATE TABLE CUSTOMER_HIST ( CUST_NO INTEGER NOT NULL, … SYS_START TIMESTAMP(12) NOT NULL, SYS_END TIMESTAMP(12) NOT NULL ); ALTER TABLE CUSTOMER ADD VERSIONING USE HISTORY TABLE CUSTOMER_HIST;




Example – System Temporal System

Event

Time T1 T3 T2

EmpNo Salary Start_TS End_TS

300 10000 T0 31-12-9999

321 20000 T1 31-12-9999

Main

Table


300 10000 T0 31-12-9999

321 25000 T3 31-12-9999

INSERT INTO EMP

VALUES (321, 20000)

UPDATE EMP

WHERE EMPNO = 321

SET SALARY = 25000


321 20000 T1 T3

History Table SELECT SALARY

FROM EMP

FOR SYSTEM TIME AS OF T2

WHERE EMPNO = 321

= 20000

SELECT SALARY

FROM EMP

WHERE EMPNO = 321 = 25000




Implementing Business Temporal

• BUSINESS_TIME row begin and row end columns can be defined as either: TIMESTAMP(6) NOT NULL DATE NOT NULL

• BUSINESS_TIME periods have a check constraint in which end > begin column

• New optional unique BUSINESS_TIME WITHOUT OVERLAPS index is useful in having a built-in mechanism for enforcing keys being unique over a period of time Adds start and end columns to index

• UPDATE and DELETE statements have been enhanced to allow updating and deleting based upon a period of time FOR PORTION OF BUSINESS_TIME FROM expression1 TO expression2




FOR PORTION OF… UPDATE DELETE

Rows Contained

FOR PORTION OF

Business Time Row

Result

Span FROM or TO

FOR PORTION OF

Business Time Row

Result

Span FROM and TO

FOR PORTION OF

Business Time Row

Result




Business Temporal DDL CREATE TABLE CUSTOMER ( CUST_NO INTEGER NOT NULL, … BUS_START TIMESTAMP(6) NOT NULL GENERATED ALWAYS AS ROW BEGIN, BUS_END TIMESTAMP(6) NOT NULL GENERATED ALWAYS AS ROW END, PERIOD BUSINESS_TIME(BUS_START, BUS_END) ); CREATE UNIQUE INDEX CUST_IX ON CUSTOMER ( CUST_NO, BUSINESS_TIME WITHOUT OVERLAPS );




Topics

• Introduction



• Performance


• Further Reading

• Summary




Performance – System Temporal

• Redbook measurement • Transaction mix of 70% read

(SELECT), 30% write (10% INSERT + 20% UPDATE/DELETE)

• Compare: Baseline (no history) History with triggers History with system temporal

• Result: 56% less CPU overhead using system temporal when compared to triggers

Source: DB2 10 Performance Topics Redbook




Performance – System Temporal

• Redbook measurement • Compare, for varying number of

rows UPDATE/DELETE with triggers UPDATE/DELETE with system

temporal

• Result 30-39% less CPU overhead using

system temporal for UPDATE when compared to triggers

10-19% less CPU for DELETE





Performance – Business Temporal

• Redbook measurement

• Compare performance of UPDATE which results in two or three rows being split

UPDATE with business temporal

INSERT/UPDATE via stored procedure





Experiences – System Temporal

• System temporal is not a watertight audit strategy Can ALTER TABLE to remove history relationship if you have the necessary

authority (no record in SYSCOPY if versioning added or dropped)

• Schema change is painful (e.g. adding a column) Need to prevent access, drop versioning, add column to both tables, add

versioning again PM31313 addresses this – new function to support ALTER ADD COLUMN and

keep history table in step

• Need to integrate with archival strategy (if you have one…) • Need to treat base and history table as if they are referentially connected

– backup and recover as a set • Need to be able to supress history data from being replicated – open PMR

(PM31315)




Experiences – Business Temporal / General • Some applications with user-managed temporal functionality

can be ported relatively easily As long as DB2 implementation is consistent with your site/application

approach

DSNZPARM to set inclusive/inclusive or inclusive/exclusive is a common request

Benefits must be clear

• Non-temporal UPDATE/DELETE still allowed – care is required

• Limited 3rd party support for SQL extensions




Topics

• Introduction



• Performance


• Further Reading

• Summary




Further Reading

• IBM DB2 10 Home Page

http://www-01.ibm.com/software/data/db2/zos/db2-10/

• White Paper – DB2 10: A Smarter Database for a Smarter Planet

https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=sw-infomgt&S_PKG=wp-z-db2-smarter

Also available as part of a “flashbook” - ISBN: 1583473610

• DB2 10 for z/OS Performance Topics Redbook (SG24-7942)

• IDUG – International DB2 User Group

http://www.idug.org/






















Topics

• Introduction



• Further Reading

• Summary




Summary

• DB2 10 contains a long list of significant enhancements to improve performance, scalability and productivity

• Temporal support is an industry first, and offers some compelling advantages for new applications than need a historical data perspective Improve DBAs and developer productivity Reduce errors in application code Reduce time-to-market for new applications Improve performance by driving function into the database engine Improve application consistency Support compliance / audit requirements



Time Travelling With DB2 10 For zOS

Documents

Transcript of Time Travelling With DB2 10 For zOS