SQL Life Cycle Management · 2019. 1. 15. · SQL life cycle management Operational System Test...

SQL – Life Cycle ManagementSQL Life Cycle Management

© 2011 SOFTWARE ENGINEERING GMBH and SEGUS Inc. 0

DB2 Innovations–Addressing Business Exposures

SYSTEM

DATABASE DB2 continues to be vital:APPLICATION

UnpredictablePerformance

High rate of change Growing data

volumes

BusinessUnits

Performance

Degraded

volumes New workloads

All creating new businessUnplanned Outages

DegradedServiceLevels

All creating new business exposures


SQL life cycle management

Operational System

Test APPL DB2 Tables& DB2 Catalogs

d bl

Recovery AssuranceExpert

Enterprise Prod APPL DB2 Tables& DB2 Catalogs

pStatistics

DistributionInstant

CloningExpert

Create Application SQL

QA APPL DB2 Tables& DB2 Catalogs

SQL PerformanceExpert

RealTime DBAExpertBind ImpactExpert

g pProduction

Bind ImpactExpert

Bind ImpactExpert

QA System


SQL Development life cycle / SPX - BIX

SQL SQL

Q/A

Detect performance i i i li

Dev

Development

Production

Preserve production issue prior going live

Duplicate Production Statistics on test environment (Production Si l t )

Interactive SQL development

Access path analysis based on production statistics

performance

Optimize maintenance for best performance and low risk by identifying impact of REBINDSimulator)

Highly automated quality insurance for changed application

on production statistics REBIND

Quickly find and tune bad SQL (static/dynamic)

Pre-Check impact of newapplication

Identify changed application SQL, changed access path

Pre Check impact of new DB2 version


Iterative SQL Life Cycle Management Phase

100-times

ts

15 timess an

d e

ffo

rt

Source: IBM Systems Science Institute

6,5X

15-times

ase

in c

ost

s

1X

Planing

Coding

(De elopment)

Integration- & Stresstest

(Quality Assurance)

Implementation & Produktiver Betrieb

(Produktion)

Incr

ea

g (Development) (Quality Assurance) (Produktion)



RuntimeData

Productioncopy production statistics

SQLFire-Fighting

Fire-Prevention

statistics

SQLl

QA for changed apps

SQL

Quality Assurance

Development

SQLInteractive SQL analysis


ESD – Key Features

Provide a production like framework for developers To simulate production environment behavior of

SQLSQL Interactively strengthen developers coding

To allow access path and workload comparison Proactively forecast effect on the prod Proactively forecast effect on the prod.

environment Support flexible schema naming conversions Ensure most efficient resource usage Ensure most efficient resource usage Generate statistics for non-existing schema Includes hardware virtualization for

CPU (model and # of CPs) CPU (model and # of CPs) Memory (BP, RID Pool, …)


ESD for static SQL – Concept

(Optimizer-)RUNSTATS

SELECT …FROM …WHERE RUNSTATSWHERE …

DB2 CatalogStatistics

BIND REBIND Real-timeStatistics

Optimizer Optimizer

ExecutableCode

Utility Decision(REORG, COPY etc)



Production Central PLAN_TABLE - EXPLAIN(YES) SYSIBM.SYSPACKAGE package statements Capture schema



Development Homogeneous System Copy (without user VSAM

cluster) orcluster) or Clone DDL Update DB2 Catalog with production statistics

gathered Populate the central PLAN_TABLE from production Bundle and offer all data as a baseline for easy

analysis

Allows interactive and iterative analysis Allows interactive and iterative analysis


ESD within SPX - BIX

production system development system

ESDCatalogStatistics

Catalog Statistics

Plan-t bl

Plan-t bl

ESDSPXBIX

table tableShadow

SYSPKG SYSPKGShadow

ESD


ESD for dynamic SQL – concept

(Optimizer-)RUNSTATS

SELECT …FROM …WHERE RUNSTATSWHERE …

DB2 CatalogStatistics

Mini-BIND

OptimizerOptimizer

DynamicStatement

Cache



Production Snapshot of Dynamic Statement Cache Explain of all captured statements to central Explain of all captured statements to central

PLAN_TABLE Capture schema



Development Homogeneous System Copy (without user VSAM

cluster) orcluster) or Clone DDL Update DB2 Catalog with production statistics Copy central PLAN TABLE from production Copy central PLAN_TABLE from production Copy snapshot of Dynamic Statement Cache Explain of all extracted statements to PLAN_TABLE Bundle and offer all data as a baseline for easy Bundle and offer all data as a baseline for easy

analysis

Allows interactive and iterative analysis Allows interactive and iterative analysis


ESD within SPX - BIX

production system development system

ESDCatalogStatistics

CatalogStatistics

Plan-table

Plan-table

Shadow

SPXBIXESD

ESD

Dynamic SQL

Dynamic SQL

Dynamic StmtCache

ESD ESD


HSC – Key Features

Exploits storage technology + fast rename technology Supported hardware

IBM ESS (Flashcopy) IBM ESS (Flashcopy) RVA (Snapshot) EMC² (Timefinder)

Easy “sandbox” set-up for test and development

Ensures data consistency (online & offline) Ensures data consistency (online & offline)

XML based “one-click” automation

Allows full scope of test szenarios


HSC highly efficient database cloning

Cloning Management

AllowsAllows Incremental/refresh

cloning Merge cloning for multiple Merge cloning for multiple

DB2 systems


SQL development life cycle / SPX - BIX

Development

Interative SQL development

Access path analysis based on production statistics Quality assurance / Integration

Top 10 approach Top 10 approach

Check changed application / SQL

Check changed access paths

Performance tests / comparison based on clone Production

Performance monitoringPerformance monitoring Change management for access paths



Development


SPX – Key Features

Extensible Rule System Edit Macro

Assembler/Fortran/C/Cobol/PL1/Spufi mode SQL execution with hostvariable support Predicate report (Stage/Filter-Factor)p g Global Explain Table Pool

DRDA Support for Single Point of Control DRDA Support for Single Point of Control

Integrated Catalog Statistics Browser Copy / Transfer Statistics Ad-Hoc change of statistics (What-If analysis)


SPX – SQL Sources

PackageTrace

Plan/DBRM

DSCSnap

SQLPerformance

ExpertDBRM

SourceCode

d

OnlineExits

Bind ImpactExpertSQL PerformanceExpert EditMacro

TableIndex

…

SQL PerformanceExpert EditMacro

QMF


SPX – Explain Table Pool

Explain

Explain Table Pool No need for User Plan

Tables when using a pool

Explain

pTables 1

ExplainTables 2

g pof Explain tables.

Explain results are copiedto a set of global temporarytables for further

Rule Explain Table

ExplainTables 3

Global

tables for furtherprocessing.

checkingp

preparationExplainTables 4

Explainl

GlobalExpl. Tb.

Explain Tables Used:Tables 5

….

PLAN_TABLE - Access pathDSN_STATEMNT_TABLE - Cost, Statement TypeDSN_PREDICAT_TABLE - PredicatesDSN_FILTER_TABLE - IndexabilityDSN_DETCOST_TABLE - Detailed cost information


SPX – Explain (SQL browser)Analyze for DB2 z/OS ----- Package Statements ------------- Statement 1 from 3 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, CAN(cel), SE(tup), BA(Batch Prepare All),

L(ocate) stmtnoL(ocate) stmtnoLine cmd: Z(oom), A(nalyze), D(ynamic Analyze), E(dit Statement),

B(atch Prepare), X(EXecute Statement)

Collection : IQA_COLLECTION_510Package : ADB2UCTSVersion : 2011-09-28-13.05.36.020867Bindtime : 2011-09-28-15.05.46.909180 PreComp : 2011-09-28-15.05.36.020867

StmtNo Millisec Servunits Statement Text--------- --------- --------- ---------------------------------------------582 4 13 DECLARE TB IX CO KE 6 CURSOR FOR SELECT TB582 4 13 DECLARE TB-IX-CO-KE-6 CURSOR FOR SELECT TB .675 4 13 DECLARE TB-IX-CO-KE-7 CURSOR FOR SELECT TB .768 0 0 DECLARE TB-IX-CO-KE-HIST CURSOR FOR SELECT TB--------- --------- --------- ---------------------------------------------


SPX – Explain (Access Path Overview)

Analyze for DB2 z/OS ----- Explain Data (1/5) ----------------- Entry 1 from 1 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, T(Explain Text), V(iolations), R(unstats), P(redicates),

S(tatement Te t) PR(int Reports) Z(oom) SAVE SHOWS(tatement Text), PR(int Reports), Z(oom), SAVExxx, SHOWxxxLine cmd: Z(oom), I(ndexes of table), S(hort catalog), T(able), X(IndeX)

DSN = NEWMAN.ADB2.IN Member = DELMEStmt = 1Milliseconds: 433 Service Units: 1696 Cost Category: BMilliseconds: 433 Service Units: 1696 Cost Category: B

QBNO QBTYPE CREATOR TABLE NAME ACCS MTCH IX METH PRNT TABL PRE MXOPLNO TABNO XCREATOR INDEX NAME TYPE COLS ON OD QBLK TYPE FTCH PSQ---- ------ -------- ------------------ ---- ---- --- ---- ---- ---- ---- ---

1 SELECT SYSIBM SYSTABLES R 0 N 0 0 T S 01 1

---- ------ -------- ------------------ ---- ---- --- ---- ---- ---- ---- ---


SPX – Statistics Browser

Analyze for DB2 z/OS ----- Index Overview --------------------- Index 1 from 3 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, CAN(cel), SE(tup), Z(oom), L(ocate) creatorLine cmd: C(olumns) D(atabase) K(PacKages) P(artitions) T(able) Z(oom)Line cmd: C(olumns), D(atabase), K(PacKages), P(artitions), T(able), Z(oom)

IX Creator IX Name Created by Created timestampTB Creator TB Name Database Statstime

Indexspace-------------------------------------------------------------------------SYSIBM DSNDTX01 SYSIBM 0001-01-01-00.00.00.000000 SYSIBM SYSTABLES DSNDB06 0001-01-01-00.00.00.000000

DSNDTX01 -------------------------------------------------------------------------SYSIBM DSNDTX02 SYSIBM 0001-01-01-00.00.00.000000 SYSIBM SYSTABLES DSNDB06 0001 01 01 00 00 00 000000SYSIBM SYSTABLES DSNDB06 0001-01-01-00.00.00.000000

DSNDTX02 -------------------------------------------------------------------------SYSIBM DSNDTX03 SYSIBM 2011-09-21-23.27.05.275288 SYSIBM SYSTABLES DSNDB06 0001-01-01-00.00.00.000000

DSNDTX03DSNDTX03 -------------------------------------------------------------------------


SPX – Rule violations and recommendationsAnalyze for DB2 z/OS ----- Violations -------------- LINE 00000001 COL 001 080 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, E(xplain Data), T(Explain Text), R(unstats), P(redicates),

S(tatement Text), PR(int Reports), SAVExxx, SHOWxxxS(tatement Text), PR(int Reports), SAVExxx, SHOWxxx

DSN = NEWMAN.ADB2.IN MEMBER = DELMESTMT = 1

------------------------------------------------------------------------------------ RULE-NO.: 9072 (WARNING) -----

Predicate is stage 2 (neither stage 1 nor indexable)). QBLOCKNO: 1, Access:STAGE2, Predicate: 5 BETWEEN SYSIBM.SYSTABLES.DBID AND SYSIBM.SYSTABLES.OBIDTry to rewrite the predicate as stage 1 or indexable or try to add another (stage 1 or indexable) predicate for this column(s) to the WHERE or ON clause.

RULE NO : 9201 (WARNING)----- RULE-NO.: 9201 (WARNING) -----A predicate like: '(EXPR) BETWEEN COL1 AND COL2' should be rewritten like:'(EXPR) >= COL1 AND (EXPR) <= COL2'.Then the predicates are INDEXABLE.

----- RULE-NO.: 9065 (WARNING) -----SELECT * can lead to unnecessary data transfer. QBLOCKNO(s) affected: 1.SELECT can lead to unnecessary data transfer. QBLOCKNO(s) affected: 1.Select only columns which are really used by your application.

----- RULE-NO.: 9070 (SEVERE-ERROR) -----Runstats check found critical rule violations.Please look into the runstats report.

----- RULE-NO.: 9099 (WARNING) -----


SPX – Predicate OverviewAnalyze for DB2 z/OS ----- Predicates (1/3) --------------- Predicate 1 from 1 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, E(xplain Data), T(Explain Text), V(iolations), R(unstats),

S(tatement Text), PR(int Reports), Z(oom), SAVExxx, SHOWxxxS(tatement Text), PR(int Reports), Z(oom), SAVExxx, SHOWxxxLine cmd: Z(oom)

DSN = NEWMAN.ADB2.IN Member = DELMEStmt = 1

Qblkno Predno Left_hand_side Right_hand_side Stage TYPE------ -------- ------------------ ------------------ --------- --------

1 1 VALUE EXPRESSION STAGE2 BETWEEN------ -------- ------------------ ------------------ --------- --------


SPX – Extensible Rule System

Rules based on Parsed SQL syntax Access Path Access Path DB2 Version specific Explain Tables (DSN_*) Catalog Statistics

Rule Programs

Rules build fro conditions Rules build from conditions

Statistics HealthCheck integrated

Easy to use ISPF online interface for user defined rules


SPX – Ruleset overviewAnalyze for DB2 z/OS --- Rule Sets --------------------------- Row 1 to 5 of 5 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, CAN(cel), L(ocate) rule-setLine cmd: S(elect), I(nsert), D(elete), U(pdate)Line cmd: S(elect), I(nsert), D(elete), U(pdate)

Rule-Set DB2-Rel. Type SE ---------------------------------0001 810 DML 0002 910 DML

S 0003 810 DML 8008 810 DDL Y 9008 910 DML Y 1008 110 DML Y ---------------------------------


SPX – Active Rules

Analyze for DB2 z/OS --- Rules -------------------------------- Row 19 from 45 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, CAN(cel), USED, ALL, L(ocate) rule-no.Line cmd: S(elect) I(nsert) D(elete) U(pdate) T(exts) O(n Off)Line cmd: S(elect), I(nsert), D(elete), U(pdate), T(exts), O(n,Off)

Rule Set: 0001 DB2 Rel.: 110 Type: DML Rules shown: USED

Rule-No. Severity Rule-Type SE y ypDescrition Text---------------------------------------------------------------------------1058 W Condition Y

* Access path reoptimization at run time. Access path shown may not be the on---------------------------------------------------------------------------1059 I Condition Y

* Non matching index scan. Probably because of a simple tablespace with more---------------------------------------------------------------------------1060 W Condition Y

* OPTIMIZE HINT is used. DB2 uses a predefined access path.---------------------------------------------------------------------------1061 W Condition Y

* DB2 has to materialize a result table---------------------------------------------------------------------------1063 W Program (ADB2R063,RT=1) Y

* Clustered index &&P4 probably not used because of bad clusterratio of &&P5%


p y

SPX – Active Conditions (Sample Rule 9044)Analyze for DB2 z/OS --- Conditions -------------------------- Row 21 from 364 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END, CAN(cel), USED, ALL, L(ocate) cond-no.Line cmd: Z(oom), I(nsert), D(elete), U(pdate), O(n, off)Line cmd: Z(oom), I(nsert), D(elete), U(pdate), O(n, off)

Rule-No: 1044 Severity: W Type: DML SE: Y Descrition: Non matching index scan, NLEAF > 32 and INDEXONLY = YConditions shown: USED

Cond-No. Cond-Type SEExplanation---------------------------------------------------------------------------1019 SQL Y

* PLAN_TABLE: MATCHCOLS = 0---------------------------------------------------------------------------1021 SQL Y

* PLAN_TABLE: ACCESSNAME NOT = " "---------------------------------------------------------------------------1055 SQL Y

* PLAN TABLE: INDEXONLY = Y PLAN_TABLE: INDEXONLY Y---------------------------------------------------------------------------1059 SQL Y

* SYSINDEXES: NLEAF > 32---------------------------------------------------------------------------1080 SQL Y


SPX – Dyn. Condition Source (Sample Rule 9019)

Analyze for DB2 z/OS ----- Zoom SQL Condition ------ LINE 00000001 COL 001 080 Command ===> Scroll ===> CSR

DB2: Q10APrimary cmd: END

Cond-Number: 1019Cond-Type : SQLExplanation: PLAN_TABLE: MATCHCOLS = 0-------------------------------------------------------------------------------SELECTSELECT

'1' FROM

&&IQA_PLAN_TABLE PLWHERE

PL.QBLOCKNO = &&QBLOCKNOAND

PL.PLANNO = &&PLANNOAND

PL.MIXOPSEQ = &&MIXOPSEQAND

PL MATCHCOLS = 0PL.MATCHCOLS = 0******************************** Bottom of Data ********************************


Dynamic SQL Management – Customer Example

Step 1

Capture SQL

Extract and IdentifyBaseline Extract and IdentifyBaseline

Tune / Fix



Step 1: Capture SQL

SQL PerformanceExpert for DB2

Capture SQL


DSC Capture: Online extraction Batch extraction

Threshold based SQL extract and SQL filtering

S t t t t bSort statements byCPU utilization, frequency, timestamps



Step 2: Extract and identify


Extract and Identify

Q p

Sort statements byCPU tili ti f ti t

Identify bad statements with dynamic explain

CPU utilization, frequency, timestamps

explain



Step 3: Tune and Fix

SQL PerformanceExpert for DB2SQL PerformanceExpert for DB2

Apply rules based system to identify SQL

Tune / Fix

Apply rules based system to identify SQL problems

Online Batch for mass analysis

Dynamically Explain SQL Statement



Step 4: Baseline

Bind ImpactExpert for DB2

Baseline


Stores plan table entry for dynamic SQL statement

Mimics analysis of static SQL

Enables comparison over time, speeds t t t l istatement analysis



Step n – Repeat as necessary

Capture SQL

Baseline analysisTune / Fix

Identify problems



Step 5: Identify problems


h li i

Identify problems

d pac pe o

Detect degraded access path

Match SQL statement across application version

Online Batch



Quality Assurance


BIX – QA Key Features

Pre-Screen application changes Matches all statements in a new DBRM to the

current DB2 catalog Compares access paths old/new

Highlights state ents that are less efficient Highlights statements that are less efficient than the original

Supports trigger packagesSupports trigger packages “Top 10” Lists Configurable batch reports with XML support Integration into Compile-Procedure possible


BIX – QA szenarios

ImpactExpert for DB2 z/OS ------ Main Menu ---------------------- Version 6.10 Command ===> ________ DB2: Q10A

Primary cmd: END, S(ettings), C(leanup), A(bout), O(ld interface), H(istory) Li d S( l t) I( f )Line cmd: S(elect), I(nfo)

Dyn Migr. ConvertScenario Base Recent Expl Rules Qual. DRDA----------------------- ------------ ------------ ---- ----- ----- ----REBIND Analysis Catalog - YES N - -Pre-BIND Local Catalog DBRM YES - - -Pre BIND Local Catalog DBRM YES Post-BIND Local History Catalog NO - - -Pre-BIND Prod-Baseline Export (*) DBRM YES - N NPost-BIND Prod-Baseline Export (*) Catalog NO - N NEarly Precheck Static Export (*) - YES Y N NEarly Precheck Dynamic Export (*) - YES Y N -DSC Protection Export (*) - YES - N -Dynamic SQL DynStmtCache - YES - - -Static and dynamic SQL Trace - YES - - -

Plan_table compare Plan_table 1 Plan_table 2DBRM reconstruct CatalogDBRM reconstruct Catalog

NOTE (*): Use export/import function to update BIX export tables.


BIX – Bind Control

Automatically finds BIND impacts of changed applications

New DBRM for a changed programOriginal DB2 changed programOriginal DB2

catalog

EXPLAIN BIXEXPLAIN BIX

LOG


LOG


ImpactExpert for DB2 z/OS ----- Job Overview (1/5) -------------- Job 1 from 2 Command ===> ________ Scroll ===> CSR Mode: PRE-BIND LOCAL DB2: Q10APrimary cmd: END, A(ll), N(ew), R(efresh), Z(oom), L(ocate) -Line cmd: P(rograms), D(elete), G(raphic), L(PLans), S(tatements), Z(oom)

Job name Step name Submitter Submit time PLTB owner Job status-------- --------- --------- ------------------- ---------- ----------

S BINDPACK CHKRBIND NEWMANN 2011-08-27-08 44 06 NEWMANN DONES BINDPACK CHKRBIND NEWMANN 2011 08 27 08.44.06 NEWMANN DONE_ BINDPACK CHKBIND SMITH 2011-09-02-09.55.43 IMPACT DONE

-------- --------- --------- ------------------- ---------- ----------



ImpactExpert for DB2 z/OS ----- Programs Summary ------------- Status 1 from 7 Command ===> ________ Scroll ===> CSR Mode: PRE-BIND LOCAL DB2: Q10APrimary cmd: END, R(efresh), Z(oom) Line cmd: S(elect)

COLLECTION : *______________________________________________________________ PACKAGE : *______________________________________________________________ PLAN : *______

Status Programs Statements---------------------------------------- ---------- ----------

S PROCESSED 2 7 _ - UNCHANGED 1 5

- WORSENED 1 1 __ - NEW 0 1 _ NON-DETERMINABLE 0 0 _ PLAN_TABLE ISSUES 0 0 _ ERROR 0 0

---------------------------------------- ---------- ----------



ImpactExpert for DB2 z/OS -- All Programs (1/6) ------------- Program 1 from 2 Command ===> ________ Scroll ===> CSR Mode: PRE-BIND LOCAL DB2: Q10APrimary cmd: END, S(how prepared), Z(oom), L(ocate) programsLine cmd: S(tatements), A(nalyze), B(IND prepare), Z(oom)

Imp Collection / Plan Program Job submit time Rel Rec EX--- ------------------ -------- ------------------- --- --- ---

_ EQ BINDDEMO BINDDEMO 2011-08-27-08.44.06 N/A YES S WRS BINDDEMO SQLZDEMO 2011 08 27 08 44 06 V9 NOS WRS BINDDEMO SQLZDEMO 2011-08-27-08.44.06 V9 NO

--- ------------------ -------- ------------------- --- --- ---



ImpactExpert for DB2 z/OS -- Pre-BIND Overview (1/4) ----------- Stmt 1 from 4 Command ===> ________ Scroll ===> CSR Mode: PRE-BIND LOCAL DB2: Q10APrimary cmd: END, SE(tup Analyze), Z(oom), L(ocate) stmtnoLi d S( l t) AO/AN(A l Old/N ) DO/DN(D i l Old/N )Line cmd: S(elect), AO/AN(Analyze Old/New), DO/DN(Dynamic analyze Old/New),

EO/EN(Edit and analyze Old/New), VO/VN(View Old/New), Z(oom)

Job submit time . 2011-08-27-08.44.06 Bindtime . . 2011-08-27-08.41.08.053757Collection/Plan . BINDDEMOProgram SQLZDEMOProgram . . . . . SQLZDEMOLibrary . . . . . SE.PRODUCT.IDUG.DEMO.DBRM

StmtNo StmtNo Bad access types Bad access types ExplainCatlg DBRM Imp before BIND with BIND sqlcode----- ----- --- ---------------------- ---------------------- -------

__ 0 66 NEW NMIX 0__ 10 40 EQ NMIX NMIX 0__ 18 24 EQ 0S 26 58 WRS SORT 0

----- ----- --- ---------------------- ---------------------- -------



ImpactExpert for DB2 z/OS ----- Access Paths ------- LINE 00000001 COL 001 080 Command ===> ________ Scroll ===> PAGEMode: PRE-BIND LOCAL DB2: Q10APrimary cmd: END, C(atalog data), D(etails on/off), S(tatement text)

Collection . BINDDEMO Timestamp. . 2011-08-27-08.44.16.630000Package. . . SQLZDEMO Version. . . V1StmtNo old . 26 Bindtime . . 2011-08-27-08.41.08.053757StmtNo new . 58 SE.PRODUCT.IDUG.DEMO.DBRM

Access path before BIND ---------------| Access path with BIND ----------------

TABLE QB PN AC MA ME IX PR | TABLE QB PN AC MA ME IX PRINDEX TY CO TH ON FT | INDEX TY CO TH ON FT

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -SYSCOLUMNS 1 1 I 1 Y | SYSCOLUMNS 1 1 I 1 Y S

DSNDCX01 | DSNDCX01| 1 2 0 3 N|

Milliseconds: 21 | Milliseconds: 89S i it 79 | S i it 348Serviceunits: 79 | Serviceunits: 348-------------------------------------------------------------------------------

******************************** Bottom of Data ********************************


Application Release Rollout – Customer Example



414 Packages processed ImpactExpert - Summary Package Level

Packages processed 414g p

Packages analyzed 414bound with explain(no)0

Unchanged 395W d 5

unchanged access paths395

Worsened 5Improved 14

QA omitted 409QA needed 5

worsened access paths

improved access paths

5

14

0 bound with explain(no)

ImpactExpert - Summary SQL Level

SQL analyzed 4304

Unchanged 4264

5 QA needed

gWorsened 7Improved 33

QA omitted: 98,79 %Critical access path changes


5 QA neededp g

prevented: 7


678

CPU600700

SQL

23456 CPU

consumptionoldCPUconsumptionnew 200

300

400500 SQL

statementsexaminedSQLstatementschanged

01

0100

Savings of CPU consumption are in average more than 80%. Additi ll th ff t f l i ti ti f SQLAdditionally the effort for manual investigation of SQL can be reduced to only 20 percent of the original effort.



Production


BIX – Production Key Features

Secures Performance resulting from REBINDs

Nightly production maintenance, i.e., REBIND after RUNSTATS.

Migration to new DB2 releases and application of APARs, or anytime when global REBINDs are recommended

Protects dynamic SQL from degradations


BIX – Rebind Control

Automatically guarantees REBINDs that improve performance

Any utility automation tool, e.g.,RealTimeMaintain

BindEXPLAIN TheImpactExpertEXPLAIN The

Pre-Check!

LOG



ImpactExpert for DB2 z/OS ----- Programs Summary ------------- Status 1 from 8 Command ===> ________ Scroll ===> CSR Mode: REBIND ANALYSIS DB2: Q10APrimary cmd: END, R(efresh), Z(oom) y , ( ), ( )Line cmd: S(elect)

COLLECTION : *______________________________________________________________ PACKAGE : *______________________________________________________________ PLAN : *_______

Status Programs Statements---------------------------------------- ---------- ----------

_ PROCESSED 152 1325 _ - IMPROVED 6 57

- CHANGED 8 25_ - CHANGED 8 25 _ - UNCHANGED 131 1203 _ - WORSENED 7 40 _ NON-DETERMINABLE 0 0 _ PLAN_TABLE ISSUES 0 0

ERROR 0 0 _---------------------------------------- ---------- ----------



ImpactExpert for DB2 z/OS -- All Programs (1/6) --------- Program 101 from 152 Command ===> ________ Scroll ===> CSR Mode: REBIND ANALYSIS DB2: Q10APrimary cmd: END, S(how prepared), Z(oom), L(ocate) programsLine cmd: S(tatements), A(nalyze), R(EBIND prepare), Z(oom)

Imp Collection / Plan Program Job Submit Time Rel Rec EX--- ------------------ -------- ------------------- --- --- ---

_ EQ RTDX0510 MDB2MODI 2011-09-17-11.05.58 V9 NO IMP RTDX0510 MDB2RTDX 2011-09-17-11 05 58 N/A YES_ IMP RTDX0510 MDB2RTDX 2011 09 17 11.05.58 N/A YES

_ EQ RTDX0510 MDB2SUB 2011-09-17-11.05.58 N/A NO _ EQ RTDX0510 MDB2SYSC 2011-09-17-11.05.58 N/A NO _ EQ RTDX0510 MDB2SYST 2011-09-17-11.05.58 N/A NO _ EQ RTDX0510 MDB2SY8C 2011-09-17-11.05.58 V9 NO

EQ RTDX0510 MDB2SY8T 2011-09-17-11.05.58 N/A NO __ EQ RTDX0510 MDB2T001 2011-09-17-11.05.58 N/A NO _ CHG RTDX0510 MDB2T002 2011-09-17-11.05.58 N/A NO _ EQ RTDX0510 MDB2T003 2011-09-17-11.05.58 V9 NO S WRS RTDX0510 MDB2T004 2011-09-17-11.05.58 V9 NO _ EQ RTDX0510 M2DBRIV9 2011-09-17-11.05.58 V9 NO

Q 0510 2 9 2011 09 17 11 05 58 9 O_ EQ RTDX0510 M2DBRTV9 2011-09-17-11.05.58 V9 NO _ EQ RTDX0510 O2DBIX 2011-09-17-11.05.58 N/A NO _ CHG RTDX0510 O2DBJB 2011-09-17-11.05.58 V9 NO _ EQ RTDX0510 O2DBSH 2011-09-17-11.05.58 N/A NO



ImpactExpert for DB2 z/OS -- REBIND Overview (1/4) ------------ Stmt 1 from 40 Command ===> ________ Scroll ===> CSR Mode: REBIND ANALYSIS DB2: Q10APrimary cmd: END, SE(tup Analyze), Z(oom), L(ocate) stmtnoLine cmd: S(elect), A(nalyze), D(ynamic analyze), E(dit and analyze),

V(iew), Z(oom)

Job submit time . 2011-09-17-11.05.58 Bindtime . . 2011-09-16-18.09.15.916093Collection/Plan . RTDX0510Program MDB2T004Program . . . . . MDB2T004Version . . . . .

Bad access types Bad access types ExplainStmtNo Imp before REBIND with REBIND Hint sqlcode------ --- ---------------------- ---------------------- ---- -------

_ 1091 WRS TS 0_ 1098 IMP LP,NXO 0_ 1113 EQ TS TS 0_ 1120 EQ TS TS 0_ 1135 EQ TS TS 0_ 1142 EQ 0_ 1157 WRS TS 0_ 1164 IMP LP,NXO 0_ 1179 WRS TS 0


BIX – DSC Protection

Dynamic SQL management and protection: Proactive QA

DB2P production system DB2T test systemDB2P production system DB2T test system

_HISTCatalog

StatisticsCatalog

Statistics

RUNSTATSwith DSC

ProtectionTM

Plantable BIXPlantable

BIXDynamicExplain

Plantable

Explain DSC

ProductionRUNSTATS

Dynamic SQL

Dynamic SQL

Dynamic Stmt

Cache

CaptureDSC


BIX – AP Restore

Dynamic SQL management and protection:Reactive QA

DB2P production systemDB2P production system

AP restore featureTM

CatalogStatistics

RUNSTATSwithout DSC ProtectionTM

CatalogStatisticsReposi-

t

BIXImpact-A l i

Plantable

tory

PlantableHistory

Comparison

Analysis

Explain DSC

History

Dynamic SQL

Dynamic Stmt

Cache

CaptureDSC



ImpactExpert for DB2 z/OS ----- Job Overview (1/4) ------------- Job 1 from 1 Command ===> Scroll ===> CSR Mode: DSC PROTECTION DB2: Q10APrimary cmd: END, A(ll), D(SC Extracts), E(Imported DSC Extracts), N(ew),

I(mport), X(EXport), R(efresh), Z(oom), L(ocate) submit timeLine cmd: S(tatements), A(uthIDs), D(elete), R(eset statistics),

V(iew extract), Z(oom)

Job name Step name Submitter Submit time PLTB owner Job status-------- --------- --------- ------------------- ---------- ----------BAIMDSC CHECKDYN HOPPE 2011-08-27-15.13.57 HOPPE DONE-------- --------- --------- ------------------- ---------- ----------



ImpactExpert for DB2 z/OS -- Dynamic SQL Overview (1/12) ------- Stmt 1 from 3 Command ===> Scroll ===> CSR Mode: DSC PROTECTION DB2: Q10APrimary cmd: END, SE(tup Analyze), Z(oom), L(ocate) stmtid oldLine cmd: S(elect), DO/DN(Dynamic analyze Old/New), R(eset statistics),

EO/EN(Edit and analyze Old/New), VO/VN(View Old/New), Z(oom)( y ), ( ), ( )

Bad access types Bad access typesStmtID old StmtID new Imp old new---------- ---------- --- ---------------------- ----------------------

90 90 WRS NMIX LP,SORT93 93 WRS NMIX LP,SORT250 250 WRS TS,LP TS,NMIX,MSJN

---------- ---------- --- ---------------------- ----------------------


BIX – Workload Analysis

Trace Facility

Dynamic Statement Cache Capture (Online / Batch) Dynamic Statement Cache Capture (Online / Batch)

SQL Aggregation Feature

Trace Hostvar values

Su orts Catalog History Tables Supports Catalog History Tables

Integrated Catalog Statistics Browser

Index what-if analysis


BIX – DSC Browser

Analyze for DB2 z/OS ----- Limit DSC Snapshot --------------------------------Command ===> DB2: Q10A

Primary cmd: END

MEMBER : Blank(Connected DB2) / *(All members) / member name

NO LIMITATION : XHIGHEST VALUES : EXCEED THRESHOLD: THRESHOLD:EXCEED THRESHOLD: THRESHOLD:

For limitation to highest values or exceeding of specified thresholdEXECUTIONS : ROWS PROCESSED : SORTS : BUFFER READS : ROWS EXAMINED : PARALLEL GROUPS : BUFFER WRITES : INDEX SCANS : RID EXCEED DB2 LIMITS : GETPAGES : TABLE SPACE SCANS : RID EXCEED STORAGE :

For limitation to highest values onlyELAPSE TIME : CPU TIME :

WAIT TIME FORWAIT TIME FOR ...SYNCRONOUS I/O : SYNCR. EXECUTION : READS OTHER THREADS : LOCK AND LATCH : GLOBAL LOCKS : WRITES OTHER THREADS :


BIX – DSC Browser

Analyze for DB2 z/OS ----- Filter Dynamic Statement Cache --------------------Command ===> DB2: Q10A

Primary cmd: END y

FIRST TABLE : ________________________________________________________________CREATOR ________________________________________________________________

FIRST TABLE : ________________________________________________________________NAME ________________________________________________________________

QUALIFIER : ________________________________________________________________

PRIMARY : ________________________________________________________________AUTHIDAUTHID ________________________________________________________________

SELECT X CURRENT USERS between and (Integer)INSERT X STMT COUNT between and (Integer)UPDATE X AVG CPU TIME between and (MM:SS.TTT)DELETE X AVG ELAPSE TIME between and (MM:SS.TTT)

AVG GETPAGES between and (Integer)Total stmts 104 OUTPUT LIMIT: 10000 0 - 25000 Max number of statements to be displayed


BIX – DSC Browser

ImpactExpert for DB2 z/OS --- Dynamic Statement Cache (4/8) -- Stmt 1 from 182 Command ===> ________ Scroll ===> CSR

DB2: Q10APrimary cmd: END, SE(tup), Z(oom), L(ocate) getpagesPrimary cmd: END, SE(tup), Z(oom), L(ocate) getpagesLine cmd: Z(oom), A(nalyze), D(ynamic Analyze), E(dit Statement), O(bjects),

S(tatement Text), T(able), X(EXecute)Synchr. Synchr. Rows Rows Index Tablespc.

StmtID Buffer Rd Buffer Wr examined processed Scans Scans---------- ---------- --------- ---------- ---------- ---------- ----------

_ 323 0 0 4473216 2290 0 2468_ 314 0 0 2996152 2295 0 2468_ 324 0 0 2284751 1234 0 1234_ 322 0 0 1621476 1234 0 1234_ 502 0 0 678110 2290 0 159

503 0 0 675998 1234 0 159_ 503 0 0 675998 1234 0 159_ 511 0 0 0 2144 0 0_ 493 0 0 0 2144 0 0_ 274 0 0 0 2290 0 0_ 381 34 0 0 77 0 0

391 0 0 3495 2819 316 158__ 276 0 0 0 1234 0 0_ 494 0 0 0 156 156 0_ 513 0 0 0 157 0 0_ 477 0 0 2628 49 98 67


BIX – DSC Browser


DB2: Q10APrimary cmd: END, SE(tup), Z(oom), L(ocate) getpagesLine cmd: Z(oom), A(nalyze), D(ynamic Analyze), E(dit Statement), O(bjects),

S(tatement Text), T(able), X(EXecute)Synchr. Synchr. Rows Rows Index Tablespc.

StmtID Buffer Rd Buffer Wr examined processed Scans Scans---------- ---------- --------- ---------- ---------- ---------- ----------

323 0 0 4473216 2290 0 2468_ 323 0 0 4473216 2290 0 2468_ 314 0 0 2996152 2295 0 2468_ 324 0 0 2284751 1234 0 1234_ 322 0 0 1621476 1234 0 1234_ 502 0 0 678110 2290 0 159

503 0 0 675998 1234 0 159_ 503 0 0 675998 1234 0 159_ 511 0 0 0 2144 0 0_ 493 0 0 0 2144 0 0_ 274 0 0 0 2290 0 0_ 381 34 0 0 77 0 0_ 391 0 0 3495 2819 316 158_ 276 0 0 0 1234 0 0_ 494 0 0 0 156 156 0_ 513 0 0 0 157 0 0_ 477 0 0 2628 49 98 67


BIX – DSC Browser


DB2: Q10APrimary cmd: END, SE(tup), Z(oom), L(ocate) getpagesLine cmd: Z(oom), A(nalyze), D(ynamic Analyze), E(dit Statement), O(bjects),

S(tatement Text), T(able), X(EXecute)

StmtID Total CPU Average CPU Total Elapse Average Elapse---------- HHHH:MM:SS.ttt HHHH:MM:SS.ttt HHHH:MM:SS.ttt HHHH:MM:SS.ttt

323 23 457 0 019 1 34 655 0 077_ 323 23.457 0.019 1:34.655 0.077 _ 314 17.138 0.014 1:16.865 0.062 _ 324 6.339 0.005 26.839 0.022 _ 322 4.753 0.004 18.608 0.015 _ 502 2.108 2.108 6.242 6.242

503 2.039 2.039 5.120 5.120_ 503 2.039 2.039 5.120 5.120 _ 511 0.446 - 1.205 0.001 _ 493 1.325 0.001 5.583 0.003 _ 274 0.493 - 1.278 0.001 _ 381 0.536 0.007 4.194 0.054 _ 391 0.979 0.006 2.785 0.018 _ 276 0.231 - 0.879 0.001 _ 494 0.175 0.001 0.603 0.004 _ 513 0.051 - 0.129 0.001 _ 477 0.036 0.001 0.094 0.001


BIX – SQL Aggregation

SQL Te t Count CPU Ti e

Level 1: Ignore values, spacing, cursor names, select clauses

SQL-Text Count CPU-TimeSELECT … WHERE COL = ‘ABC‘ 1 1sSELECT … WHERE COL = ‘BCD‘ 1 1sSELECT … WHERE COL = ‘CDE‘ 1 1sSELECT … WHERE COL = ‘DEF‘ 1 1sSELECT … WHERE COL = ‘EFG‘ 1 1s…

SQL-Text Count CPU-TimeSELECT … WHERE COL = ‘ABC‘’ 10.000 10.000s…


BIX – Version Migration control (EarlyPrecheckTM)

Goal:

Detection of potential access paths degradations before

migration of the production system by comparing the

access paths of a DB2 Vnow production system with the

access paths on a Vnext test system (based on Vnow

production statistics) without affecting the production

system at the earliest possible stage for static and

dynamic SQLdynamic SQL.


BIX – Version Migration control (EarlyPrecheckTM)

Runs stand-alone for global package processing

PlanTable

PlanTable

Comparison

DB2I V10 DB2P V8

Impact-Analysis

Worsened

improved New AccessPath Patternor equal

DB2 10 behaviour Report


BIX – Access Path Comparison

ImpactExpert for DB2 z/OS ----- Comparison --------- LINE 00000077 COL 001 080Command ===> Scroll ===> CSRMode: Precheck Static DB2: Q10APrimary cmd: END, C(atalog data), D(etails on/off), S(tatement text)

Collection . RTDX0510 Timestamp. . 2011-09-24-08.13.50.890000 Package. . . DSMORPB Contoken . . 184177A60269695E StmtNo . . . 332 Bindtime . . 2011-08-23-14.41.12.401132

Verify the access path changes

Access path before REBIND -------------! Access path with REBIND --------------

TABLE QB PN AC MA ME IX PR ! TABLE QB PN AC MA ME IX PRINDEX TY CO TH ON FT ! INDEX TY CO TH ON FT

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -SYSTABLESPACE 1 1 R 0 N S ! SYSTABLESPACE 1 1 R 0 N SSYSTABLEPART 1 2 I 2 1 N ! SYSTABLEPART 1 2 I 2 4 N L

DSNDPX01 ! DSNDPX011 3 0 3 N ! 1 3 0 3 N

SYSDATABASE 2 1 R 0 N S ! SYSDATABASE 2 1 I 0 N! DSNDDX02

2 2 0 3 N ! 2 2 0 3 N!

Milliseconds: 32 ! Milliseconds: 28Serviceunits: 125 ! Serviceunits: 110


-------------------------------------------------------------------------------

BIX – Version Rule Set

Each statement is classified to one of the following categories: UNCHANGED – Statements without Access Path

Change IMPROVED – Statements with Improved Access

Path DB2 Vnext special patterns (examples) DB2 Vnext special patterns (examples) V9 pattern 1* – Usage of smaller index V9 pattern 2* – Usage of bigger index, sort

avoided V9 pattern 3* – Usage of smaller index, index

only get lost V10 pattern 1 – Range list index scan

V10 pattern 2 In memory in list V10 pattern 2 – In memory in list

*considered as a result from skip migration


BIX – Version Rule Set

WORSENED – Statements with Worsened Access PathCHANGED Statements with Changed Access Path CHANGED – Statements with Changed Access Path (not classified)


Repeatable Precheck – Customer Example

Aug...Sep Okt Nov Mai...k k k k

DB2P Produktion Vnow

DB2P VnextCM

DB2P VnextNFM

rech

ec

rech

ec

rech

ec

rech

ec

DB2T Testsystem VnowDB2T Vnext

CMDB2T Vnext NFM

py

& P

r

py

& P

r

py

& P

r

py

& P

r

DB2I Impact VnextNFM

DB2I Impact VnextCM

DB2I Vnow

Co

p

Co

p

Co

p

Co

p

NFMCMVnow


BIX/SPX – Module List

Migration Precheck incl. ESD


Bind Management incl. ESD

Rebind Management incl. ESD

Enterprise Statistics Distribution (ESD)

i St t t C h SC S t


Dynamic Statement Cache (DSC) Support

Conditional Bind for DB2

DBRM Reconstruct for DB2

PackageManager for DB2


SPT01 Transparency for DB2

Conditional Bind – Key Features

Avoid unnecessary binds Optimize and accelerate the compile procedure Reduce CPU load Reduce locks the catalog Compatible with the new DB2 CoprocessorCompatible with the new DB2 Coprocessor


DBRM Reconstruct – Key Features

Recreates DBRMs from the DB2 catalog

Generates the original BIND command

Supports PLANs and PACKAGEs Supports PLANs and PACKAGEs

Allows to recreate multiple PACKAGE versions


PackageManager – Key Features

Examine the consistency of load modules and/or DBRMs to avoid SQLCODEs -805 and -818

Determine unnecessary ypackages

Generate and execute the FREE PACKAGE statements to clean up unnecessary package versions

Supports statically linked load modules


SPT01 Transparency – Key Features

Reduce the risk of bad access paths after REBIND Adds transparency to the new DB2 feature

Show existing copies (and their status!) Shows access paths for package copies

Allow to execute REBINDs for packages that e.g. are rated CHANGED with the ability to switch back in case of a performance degradation*

Reduces the amount of space needed in SPT01 due to the fact that no REBIND is generated by BIX for unchanged packages*

Allows to REBIND unchanged packages in a migration scenario without creating a copy*

* Along with BIX


Along with BIX

SQL Life Cycle Management · 2019. 1. 15. · SQL life cycle management Operational System Test...

Documents

Transcript of SQL Life Cycle Management · 2019. 1. 15. · SQL life cycle management Operational System Test...