SQL Performance Analyzer: Performance...• Usage Scenarios • Upgrade testing: 9i R2, 10 g R1, 10...

SQL Performance Analyzer: Eliminating the Guesswork from SQL PerformancePrabhaker Gongloor (GP) Khaled YagoubPete BelknapDatabase Manageability GroupOracle Corporation

Outline

• Motivation• SQL Performance Analyzer (SPA) Overview• SPA for Pre-11g Database Releases • Usage Scenarios

• Upgrade testing:9i R2, 10g R1, 10g R2 � 10g R2 OR 11g

• Any change that impacts SQL performance

• Real-world Deployments• Conclusion

Please visit us at the OOW Demogrounds Booth Moscone West: L52/L51 : Database Replay/SPA

New!

Why SQL Performance Analyzer (SPA)?

• Businesses want systems that are performant and meet SLA’s

• SQL performance regressions are #1 cause of poor system performance

• Solution for proactively detecting all SQL regressions resulting from changes not available

• DBA’s use ineffective and time-consuming manual scripts to identify problems

SPA identifies all changes in SQL performance before impacting users

Real Application Testing

• SQL Performance Analyzer• Predicts impact of change on

SQL response time• Assess change by executing

query part of SQL in isolation without concurrency

• Unit testing of SQL, e.g., optimizer stats/config, drop index, etc.

• Can use for upgrades from • 9.2/10.1 ��10.2 or 11g• 10.2 ��10.2 or 11g

• SQL Performance Analyzer• Predicts impact of change on

SQL response time• Assess change by executing

query part of SQL in isolation without concurrency

• Unit testing of SQL, e.g., optimizer stats/config, drop index, etc.

• Can use for upgrades from • 9.2/10.1 ��10.2 or 11g• 10.2 ��10.2 or 11g

• Database Replay• Predicts impact of change on

workload throughput

• Assess change by replaying workload with concurrency

• Comprehensive testing of DB stack, e.g, memory, RAC, concurrency related

• Can use for upgrades from

• 9.2 or 10.2 ��11g

• Database Replay• Predicts impact of change on

workload throughput

• Assess change by replaying workload with concurrency

• Comprehensive testing of DB stack, e.g, memory, RAC, concurrency related

• Can use for upgrades from

• 9.2 or 10.2 ��11g

� Real Application Testing = SPA + Database Replay� SPA + Database Replay are complementary � Together, provide comprehensive & flexible testing solution

SQL Performance Analyzer (SPA)

SPA : Common Usage Scenarios

• Database upgrades and patch-set releases• 9.2/10.1 �10.2 or 11.1 releases• 10.2.0.x �10.2.0.y or 11.1 releases

• Database parameter changes• Schema changes• Optimizer statistics refresh • Implementation of tuning recommendations• I/O subsystem changes

SPA can be used for any change that affects SQL execution plan and performance

SQL Performance Analyzer: Overview• Helps users predict the impact of

system changes on SQL workload response time

• Builds different trials (experiment) of SQL workload performance (i.e., SQL execution plans and statistics)

• Analyzes performance differences

• Offers fine-grained performance analysis on individual SQL

• Integrated with SQL tuning set, SQL plan baselines, and SQL tuning advisor to form an End-to-end solution

Analysis Report

Compare

SQL Performance

SQL plans + stats SQL plans + stats

Pre-change Trial Post-change Trial

SQL Workload

SQL Performance Analyzer Generic Workflow

SQL Tuning Set

Fetch Next SQL

Execution Plan &Statistics

Test Execute

After Change

Trial

Compare

SQL Performance

Test Database

Before Change

Trial

Production Database

Cursor Cache

SQL Tuning Set

Incremental Capture*

FtpExport/Import

*Negligible Overhead , < 1- 2%

SPA Report

4

1

3

2

5

SPA Report Regressed SQL Statements

SPA Enterprise Manager Interface • Rich GUI through Enterprise Manager

• DBMS_SQLPA package PL/SQL API

SPA Usage Scenarios

Database Upgrade: 9.2/10.1* �� 10.2

Scenario 1:I have heard premier DB support for 9.2 has

ended, I want to upgrade 10.2 database release. How can I use 11g SPA functionality to accomplish the upgrade?

Goal:Assess impact of upgrade on SQL workload

performance on a test system using SPA so that there are no surprises after upgrade.

* For 10.1, the solution is similar to 9.2

No patches needed

11g Test System: No application schema/data, stores SPA results11.1.0.7 or 11.1.0.6 + patch

Clone of Production10.2 system + Test Exec patch installed

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2System Setup

Test DatabaseProduction Database

Prod DB (9.2/10.1) Test DB (10.2)

11g SPA System

Metalink Note: 560977.1

Upgrade

2. Remote Test Execute Workload over DB Link

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2SPA Enhancements


Prod DB (9.2/10.1) Test DB (10.2)

11g SPA System

1. Consume 9i SQL Trace files and generate STS

Upgrade

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2Test Database

Production Database

Prod DB (9.2/10.1)Test DB (10.2)

Collect execution stats

Send SQL for Remote Execution

2. Generate STS from trace file

3. Generate Pre-Change Trial

4. Send SQL to 10.2 DB to execute

5. Compare performance and generate SPA report

11g SPA System

Move trace file to 11g SPA system

1. Capture SQL workload using SQL*Trace

Upgrade


Production Database

Prod DB (9.2/10.1)Test DB (10.2)





4. Compare performance and generate SPA report 11g SPA System



Upgrade

Scenario 1: Database Upgrade: 9.2/10.1 ��10.2Step 1: Capture SQL workload using SQL Trace

• Identify all interesting workloads such as month-end, daily peak, etc.

• Capture SQL trace for the workload, few sessions at a time • Use dbms_support/dbms_monitor package, these support

• bind value capture

• tracing other running sessions

• SQL trace considerations• time_statistics=true: Important for performance data

• user_dump_dest• max_dump_file_size

• trace_file_identifier

• Performance overhead: 10-15% for traced sessions

• SQL Trace files only have object identifiers• Create mapping table to map object identifiers in

trace files to schema names

Scenario 1: Database Upgrade: 9.2/10.1 ��10.2Step 1: Capture SQL workload using SQL Trace (contd.)

DEPTSCOTT124

EMPSCOTT123

NameOwnerId

SQL trace files

* SQL in Note pages/OTN

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2Step 2: Transport Workload and Create STS

SQL trace files FtpExport/Import

• Transport SQL trace files, mapping table using ftp/expdp/impdp.., etc.

• Create STS from trace files using dbms_sqltune API (SQL in Notespage/OTN)

• Specify directory object containing trace files, mapping table, STS name as input

DEPTSCOTT124

EMPSCOTT123

NameOwnerId

Mapping Table

DEPTSCOTT124

EMPSCOTT123

NameOwnerId

Mapping Table

SQL trace files

Production Database11g SPA System

SQL Tuning Set

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2Step 3a: Create SPA Task

• Create SPA task on 11g SPA System

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2Step 3a: Create SPA Task

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2 Step 3b: Establish “Before Change” Trial

• Trial Creation Method: Select “Build From SQL Tuning Set” option to use plans and statistics from 9i

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2 Step 4: Establish “After Change” Trial


Production Database

Prod DB (9.2/10.1)Test DB (10.2)






11g SPA System



Upgrade

Scenario 1: Database Upgrade: 9.2/10.1 �� 10.2 Step 5: Compare and Generate Report

• Compare Pre-Change and After-Change Trials based on a performance metric• Oracle recommends using CPU_TIME and BUFFER_GETS

metrics• Use multiple metrics that provide repeatable and

comprehensive statistics

Fixing Regressed SQL• Systematic problems

• Check un-analyzed tables, PGA memory, statistics collection,

system statistics

• Refer “Upgrading from Oracle 9i to 10g: What to expect from the

Optimizer” on OTN

• For statements suffering from isolated problems use one of the following fixes• SQL Profiles: Implement Profiles recommended by SQL Tuning

Advisor (STA)• Stored Outlines**: If no profile was recommended by STA, then

capture Stored Outlines in 9i for the targeted SQL statements. Import stored outline into 10g.

Database Upgrade: 10.2.0.x �� 10.2.0.y

Scenario 2:One of the database I’m managing is on

10.2.0.2. How can I use 11g SPA functionality to accomplish an 10.2.0.4 patchset upgrade?

Goal:Assess impact of upgrade on SQL workload

performance using SPA so that there are no surprises after upgrade.

No patches needed

11g Test System: No application schema/data, stores SPA results11.1.0.7 or 11.1.0.6 + patch

10.2.0.x: Clone of Prod

10.2 systems + Test Exec patch installed

Scenario 2: Database Upgrade: 10.2.0.x �� 10.2.0.ySystem Setup


Prod DB (10.2.0.x)Test DB (10.2.0.y)

11g SPA System

Metalink Note: 560977.1

Test DB (10.2.0.x)

UpgradeUpgrade

Scenario 1: Database Upgrade: 10.2.0.x �� 10.2.0.yWorkflowProduction Database

Prod DB (10.2.0.x)

11g SPA System

Move STS dmp file to 11g SPA

system

1. Capture SQL workload to STS using Increment Cursor Cache Capture

Test Database

Test DB (10.2.0.y)

UpgradeUpgrade

Test DB (10.2.0.x)


2. Import STS

3. Send SQL for test execution on 10.2 before and after upgrade



Scenario 3: Using SPA Functionality for 9i/10g ��11g Upgrades

• Similar workflow as Scenario 2• Use 11g SPA system and test execute on 10g/11g

source and destination target databases • Stores results of experiments separately• Allows use of latest releases for 11g SPA system

Schema Changes

Scenario 4:A few months after upgrade, the database

engineering team wants to add few indexes on the production system. They want my input on how it will impact the database and application

Goal:Assess impact of schema changes on SQL

workload on test system performance & make sure are no negative effects of the change

Schema Changes

Schema Changes

Change Accepted

1

Evaluating Optimizer Statistics Refresh

Scenario 5:Can I use SPA to check if any SQL statements

regressed due to optimizer statistics refresh on my 10.2 production databases. If so, how can I evaluate the refreshed optimizer statistics?

Goal:Assess impact of optimizer statistics gathering

on SQL workload performance on production system & make sure are no negative effects of the change

Evaluating Optimizer Statistics Refresh

• Assumptions• Optimizer has already gathered statistics on the database• Statistics refreshed periodically• No prod copy is available on test

• Use “11g SPA system” to evaluate optimizer statistics on 10.2 production database• Remote test execute before/after statistics refresh

• Analyze SPA report and take appropriate action• Overall improvement but few SQL regressions

• Solution: Use SQL Profiles for regressed SQL• No improvement and many regressions

• Solution: Revert to old statistics: Use optimizer statistics retention/history feature

• For Oracle Database 11g, use publish pending statistics feature to publish statistics after evaluation of statistics

Evaluating Optimizer Statistics Refresh for 10.2

Production Database

2. Import STS

3. Use SPA to detect plan changes


11g SPA System

1. Capture SQL workload to STS

Prod DB (10.2)

Prod DB (10.2)

Opt Statistics Refresh

5. Retain new statistics or revert to old statistics or use SQL Profiles for regression tuning

SPA Real-World Case Studies

Case Study 1:Large Hotel Chain

• Upgrade critical customer-facing application providing rates for room reservations from Oracle Database 10.2.0.4 to 11.1

• Highly volatile data where plan stability is critical

• Unsuccessfully used synthetic queries to test previous upgrades

Challenge

• SQL Performance Analyzer to identify SQL regressions

• SQL Profiles to tune SQL transparently

• SQL Plan Baselines for plan stability

SolutionApproach

Benefit• Very successful upgrade. No surprises!

• Predictable performance and SLAs

• Reduced testing time from 5 months to 10 days

Large Hotel ChainProblems Resolved

• Issues Prior to Upgrade• Volatile nature of data sometimes produced inefficient queries based

on stale stats. • Impossible to maintain good statistics due to the nature of data load

– frequent and widely changing• Bind variables didn't always help• Outlines hurt in some cases due to nature of the data values

• After Oracle Database 11g Upgrade:• No performance problems noticed• Predictable query performance and plan stability due to Plan

Baselines• Provided a history of plan changes• Allowed a more efficient usage of bind variables due to adaptive

cursor sharing

SPA Case Study 2:E-Business Suite (EBS) Certification and Testing

• Certify EBS release 11i, R12 against Oracle Database 11g

• Complex & large workload: More than 650K unique SQL statements need to be validated

• Ensure application optimized for Oracle Database 11g

• Difficult to perform realistic and efficient testing with previous (home-grown) tools

Challenge

• SQL Performance Analyzer to run regression tests and identify performance deviations

• Regressions reported to base development for fixes

SolutionApproach

Benefit• Reduced testing time from 21 to 2 days for each release

• Faster and higher quality testing

• Faster adoption and certification of newer features

SPA: E-Business Suite Certification and Testing

• SPA has been extensively used for Apps certification and testing• R12.0.4 E-Business Suite w/ 11.1.0.6 DB: 650K unique SQL stmts• 11i E-Business Suite w/ 11.1.0.6 DB: 550K unique SQL stmts• Workload is complex, large

• Challenges before SPA deployment• Manually maintain SQL repository of statements • Need to maintain database hosting the SQL repository and java

application• SPA deployment

• Eliminated need to maintain SQL repository, java application, database hosting the repository

• Higher quality and realistic testing• Fine-grain, reporting analysis possible• Automatic tracking of trials and performance data

SPA: E-Business Suite Certification and Testing

• SPA is already being used in many scenarios • Certification of EBS (11i and R12) with Oracle Database 11.1.0.6

• Impact of new features on EBS• Extended Optimizer Statistics, SQL Plan Baselines, SQL Profiles

• Performance Testing: 10.2.0.3 Vs 11.1.0.6 Vs 11.2

• Impact of Parameter Changes: Obsolete/new parameters• Impact of HW changes

SPA Case Study 3: Internal Mail System• Background: Internal mail system, critical application

• Collaboration Suite 10.1.2.3, 4000 named users, 25 Average Active Sessions, RDBMS 4-node RAC, DG Physical Standby, 4 nodes x 8 (4-dual core) CPUs, 15Gb SGA per instance, ASM

• 4.5 TB DB size (allocated data), Apple XServe RAID, Linux-64 bit OEL4U4• Before SPA: Used manual processes, captured SQL with no bind data for testing and

checked plan regression, 80 hrs, but still not reliable

• SPA used for DB upgrade testing, from release 10.2.0.3 to 11.1.0.6 (Jan 2008)• Switching to 11g optimizer statistics gathering• Removing legacy, custom optimizer parameters• Moving to 11g optimizer code path

• SPA reported about 20% overall performance improvement • SQL Workload consisted of 818 statements• Testing revealed most workload would be unchanged (by buffer gets/elapsed time, etc)• Few statements improved, but for 5 critical statements plans regressed • SQL Plan Baselines used in production successfully for remediating these 5 queries

Case Study 3: Internal Mail system

• After upgrade, SLAs were same as before, and no instabilities were noticed in the last 8 months

• Upgrade Testing Summary with SPA• Resulted in reduced time/effort for actual testing from 80 hrs

to 2 hrs• More accurate and realistic testing than before• SLAs being met and successful production deployment

Resources on OTN

• Real Application Testing for Earlier Releases• Testing Performance Impact of an Oracle Database 9i/10g Release

1 to Oracle Database 10g Release 2 Upgrade with SQL Performance Analyzer

• Migration to Cost-Based Optimizer

• Upgrading from Oracle 9i to 10g: What to expect from the Optimizer

• Upgrade Companion: Metalink Note: 466181.1: One-stop shop for

Upgrades

• Real Application Testing Users Guide

“Oracle Real Application Testing reduces the time required to test changes by as much as 80%, lower testing costs by as much as 70%, mitigate risks by reducing the number of unexpected outages, and improve the quality of service for their IT operations.”

David MitchellSenior Vice President, OVUM

Source: Oracle Real ApplicationTesting – business agility through superior testing, Jan 2008

Conclusion

• SPA provides comprehensive and easy to use solution for assessing impact of changes on SQL response time• Eliminates SQL performance guess work!!

• SPA can be used to test many changes• Upgrade testing including 9i, 10g to 10.2 or 11g releases• Index creation/drop, statistics gathering, etc.

• Helps adopt technology faster by cutting down testing time from months for days

• With SPA and Real Application Testing businesses can• Stay competitive• Improve profitability• Be compliant

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Recommended Campground Demos

Moscone West

Exhibit Hall

Change Management & Data Masking for DBAs

Moscone West

Exhibit Hall

Complete Data Center Management

Moscone West Exhibit Hall

Application Quality Management

Moscone West

Exhibit Hall

Self-Managing Database: Automatic Application & SQL Tuning


Self-Managing Database: Automatic Fault Diagnostics


Self-Managing Database: Automatic Performance Diagnostics

Moscone West

Exhibit Hall

Oracle Real Application Testing: Database Replay and

SQL Performance Analyzer

LocationDemo

Recommended Sessions (Monday sessions)

Moscone South 304

2:30 pm

Monday, Sept. 22

First-Failure Fault Diagnosability and Diagnostics: Oracle Database 11g Features and Novel Approaches

Moscone West 2003

4:00 pm

Monday, Sept. 22

Oracle Enterprise Manager: Oracle's Management Solution for Your Enterprise

Moscone South 302

2:30 pm

Monday, Sept. 22

Performance Fundamentals for Oracle Database 10g and Oracle Database 11g

Moscone West 2001

1:00 pm

Monday, Sept. 22

Optimizing Application Performance: Application Testing Suite to the Rescue

LocationTimeDateSession Title

Recommended Sessions (Tuesday sessions)

Moscone South 303

5:00 pm

Tuesday, Sept. 23

Successful Upgrade Secrets: Preventing Performance Problems with Database Replay

Moscone South 303

1:00 pm

Tuesday, Sept. 23

Demystifying SQL Tuning: Tips and Techniques for SQL Experts

Moscone West 2003

11:30 am

Tuesday, Sept. 23

Advanced Performance Diagnostics: What the GUI Doesn't Tell You


Moscone West 2003

9:00 am

Tuesday, Sept. 23

Application Upgrade Secrets: Avoid Surprises While Making Database Changes

Moscone South 304

9:00 am

Tuesday, Sept. 23

Using Oracle Database 11g Real Application Testing to Simulate Production System Patching

Recommended Sessions (Wednesday sessions)

Moscone West 2001

5:00pm

Wednesday, Sept. 24

Deploying Oracle Enterprise Manager in a Secure Maximum Availability Architecture

Moscone West 2001

1:00 pm

Wednesday, Sept. 24

SQL Tuning Roundtable with the Experts


Moscone South 303

9:00 am

Wednesday, Sept. 24

Storage Monitoring Made Easy: Diagnosing I/O Performance Problems

Moscone West 2001

11:30 am

Wednesday, Sept. 24

Applications Data Privacy: An Expert Panel Discussion

Create SQL Set : Filters

Recommended Sessions (Thursday sessions)

Moscone South 303

3:00pm

Thursday, Sept. 25

Managing Oracle Grid Computing: Oracle Real Application Clusters, Oracle Automatic Storage Management, Oracle Data Guard

Moscone South 303

1:30 pm

Thursday, Sept. 25

Proactive Performance Monitoring with Baselines and Adaptive Thresholds

Moscone South 303

12:00 pm

Thursday, Sept. 25

Application Testing Best Practices: Real-World Customer Testimonials


Moscone South 301

9:00am

Thursday, Sept. 25

The Danish Experiment: Oracle Database 11g Shock Upgrades and Massive Workload Reduction via COBS

Moscone South 303

10:30 am

Thursday, Sept. 25

Integrating 40 Data Centers in Three Years: How Oracle'sDBAs Control the Data Center Explosion

SQL Performance Analyzer: Performance...• Usage Scenarios • Upgrade testing: 9i R2, 10 g R1, 10...

Documents

Transcript of SQL Performance Analyzer: Performance...• Usage Scenarios • Upgrade testing: 9i R2, 10 g R1, 10...