Post on 15-Sep-2020
“This presentation is for informational purposes only and may not be incorporated into a contract or agreement.”
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not
a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole
discretion of Oracle.
Optimizing the Optimizer:Essential SQL Tuning Tips and
Techniques
Benoit DagevilleConsulting Member of Technical Staff
Khaled YagoubSenior Member of Technical Staff
Pete BelknapMember of Technical Staff
Mughees A. MinhasGroup Product Manager
Outline
• SQL Tuning Overview• SQL Tuning Best Practices
• Production system • Live Tuning• Remote Tuning
• Development system
• Techniques for Avoiding Plan Regressions After Database Upgrades
• Q & A
SQL Tuning Overview
Common Causes of Poor SQL Performance
• Stale optimizer statistics• CBO requires representative statistics to optimize
SQL execution• Not easy to determine when statistics are really stale• Not easy to determine when stale statistics are
actually impacting plan selection• Missing access structures
• Identifying useful indexes and MVs non-trivial
• Difficult to ascertain performance impact of new structures on DMLs
Common Causes of Poor SQL Performance
• Poor plan selection due to incorrect optimizer estimates• Manually hinting SQL a solution, but …
• Requires significant expertise• Is time consuming – trial and error method• Does not work for packaged applications
• Bad SQL design• Only real remedy is to restructure SQL• Requires expertise, time, application knowledge
Statistics Analysis
Tuning Using SQL Advisors
Access Structure Analysis
SQL Design Analysis
Plan Tuning (SQL Profiling)
Comprehensive Analysis
Gather Missing or Stale Statistics
Create a SQL Profile
Add Missing Indexes
Modify SQL Constructs
RecommendationSQL
Tuning/Access Advisor
SQL
SQL Tuning & Access Advisor Feature Comparison
WorkloadSQLModeAnalysis:
NoNo
Yes***YesYesYes
STA*
YesAccess Path: Materialized View LogsYesAccess Path: Materialized ViewsYesAccess Path: IndexesNoSQL Structure NoSQL ProfileNoStatistics
SAA**Feature
* STA: SQL Tuning Advisor** SAA: SQL Access Advisor*** B-tree indexes only
SQL Tuning Best Practices
Identifying Problem SQL• Two key sources for identifying problem SQL
• Automatic Database Diagnostic Monitor (ADDM)• Shows high-load SQL with impact %• Based on analysis of SQL, recommends SQL advisors as
needed• Not all high-load SQL are good candidates for advisors • E.g., SQL with HWM enqueue wait problem cannot be
tuned by SQL advisors but requires space reconfiguration• Top Activity Enterprise Manager (EM) screens
• Real Time Mode: • Source: v$active_session_history (ASH)• Period: Last one hour
• Historical Mode: • Source: Automatic Workload Repository (AWR)• Period: Last 7 days (default)
Using ADDM to Identify High-load SQL
Top Activity EM Screens: ASH
Top Activity EM Screens: AWR
SQL Tuning of Production System
SQL Tuning of Production System
• Tuning Considerations• Resource consumption
• Running SQL advisors consumes CPU, I/O, memory and can affect system performance
• Potential negative impact of recommendation• Implementing recommendations may impact system negatively,
e.g., optimizer stats refresh
• Before initiating tuning, answer the following questions• How much system resources will be consumed by tuning
activity?• Can the system spare resources needed for tuning?• How can the production system be shielded from possible
negative impact of tuning actions?
Resource Consumption of SQL Advisors
• SQL Tuning Advisor• Limited mode: Resource consumption minimal
• Stats, index and SQL restructure analysis is cheap• Average is less than 1 second per SQL statement
• Comprehensive mode: Resource consumption may be significant
• SQL Profiling can potentially consume non-trivial resources• Roughly comparable to amount of resources/time consumed
when executing SQL statement(s)
• SQL Access Advisor• Resource consumption depends on size of SQL workload• For small number of SQL, resource consumption not very high
Production System Tuning Options
• Options• Direct tuning of live system• Remote tuning
• Live system tuning• Run SQL Tuning Advisor in Comprehensive mode • Run SQL Tuning Advisor in Limited mode only if …
• System does not have spare resources to tune SQL
• Run SQL Access Advisor for few SQL at a time
• Perform remote tuning, if …• Cumulative resources/time consumed by all SQL statements
being tuned significant• System cannot spare resources
Live System Tuning Techniques1. Ensure tables referenced in SQL have representative optimizer
stats2. Run SQL Access Advisor
• For individual SQL, set “Recommendation Type” to indexes• MV not suitable for tuning individual statements
3. Run SQL Tuning Advisor• Test profile before making it PUBLIC
DBMS_SQLTUNE.ACCEPT_SQL_PROFILE (task_name => ‘<tuning task name>’,category => ‘MY_CATEGORY’);
ALTER SESSION SET SQLTUNE_CATEGORY=‘MY_CATEGORY’ ;
• Once satisfied with results set category to DEFAULTDBMS_SQLTUNE.ALTER_SQL_PROFILE(
name => ‘<SQL Profile Name>’,attribute_name => ‘category’, value => ‘DEFAULT’);
Remote SQL Tuning Techniques
1. Move SQL to Test
Production System Test System
2. Tune SQL3. Apply recomm. on production
• Performed to shield production system from performance impact of running SQL advisors
Remote Tuning Method
• Test system setup• Test system should be identical to production
system in• Schema• Data distribution• Volume
• If system smaller than production system, ensure data distribution is similar
Remote Tuning Method
1. Move SQL to test system• Create STS of high-load SQL on production
system• STS is a new object in Oracle 10g for managing SQL
workloads• STS stores SQL statements along with …
• Execution context: parsing user, bind values, etc.• Execution statistics: buffer gets, CPU time, elapse time,
number of executions, etc.• Use EM screen “Period SQL” to locate high-load SQL• Select “Create SQL Tuning Set” option to create STS
• Export STS to test
Creating STS of High-load SQL
Remote Tuning Method
• Export STS to testi. Create staging table
DBMS_SQLTUNE.CREATE_STGTAB_SQLSET( table_name => 'staging_table' );
ii. Move STS to staging tableDBMS_SQLTUNE.PACK_STGTAB_SQLSET(sqlset_name => 'my_sts',staging_table_name => 'staging_table');
iii. Export staging table to test system• Export/import, DataPump, DB Links, etc., can be used
iv. Unload STS from staging table into dictionary on test systemDBMS_SQLTUNE.UNPACK_STGTAB_SQLSET(sqlset_name => ‘my_sts’, replace => TRUE,staging_table_name => 'staging_table');
Remote Tuning Method
2. Tune SQL• Run SQL Tuning Advisor in comprehensive mode to tune
STS• Implement/verify recommendations• If SQL Profile recommended, set profile category to non-
default value before moving profile to production system
DBMS_SQLTUNE.ALTER_SQL_PROFILE(name => ‘<SQL Profile Name>’, attribute_name => ‘category’,
value => ‘MY_CATEGORY’);
Remote Tuning Method3. Apply recommendation to production
• Move profile from test to production system
i. Create staging tableDBMS_SQLTUNE.CREATE_STGTAB_SQLPROF( table_name => 'PROFILE_STGTAB');
ii. Move SQL Profile to staging tableDBMS_SQLTUNE.PACK_STGTAB_SQLPROF (profile_name => 'MY_PROFILE', staging_table_name => 'PROFILE_STGTAB');
iii. Export staging table to production system • Export/import, Data Pump, DB Links, etc., can be used
iv. Unload data from staging table to create SQL Profile on targetDBMS_SQLTUNE.UNPACK_STGTAB_SQLPROF(replace => FALSE, staging_table_name => 'PROFILE_STGTAB');
Remote Tuning Method
3. Apply recommendation on production (Contd.)• Test profile in private session to verify performance on
production system
ALTER SESSION SET SQLTUNE_CATEGORY=‘MY_CATEGORY’ ;
• If execution performance okay, set category to ‘DEFAULT’
DBMS_SQLTUNE.ALTER_SQL_PROFILE(name => ‘<SQL Profile Name>’, attribute_name => ‘category’, value => ‘DEFAULT’);
SQL Tuning on Development System
SQL Tuning on Development System
• Performance impact of tuning overhead not an issue
• Tuning Method• Capture SQL Workload• Redesign poorly written SQL• Identify appropriate access structures
Development System Tuning Steps
• Capture SQL Workload• Easiest way is to use cursor cache capture capability of
STS• Run workload• Capture workload in STS simultaneously
DBMS_SQLTUNE.CAPTURE_CURSOR_CACHE_SQLSET( sqlset_name => ‘MY_STS’, time_limit => 3600, repeat_interval => 60,sqlset_owner => own);
Development System Tuning Steps
• Use SQL Tuning Advisor to redesign poorly written SQL• Types of problems identified include:
• Semantic changes of SQL operators• Syntactic changes to predicates on indexed columns• SQL design issues
• Use SQL Access Advisor to optimize schema design• Identifies appropriate access structures• % improvement in performance of entire workload
for each structure recommended is provided
Techniques for Avoiding Plan Regressions After
Database Upgrades
Avoiding Plan Regressions After Upgrades
• Database Upgrade from 10.1 to 10.2• SQL execution plan regression major concern for
upgrades• Plan regressions can systematically be detected
and remedied• New Oracle 10g infrastructure makes this possible
Methodology for Avoiding Plan Regressions
Capture 10.1 Workload & Execution Plans
Step 1
Upgrade to 10.2
Step 2
Capture 10.2 Workload & Execution Plans
Step 3
Compare 10.1 and 10.2 Execution Plans
Step 4Tune SQL with Performance Regressions
Step 5
10.1 DB
Methodology for Avoiding Plan Regressions
Capture 10.1 Workload & Execution Plans
Upgrade to 10.2
Capture 10.2 Workload & Execution Plans
Tune SQL with Performance Regressions
Compare 10.1 and 10.2 Execution Plans
Step 1 Step 2 Step 3
Step 4 Step 5
10.1 DB
Upgrade Steps for Avoiding Plan Regressions
1. Capture 10.1 workload and executions plans• Increase AWR retention on 10.1 production system
• Motivation: Capture top SQL of entire workload cycle• If system upgraded is test system
• Ideally should replica of actual system to be upgraded (using RMAN backup)
• Ensure configuration of AWR, optimizer parameters, etc., same as actual system
• If data distribution/volume on test system different than production, import optimizer stats from production
• Allow normal production workload to run on system before starting upgrade testing
Methodology for Avoiding Plan Regressions
Capture 10.1 Workload & Execution Plans
Upgrade to 10.2
Capture 10.2 Workload & Execution Plans
Tune SQL with Performance Regressions
Compare 10.1 and 10.2 Execution Plans
Step 1 Step 2 Step 3
Step 4 Step 5
10.1 DB
Upgrade Steps for Avoiding Plan Regressions
2. Upgrade test system to 10.2• Use standard upgrade method• Create baseline STS
• Baseline STS created in 10.2 not 10.1• 10.2 STS also contains execution plans
• Capture all SQL from AWR in production system
• Use EM or PL/SQL API’s to create STS
Example 1: Capturing All AWR SQL in STS
declareown VARCHAR2(30) := '&owner';bid NUMBER := '&begin_snap';eid NUMBER := '&end_snap';sts_cur dbms_sqltune.sqlset_cursor;
begindbms_sqltune.create_sqlset(sqlset_name => '10.1 sts', sqlset_owner => own);
open sts_cur for select value(P) from table(dbms_sqltune.select_workload_repository(bid,eid, null, null, null, null, null, 1, null, 'ALL')) P;
dbms_sqltune.load_sqlset(sqlset_name => '10.1 sts', populate_cursor => sts_cur, load_option => 'MERGE');
end;/
Inputs: • Snapshot range: begin and end snapshot• Owner: STS owner
Methodology for Avoiding Plan Regressions
Capture 10.1 Workload & Execution Plans
Upgrade to 10.2
Capture 10.2 Workload & Execution Plans
Tune SQL with Performance Regressions
Compare 10.1 and 10.2 Execution Plans
Step 1 Step 2 Step 3
Step 4 Step 5
10.1 DB
Upgrade Steps for Avoiding Plan Regressions
3. Capture 10.2 workload and execution plans • Run workload on 10.2 and capture in new STS
simultaneously• Use dbms_sqltune.capture_cursor_cache_sqlset
procedure
Example 2: Capturing SQL from Cursor Cache in STS
declareown VARCHAR2(30) := '&owner';time_lim NUMBER := &time;
begindbms_sqltune.create_sqlset(sqlset_name => '10.2 sts',
sqlset_owner => own);
dbms_sqltune.capture_cursor_cache_sqlset (sqlset_name => '10.2 sts', time_limit => time_lim, repeat_interval => 60,sqlset_owner => own);
end;/
Inputs: • Time Limit: Workload execution time• Owner: STS owner
Methodology for Avoiding Plan Regressions
Capture 10.1 Workload & Execution Plans
Upgrade to 10.2
Capture 10.2 Workload & Execution Plans
Tune SQL with Performance Regressions
Compare 10.1 and 10.2 Execution Plans
Step 1 Step 2 Step 3
Step 4 Step 5
10.1 DB
Upgrade Steps for Avoiding Plan Regressions
4. Compare 10.1 and 10.2 SQL executions plans• Query contents of “10.1 sts” and “10.2 sts” via
DBA_SQLSET_STATEMENTS to see if:• Top 100 SQL in “10.2 sts” were also found in
“10.1 sts”• Any SQL in top 100 have had plan changes
Example 3: Query for Comparing STSselect * from (select r2_sts.sql_id,
r1_sts.plan_hash_value old_plan_hash,r2_sts.plan_hash_value new_plan_hash,r2_sts.elapsed_time/r2_sts.executions elap_time,nvl2(r1_sts.plan_hash_value, 0, 1) new_plan,r2_sts.sql_text
from dba_sqlset_statements r1_sts,dba_sqlset_statements r2_sts
where r1_sts.sqlset_name (+) = '10.1 sts' and r1_sts.sqlset_owner (+) = '&own' and r1_sts.sql_id (+) = r2_sts.sql_id and
r1_sts.plan_hash_value (+) = r2_sts.plan_hash_value and r2_sts.sqlset_name = '10.2 sts' andr2_sts.sqlset_owner = '&own'
order by elap_time desc)where rownum < 100 ;
Example 4: Query Output Comparing Contents of STS
• Sample query output• Output shows top 100 SQL in 10.2 ordered by elapsed time. • Old and new plan hash values shown• NEW_PLAN column indicates if plan was not captured in 10.1.
SQL_ID OLD_PLAN_HASH NEW_PLAN_HASH ELAP_TIME
------------- ------------- ------------- --------
NEW_PLAN SQL_TEXT
---------- --------------------------
fyrbzparfb4gb 403312434 403312434 26 0 select * from scott.dept
Methodology for Avoiding Plan Regressions
Capture 10.1 Workload & Execution Plans
Upgrade to 10.2
Capture 10.2 Workload & Execution Plans
Tune SQL with Performance Regressions
Compare 10.1 and 10.2 Execution Plans
Step 1 Step 2 Step 3
Step 4 Step 5
10.1 DB
Upgrade Steps for Avoiding Plan Regressions
5. Tune SQL with performance regressions• Examine plan changes to identify regressions
• Use dbms_xplan.display_sqlset to examine execution plans in STS
• Tune SQL with plan regressions using techniques discussed in remote tuning section
select * from table(dbms_xplan.display_sqlset(&sqlset_name, &sql_id, &plan_hash_value, 'TYPICAL',&sqlset_owner));
Summary
• Tuning techniques for production and development systems vary• Production system considerations
• Tuning activity performance overhead • Risk management of tuning actions
• Development system considerations• SQL design improvement• Optimization of access structure design
• Oracle Database 10g has powerful capabilities to minimize upgrade performance regressions
• Reduce risk by fully utilizing new capabilities