Oracle SQL tuning with SQL Plan Management
-
Upload
bjoern-rost -
Category
Software
-
view
185 -
download
1
Transcript of Oracle SQL tuning with SQL Plan Management
SQL tuning with SPMchange the way you think
Björn Rost
• consultant
• oracle database (HA and performance)
• OS systems (Linux and Solaris)
• automation puppet and ansible
• president of RAC SIG
• ACE Director
• > 42 presentations on 5 continents
BJÖRN
© 2014 Pythian Confidential
ABOUT PYTHIAN
3
11,400 Pythian currently manages more than 11,400 systems.
400+ Pythian currently employs more than 400 people in 200 cities in 35 countries
1997 Pythian was founded in 1997
Global Leader In IT Transformation And Operational Excellence
Unparalleled Expertise
• Top 5% in databases, applications, infrastructure, Big Data, Cloud, Data Science, and DevOps
Unmatched Certifications • 9 Oracle ACEs, 4 Oracle ACE Directors, 1 Oracle ACE Associate • 6 Microsoft MVPs, 1 Microsoft Certified Master • 5 Google Platform Qualified Developers • 1 Cloudera Champion of Big Data • 1 Mongo DB Certified DBA Associate Level • 1 DataStax Certified Partner, 1 MVP
Broad Technical Experience • Oracle, Microsoft, MySQL, Oracle EBS, Hadoop, Cassandra, MongoDB,
virtualization, configuration management, monitoring, trending, and more.
© 2014 Pythian Confidential
SOME OF OUR CLIENTS
4
AGENDA
• optimizer history and motivation
• SPM basics
• process design
• (our) implementation
• extra use cases
MY ORACLE HISTORY
• “installed” Oracle 8 - once
• “ran” two installations of 9i
• “serious” Oracle work not until 10g
RULE BASED OPTIMIZER
• matter of legends - a thing of the past
• not being updated since 7
• unsupported since 10g
RULE BASED OPTIMIZER
SQL> select * from attendees where gender = 'male';
RULE BASED OPTIMIZER
SQL> select * from attendees where gender = 'male';
SQL> select * from attendees where gender = 'male' and country = 'germany';
RULE BASED OPTIMIZER
SQL> select * from attendees where gender = 'male';
SQL> select * from attendees where gender = 'male' and country = 'germany';
If an index on “gender” exists, it will always be used. Even if full scan would be better or “country” had better selectivity
RBO JOINS
• driving table determined by access paths
• trying to avoid FTS on inner table
• nested loop and SMJ only (no hash join)
RULE BASED OPTIMIZER
• developers had to think when writing SQL
• make assumptions about data (in prod)
• but easy for DBAs
• plans were predictable
• and never changed (by themselves)
• blame was on developers
COST BASED OPTIMIZER (CBO)
• introduced in Oracle 7 (1992)
• based on statistics
• compares all possible plans by “cost”
• adapts to new situations
• volume of data
• skewed data and selectivity for different binds
• writing SQL is also really easy now
• don’t bother with join order, indexes etc
CBO PROBLEMS
• executions plans are now a moving target
• plenty of things to “change” and go wrong:
• statistics
• cardinality estimates
• different per variable/literal
• new access paths
• there have to be thresholds that can “flip”
FLIPPING PLANS
• noone calls about CBO finding better plans
• but plenty complain about worse plans
LOSS AVERSION
• Daniel Kahneman
• nobel price in economics
• coin-flip game payouts
• loose $5 vs finding $5
• use worse plan vs use better plan
FLIPPING PLANS
• fix is usually very easily
• some stop/manage gathering stats
• sometimes led to hint-crazyness
• some even want the RBO back!
• or “please freeze our plans”
WHY SPM
• gain plan stability
• flipping SQL execution plans
• rather “static app”
• SPM better in OLTP than DWH
• easy fix
• nasty consequences
• at “uncontrollable” times
• SQL Plan Management can really help!
PROJECT GOALS FOR SPM
• prevent random changes for the worse
• change management for plans (just like code, config …)
• introduce a process for new plans
‣ find and evaluate
‣ review
‣ approve/implement
WHAT IS SPM?
• add one more layer to CBO
• plan “quarantine”
• accept plans/baselines
• prevent new plans from being executed
• but store them anyway
• then review
BASELINES
• stored hints to reproduce a given plan
• examine with display_baseline +OUTLINE
• plan_hash_value to compare
•DBA_SQL_PLAN_BASELINES
BASELINES
select * from table ( dbms_xplan.DISPLAY_SQL_PLAN_BASELINE(
sql_handle=>’SQL_086fba8dc9c2c972', format=>'OUTLINE'));
RULE BASED OPTIMIZEROutline Data from SMB:
/*+ BEGIN_OUTLINE_DATA SWAP_JOIN_INPUTS(@"SEL$27F3751A" "THIS_1_"@"SEL$1") USE_HASH(@"SEL$27F3751A" "THIS_6_"@"SEL$11") USE_HASH(@"SEL$27F3751A" "THIS_1_"@"SEL$1") USE_NL(@"SEL$27F3751A" "THIS_4_"@"SEL$7") USE_NL(@"SEL$27F3751A" "THIS_3_"@"SEL$5") USE_NL(@"SEL$27F3751A" "THIS_2_"@"SEL$3") USE_NL(@"SEL$27F3751A" "THIS_5_"@"SEL$9") LEADING(@"SEL$27F3751A" "THIS_"@"SEL$2" "THIS_5_"@"SEL$9" "THIS_2_"@"SEL$3" "THIS_3_"@"SEL$5" "THIS_4_"@"SEL$7" "THIS_1_"@"SEL$1" "THIS_6_"@"SEL$11") FULL(@"SEL$27F3751A" "THIS_6_"@"SEL$11") FULL(@"SEL$27F3751A" "THIS_1_"@"SEL$1") INDEX_RS_ASC(@"SEL$27F3751A" "THIS_4_"@"SEL$7" ("A"."ID")) INDEX_RS_ASC(@"SEL$27F3751A" "THIS_3_"@"SEL$5" ("B_RECOMMENDATION"."ID")) INDEX_RS_ASC(@"SEL$27F3751A" "THIS_2_"@"SEL$3" ("C_MONTHLY_CREDIT"."ID")) INDEX(@"SEL$27F3751A" "THIS_5_"@"SEL$9" ("D_TARIFF"."ID")) FULL(@"SEL$27F3751A" "THIS_"@"SEL$2") OUTLINE(@"SEL$1") OUTLINE(@"SEL$2") MERGE(@"SEL$1") OUTLINE(@"SEL$58A6D7F6") OUTLINE(@"SEL$3") OUTLINE(@"SEL$4") MERGE(@"SEL$58A6D7F6") MERGE(@"SEL$3") OUTLINE(@"SEL$7237DA6D") OUTLINE(@"SEL$5") OUTLINE(@"SEL$6") OUTLINE(@"SEL$7") MERGE(@"SEL$7237DA6D") MERGE(@"SEL$5") OUTLINE(@"SEL$15E987C1") OUTLINE(@"SEL$8") OUTLINE(@"SEL$9") MERGE(@"SEL$7") MERGE(@"SEL$15E987C1") OUTLINE(@"SEL$096E5AED") OUTLINE(@"SEL$10") MERGE(@"SEL$9") MERGE(@"SEL$096E5AED") OUTLINE(@"SEL$B97648DD") OUTLINE(@"SEL$11") OUTLINE(@"SEL$12") MERGE(@"SEL$B97648DD") MERGE(@"SEL$11") OUTLINE(@"SEL$72AEFE3E") OUTLINE(@"SEL$13") MERGE(@"SEL$72AEFE3E") OUTLINE_LEAF(@"SEL$27F3751A") ALL_ROWS DB_VERSION('11.2.0.4') OPTIMIZER_FEATURES_ENABLE('11.2.0.4') IGNORE_OPTIM_EMBEDDED_HINTS END_OUTLINE_DATA */ --------------------------------------------------------------------------------
BASELINES
Plan hash value: 2175138020
-------------------------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | -------------------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 221 | 17 (6)| 00:00:01 | | 1 | SORT ORDER BY | | 1 | 221 | 17 (6)| 00:00:01 | |* 2 | HASH JOIN OUTER | | 1 | 221 | 16 (0)| 00:00:01 | |* 3 | HASH JOIN RIGHT OUTER | | 1 | 210 | 13 (0)| 00:00:01 | | 4 | TABLE ACCESS FULL | D_TARIFF | 156 | 2652 | 3 (0)| 00:00:01 | | 5 | NESTED LOOPS OUTER | | 1 | 193 | 10 (0)| 00:00:01 | | 6 | NESTED LOOPS OUTER | | 1 | 184 | 9 (0)| 00:00:01 | | 7 | NESTED LOOPS OUTER | | 1 | 175 | 8 (0)| 00:00:01 | | 8 | NESTED LOOPS OUTER | | 1 | 162 | 7 (0)| 00:00:01 | |* 9 | TABLE ACCESS FULL | PROMO | 1 | 158 | 7 (0)| 00:00:01 | |* 10 | INDEX UNIQUE SCAN | D_TARIFF_PK | 1 | 4 | 0 (0)| 00:00:01 | | 11 | TABLE ACCESS BY INDEX ROWID| C_MONTHLY_CREDIT | 1 | 13 | 1 (0)| 00:00:01 | |* 12 | INDEX UNIQUE SCAN | C_MONTHLY_CREDIT_PK | 1 | | 0 (0)| 00:00:01 | | 13 | TABLE ACCESS BY INDEX ROWID | B_RECOMMENDATION | 1 | 9 | 1 (0)| 00:00:01 | |* 14 | INDEX UNIQUE SCAN | B_RECOMMENDATION_PK | 1 | | 0 (0)| 00:00:01 | | 15 | TABLE ACCESS BY INDEX ROWID | A | 1 | 9 | 1 (0)| 00:00:01 | |* 16 | INDEX UNIQUE SCAN | A_PK | 1 | | 0 (0)| 00:00:01 | | 17 | TABLE ACCESS FULL | PROMO_SIMPLE | 551 | 6061 | 3 (0)| 00:00:01 | --------------------------------------------------------------------------------------------------------------------
BASELINE STATUS• enabled
• defaults to YES
• accepted
• this plan can actually be run
• default YES for 1st one, NO for next
• fixed
• stop collecting new baselines
• default: NO
SQL SELECT * FROM STUFF WHERE COL=?;
execute PLAN!
PARSING
SQL
sql_id
SELECT * FROM STUFF WHERE COL=?;
SELECT * FROM stuff WHERE COL=?;
PARSING
SQL
sql_id
execute ?yes
• look up sql_id in shared pool
• if this exists->soft parse
• more complex:
• adaptive cursor sharing
• bind peeking
• adaptive plan execution
• about 60 more reasons for child cursors
PARSING
SQL
sql_id
hard parseexecute ?yes no
• CBO finds plan
• tries multiple variants
• lowest “cost” plan wins
• cost influenced by:
• access paths (indexes?)
• statistics
PARSING
SQL
sql_id
hard parseexecute ?
?
execute
yes
yes
no
• check if CBO plan exists in SPM
• yes and accepted => execute
• no baseline => add this and accept
PARSING
SQL
sql_id
hard parse
add baseline
execute ?
?
execute execute
yes
yes
no
no
• add baseline (unaccepted)
• execute accepted
• unless it is invalid
PARSING
1. enable use of baselines (if they exist)
2. add baselines
• automatic or manual plan capture
3. evolve baselines
4. delete unused baselines
SPM LIFECYCLE
• bind variables
• cannot manage too many sql_ids
• have same objects in all schemas
• SPM works instance-wide
• also be careful when dropping indexes
PREREQUISITES
STEP 1: ENABLE USE OF SPM
• auto capture
• import from SQL Tuning Set
• import from cursor cache
• export/import from other system
STEP 2: CREATE BASELINES
AUTO CAPTURE
• capture on 2nd execution
• auto-accept if this is first baseline
• add as unaccepted if not
• system level vs session level
• with logon trigger
AUTO CAPTURE
• load from STS
• like AWR
DECLARE l_plans_loaded PLS_INTEGER;BEGIN l_plans_loaded := DBMS_SPM.load_plans_from_sqlset( sqlset_name => 'my_sqlset');END;/
http://www.oracle-base.com/articles/11g/sql-plan-management-11gr1.php
MANUAL IMPORT
• load from cursor cache
• filter on sql_text or sql_id
DECLARE l_plans_loaded PLS_INTEGER;BEGIN l_plans_loaded := DBMS_SPM.load_plans_from_cursor_cache( sql_id => '1fkh93md0802n');END;/
http://www.oracle-base.com/articles/11g/sql-plan-management-11gr1.php
MANUAL IMPORT
• manual gives control over single SQL
• ideal to control few “problem”-SQLs
• systemwide auto will also capture “junk”
• ad-hoc and DWH queries
• oracle internal (EM monitoring) stuff
• both add new unaccepted baselines if at least one exists already
MANUAL VS AUTO
• eventually evaluate unaccepted plans
• dbms_spm.evolve_sql_plan_baseline
• this will run statement
• against real data
• with real bind variables
• report execution time
• can choose to accept or just report
STEP 3: EVOLVE BASELINES
• Automatic SQL Tuning Maintenance Job
• job enabled by default
• but implementation of actions disabled
• also suggests indexes etc
EVOLVE JOB IN 11G
EVOLVE JOB IN 11G
EVOLVE JOB IN 11G
• run job to evaluate all new baselines
• do not actually evolve (set accepted flag)
• just create report and mark baseline
• write status in “description” column
• succesful
• failed
• error
MY OWN EVALUATE SCRIPT
• investigate new baselines
• evolve report
• execution plans before and after
• used for documentation
MY REPORT SCRIPT
create or replace PROCEDURE SPM_REPORT( V_SQL_HANDLE IN VARCHAR2)ASresult_evolve clob;BEGINresult_evolve := DBMS_SPM.evolve_sql_plan_baseline(sql_handle => v_sql_handle, plan_name => null, verify => 'YES', commit => 'NO');dbms_output.put_line(result_evolve);
for i in (select plan_table_output from table (dbms_xplan.display_sql_plan_baseline(sql_handle=> v_sql_handle)))loopdbms_output.put_line (i.plan_table_output);end loop;end;
MY REPORT SCRIPT
------------------------------------------------------- Evolve SQL Plan Baseline Report-------------------------------------------------------
Inputs:------- SQL_HANDLE = SQL_fc3b803fd3b27f5d PLAN_NAME = TIME_LIMIT = DBMS_SPM.AUTO_LIMIT VERIFY = YES COMMIT = NO
MY REPORT OUTPUT
Plan: SQL_PLAN_gsfw07z9v4zux1a7a077c------------------------------------ Plan was verified: Time used 5.422 seconds. Plan passedperformance criterion: 989.19 times better than baseline plan.
Baseline Plan Test Plan Stats Ratio ------------- --------- ----------- Execution Status: COMPLETE COMPLETE Rows Processed: 0 0 Elapsed Time(ms): 2671.4 .275 9714.18 CPU Time(ms): 2680 0 Buffer Gets: 100628 102 986.55 Physical Read Requests: 0 0 Physical Write Requests: 0 0 Physical Read Bytes: 0 0 Physical Write Bytes: 0 0 Executions: 1 1
------------------------------------------------------------------- ReportSummary-------------------------------------------------------------------Number of plans verified: 1Number of plans accepted: 0
MY REPORT OUTPUT
SQL handle: SQL_fc3b803fd3b27f5dSQL text: update SOME.TABLE set STATUSID=:1 , laststatuschanged=:2where id in (:3 , :4 , :5 , :6 , :7 , :8 , :9 , :10 , :11 ,:12 , :13 , :14 , :15 , :16 , :17 , :18 , :19 , :20 , :21 ,:22 , :23 , :24 , :25 , :26 , :27 , :28 , :29 , :30 , :31 ,:32 , :33 , :34 , :35 , :36 , :37 , :38 , :39 , :40 , :41 ,:42 )
MY REPORT OUTPUT
--------------------------------------------------------------------------------Plan name: SQL_PLAN_gsfw07z9v4zux077f8913 Plan id: 125798675Enabled: YES Fixed: NO Accepted: YES Origin: AUTO-CAPTURE--------------------------------------------------------------------------------Plan hash value: 125798675-----------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost | Time | Pstart| Pstop |-----------------------------------------------------------------------------------------------| 0 | UPDATE STATEMENT | | 45 | 945 | 137K | 00:27:29 | | || 1 | UPDATE | MYTABLE | | | | | | || 2 | PARTITION RANGE INLIST| | 45 | 945 | 137K | 00:27:29 |KEY(I) |KEY(I) ||* 3 | TABLE ACCESS FULL | MYTABLE | 45 | 945 | 137K | 00:27:29 |KEY(I) |KEY(I) |-----------------------------------------------------------------------------------------------
MY REPORT OUTPUT
--------------------------------------------------------------------------------Plan name: SQL_PLAN_gsfw07z9v4zux1a7a077c Plan id: 444204924Enabled: YES Fixed: NO Accepted: NO Origin: AUTO-CAPTURE--------------------------------------------------------------------------------Plan hash value: 444204924---------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------------| 0 | UPDATE STATEMENT | | 45 | 945 | 48 (0)| 00:00:01 || 1 | UPDATE | MYTABLE | | | | || 2 | INLIST ITERATOR | | | | | ||* 3 | INDEX UNIQUE SCAN| MYTAB_PK | 45 | 945 | 47 (0)| 00:00:01 |---------------------------------------------------------------------------------
MY REPORT OUTPUT
➡ review report (in auto-generated ticket)
➡ request approval for action/change
➡ accept this new baseline
➡ leave comment in description column
EVOLVE PROCESS
create or replace PROCEDURE SPM_EVOLVE( V_SQL_HANDLE IN VARCHAR2, V_DESCRIPTION IN VARCHAR2)ASv_alter_result pls_integer;evolve_out clob;BEGINv_alter_result := DBMS_SPM.ALTER_SQL_PLAN_BASELINE ( sql_handle => V_SQL_HANDLE, attribute_name => 'DESCRIPTION', attribute_value => V_DESCRIPTION);DBMS_OUTPUT.PUT_LINE(v_alter_result);evolve_out := DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE ( sql_handle => v_sql_handle, time_limit => DBMS_SPM.AUTO_LIMIT, verify => ‘YES',
COMMIT => 'YES' );DBMS_OUTPUT.PUT_LINE(evolve_out);end;
MANUAL EVOLVE
• new SYS_AUTO_SPM_EVOLVE_TASK
• “Adaptive SQL Plan Management”
• evaluates and auto-accepts plans
• EVOLVE_SQL_PLAN_BASELINE deprecated
• new API: EVOLVE_TASKS
http://www.oracle.com/technetwork/database/bi-datawarehousing/twp-sql-plan-mgmt-12c-1963237.pdf
http://chandlerdba.wordpress.com/2013/12/08/sql-plan-management-12c-dumb-feature/
SPM IN 12C
• delete failed baselines after a while
• to give them chance for re-evaluation
CLEANUP
• new baselines captured automatically
• but gain stability for existing plans
• potentially better plans evaluated automatically
• experience shows only few better plans/week
• ticket is automatically created each time evolve reports succeeds
• DBA reviews this, requests approval and evolve
PROCESS REVIEW
SELECT * FROM TABLE(dbms_xplan.display_cursor('ahmckrb9zn6cn'));
PLAN_TABLE_OUTPUTSQL_ID ahmckrb9zn6cn, child number 0-------------------------------------
PLAN_TABLE_OUTPUT| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|--------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 13 (100)|| 1 | NESTED LOOPS OUTER | | 4 | 264 | 13 (8)||* 2 | HASH JOIN | | 4 | 188 | 7 (15)|| 3 | PARTITION RANGE SINGLE | | 4 | 156 | 3 (0)||* 4 | TABLE ACCESS BY LOCAL INDEX ROWID| TABLE_MONTH | 4 | 156 | 3 (0)||* 5 | INDEX SKIP SCAN | PK_TAB_MONTH | 116K| | 3 (0)|| 6 | TABLE ACCESS FULL | SOMEOTHERTABLE | 503 | 4024 | 3 (0)||* 7 | INDEX RANGE SCAN | SOME_IDX | 1 | 19 | 2 (0)|--------------------------------------------------------------------------------------------
EXAMPLE: OVERRIDE STUPID HINTS
CREATE OR REPLACE VIEW "SOME"."VIEW" ("COL1","COL2","COL3") AS SELECT /*+ use_hash(c, tc) */[...]FROM SOME.TAB1_monat cJOIN SOME.TAB2 tcONtc.col42 = c.col23LEFT JOIN SOME.TAB3 vpONvp.col1 = c.col2
EXAMPLE: OVERRIDE STUPID HINTS
• SQL using some View with HASH JOIN hint
• but this SQL did not perform well with it
• NL would have been much better
• cannot replace view without looking at other SQL
• create new view without hint?
• my solution: set ignore_hints in session
• execute/generate plan and evolve to only accepted
EXAMPLE: OVERRIDE STUPID HINTS
alter session set "_optimizer_ignore_hints" = TRUE;
EXAMPLE: OVERRIDE STUPID HINTS
• export current RBO plans from 9i
• with outlines
• or import with RBO and compatible=9
• import into 11g or 12c database
• this guarantees compatibility
• but new CBO plans will get added
• can slowly review, accept, improve one-by-one
https://blogs.oracle.com/optimizer/entry/upgrading_from_9i_to_11g_and_the_implicit_migration_from_rbo
EXAMPLE: UPGRADE FROM BRO
• DBA_SQL_PLAN_BASELINES
• SQL_HANDLE != SQL_ID
• join via SIGNATURE
• PLAN_NAME != PLAN_HASH_VALUE
• plan is not stored in 11g (and may not reproduce)
• doesn’t help with parsing problems
• e.g. 12c dynamic stats
SOME ANNOYING THINGS
• export prod baselines to test/staging
• find new SQL in dev systems
• import those baselines before SQL goes to production
NEXT GEN IDEAS
• SPM is a very powerful tool
• easy to get started
• flexible to adopt to your needs
SUMMARY
thanks
Björn Rost