Adaptive Query Optimization
-
Upload
christian-antognini -
Category
Technology
-
view
883 -
download
6
description
Transcript of Adaptive Query Optimization
2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
2014 © Trivadis
Adaptive Query OptimizationOracle OpenWorld, 28 September 2014
Christian Antognini
28 September 2014Adaptive Query Optimization
1
2014 © TrivadisAdaptive Query Optimization28 September 2014
2
@ChrisAntognini
Senior principal consultant, trainer and partnerat Trivadis in Zurich (CH) [email protected] http://antognini.ch
Focus: get the most out of Oracle Database Logical and physical database design Query optimizer Application performance management
Author of Troubleshooting Oracle Performance (Apress, 2008/2014) OakTable Network, Oracle ACE Director
2014 © TrivadisAdaptive Query Optimization
Over the years Oracle has extremely improved the capabilities of the query optimizer
Most of the improvements fall into one of the following areas Enhance the quality of the inputs (e.g. objects statistics) Make the gathering and management of object statistics easier and more efficient Use new data structures Implement or enhance query transformations Improve plan stability Cope with poor estimations that leads to poor execution plans
12c is no exception, every one of these areas were improved
28 September 2014
3
Introduction
2014 © Trivadis
Adaptive plans (Enterprise Edition only) Join methods Parallel distribution methods Star transformation
Adaptive statistics SQL plan directives Automatic reoptimization
- Statistics feedback (evolution of cardinality feedback)- Performance feedback
Dynamic statistics (evolution of dynamic sampling)
28 September 2014Adaptive Query Optimization
4
Adaptive Query Optimization Features
2014 © TrivadisAdaptive Query Optimization
Enables or disables adaptive query optimization features Adaptive plans SQL plan directives Automatic reoptimization
- It isn’t the case in 12.1.0.1 (bug 16824474)
The default value is TRUE
OPTIMIZER_DYNAMIC_SAMPLING controls dynamic statistics
28 September 2014
5
OPTIMIZER_ADAPTIVE_FEATURES
2014 © TrivadisAdaptive Query Optimization
Enables or disables reporting mode for adaptive query optimization features
Useful to assess how an execution plan would change
Use DBMS_XPLAN to get detail information about the analysis Might fail with an ORA-1001 (bug 17270605)
The default value is FALSE
28 September 2014
6
OPTIMIZER_ADAPTIVE_REPORTING_ONLY
SELECT * FROM table(dbms_xplan.display_cursor(format=>'report'))
2014 © TrivadisAdaptive Query Optimization
Object statistics don’t always provide sufficient information to find an optimal execution plan
To get additional insights, the query optimizer can take advantage of features like dynamic sampling and cardinality feedback Don’t solve all issues, though
28 September 2014
7
Adaptive Plans – Challenge
2014 © TrivadisAdaptive Query Optimization
As of 12c, the query optimizer can postpone some decisions until the execution phase
The idea is to leverage information collected while executing part of an execution plan to determine how another part should be carried out
The query optimizer uses adaptive plans in three situations: To switch the join method from a nested loops join to a hash join and vice versa To switch the PX distribution method from hash to broadcast/round-robin To disable the access to a dimension for execution plans using the star transformation
28 September 2014
8
Adaptive Plans – Concept
2014 © TrivadisAdaptive Query Optimization
The query optimizer adds subplans to execution plans One represents the nested loops join, the other the hash join One of the alternatives is the default plan
One of the subplans is chosen during the first execution The choice is based on the number of rows actually processed The query optimizer computes an inflection point
A new row source operation is used to partially buffer and count the rows STATISTICS COLLECTOR Limitation: if a too large buffer is required, no adaptive plan is used
The execution plan that is actually executed is called the final plan
28 September 2014
9
Adaptive Plans – Join Method Switch
2014 © TrivadisAdaptive Query Optimization
1028 September 2014
Adaptive Plans – Join Method Switch Example
SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666
-----------------------------------------------| Id | Operation | Name |-----------------------------------------------| 0 | SELECT STATEMENT | || 1 | HASH JOIN | || 2 | NESTED LOOPS | || 3 | NESTED LOOPS | || 4 | STATISTICS COLLECTOR | || 5 | TABLE ACCESS FULL | T1 || 6 | INDEX UNIQUE SCAN | T2_PK || 7 | TABLE ACCESS BY INDEX ROWID| T2 || 8 | TABLE ACCESS FULL | T2 |-----------------------------------------------
adaptive_plan.sql
2014 © TrivadisAdaptive Query Optimization
1128 September 2014
Adaptive Plans – Join Method Switch Example
SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666
-----------------------------------------------| Id | Operation | Name |-----------------------------------------------| 0 | SELECT STATEMENT | || 1 | HASH JOIN | || 2 | NESTED LOOPS | || 3 | NESTED LOOPS | || 4 | STATISTICS COLLECTOR | || 5 | TABLE ACCESS FULL | T1 || 6 | INDEX UNIQUE SCAN | T2_PK || 7 | TABLE ACCESS BY INDEX ROWID| T2 || 8 | TABLE ACCESS FULL | T2 |-----------------------------------------------
adaptive_plan.sql
2014 © TrivadisAdaptive Query Optimization
1228 September 2014
Adaptive Plans – Join Method Switch Example
SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666
-----------------------------------------------| Id | Operation | Name |-----------------------------------------------| 0 | SELECT STATEMENT | || 1 | HASH JOIN | || 2 | NESTED LOOPS | || 3 | NESTED LOOPS | || 4 | STATISTICS COLLECTOR | || 5 | TABLE ACCESS FULL | T1 || 6 | INDEX UNIQUE SCAN | T2_PK || 7 | TABLE ACCESS BY INDEX ROWID| T2 || 8 | TABLE ACCESS FULL | T2 |-----------------------------------------------
adaptive_plan.sql
2014 © Trivadis
For both join methods, the cost associated to different cardinalities is estimated The cardinality of the outer table varies The cardinality of the inner table remains fixed
The query optimizer uses a binary search
The search takes place between a minimum and maximum cardinality
28 September 2014Adaptive Query Optimization
13
Adaptive Plans – Join Method Switch Inflection Point
2014 © Trivadis
28 September 2014Adaptive Query Optimization
14
Adaptive Plans – Join Method Switch Inflection Point Example
1
19239.93
9620.46
4810.73
2405.87
1203.43
602.22
301.61
151.3
76.15
38.58
57.36
47.97
52.67
50.32
49.14
48.56
20
200
2,000
20,000
Nested Loops Join
Hash Join
Cardinality(In Order of Execution)
Co
st
2014 © Trivadis
New distribution method HYBRID HASH It helps avoiding some data skewing problems
The actual distribution method is chosen at execution time STATISTICS COLLECTOR The decision takes place during each execution
Number of rows < 2 * DOP Left input: BROADCAST / Right input: ROUND-ROBIN
Number of rows ≥ 2 * DOP Left input: HASH / Right input: HASH
28 September 2014Adaptive Query Optimization
15
Adaptive Plans – Parallel Distribution Method
2014 © Trivadis
28 September 2014Adaptive Query Optimization
16
Adaptive Plans – Parallel Distribution Method Example
------------------------------------------------------------------------| Operation | Name | TQ |IN-OUT| PQ Distrib |------------------------------------------------------------------------| SELECT STATEMENT | | | | || SORT AGGREGATE | | | | || PX COORDINATOR | | | | || PX SEND QC (RANDOM) | :TQ10002 | Q1,02 | P->S | QC (RAND) || SORT AGGREGATE | | Q1,02 | PCWP | || HASH JOIN | | Q1,02 | PCWP | || PX RECEIVE | | Q1,02 | PCWP | || PX SEND HYBRID HASH | :TQ10000 | Q1,00 | P->P | HYBRID HASH|| STATISTICS COLLECTOR | | Q1,00 | PCWC | || PX BLOCK ITERATOR | | Q1,00 | PCWC | || TABLE ACCESS FULL | T1 | Q1,00 | PCWP | || PX RECEIVE | | Q1,02 | PCWP | || PX SEND HYBRID HASH | :TQ10001 | Q1,01 | P->P | HYBRID HASH|| PX BLOCK ITERATOR | | Q1,01 | PCWC | || TABLE ACCESS FULL | T2 | Q1,01 | PCWP | |------------------------------------------------------------------------
hybrid_hash.sql
2014 © Trivadis
With the star transformation, the data of each dimension that has a restriction applied to it might be “joined” to the corresponding bitmap index of the fact
If the number of rowids returned by such a “join” is underestimated, applying the filter can be detrimental to the performance
With an adaptive plan the access to some dimensions can be disabled Decision takes place during the first execution only
28 September 2014Adaptive Query Optimization
17
Adaptive Plans – Star Transformation
2014 © Trivadis
28 September 2014Adaptive Query Optimization
18
Adaptive Plans – Star Transformation Example
-----------------------------------------------------| Operation | Name |-----------------------------------------------------| … | || VIEW | VW_ST_5497B905 || NESTED LOOPS | || BITMAP CONVERSION TO ROWIDS | || BITMAP AND | || BITMAP MERGE | || BITMAP KEY ITERATION | || TABLE ACCESS FULL | COLORS || BITMAP INDEX RANGE SCAN | CAR_COLOR_IDX || STATISTICS COLLECTOR | || BITMAP MERGE | || BITMAP KEY ITERATION | || TABLE ACCESS FULL | MODELS || BITMAP INDEX RANGE SCAN| CAR_MODEL_IDX || … | || TABLE ACCESS BY USER ROWID | CARS |-----------------------------------------------------
2014 © Trivadis
NULL means that the execution plan associated to the cursor isn’t adaptive
N means that the final execution plan hasn’t been determined
Y means that the final execution plan was determined Also set if reporting mode is enabled
Set for join method switches and star transformation only
28 September 2014Adaptive Query Optimization
19
Adaptive Plans – V$SQL.IS_RESOLVED_ADAPTIVE_PLAN
2014 © TrivadisAdaptive Query Optimization
SQL plan directives are automatically created when misestimates occur They are temporarily stored in the SGA and flushed to disk every 15 min by a
background process
They aren’t tied to a specific SQL statement, but to specific tables/columns Several of them can be used for a single SQL statement
They instruct the database engine to automatically create extended statistics Only column groups are considered The limit of 20 extensions applies to them as well If creating extended statistics isn’t supported/possible/sensible, they instruct the query
optimizer to use dynamic sampling
28 September 2014
20
Adaptive Statistics – SQL Plan Directives
2014 © Trivadis
The database engine automatically maintains SQL plan directives They are automatically created They are automatically purged if not used (by default after 53 weeks)
They can be manually managed through DBMS_SPD Flush to disk SQL plan directives temporarily stored in the SGA Change retention period Immediate purge Alter attributes (ENABLED, AUTO_DROP) Pack/Unpack Requirement: ADMINISTER SQL MANAGEMENT OBJECT privilege
28 September 2014Adaptive Query Optimization
21
Adaptive Statistics – Management of SQL Plan Directives
2014 © Trivadis
DBA_SQL_PLAN_DIRECTIVES
DBA_SQL_PLAN_DIR_OBJECTS
28 September 2014Adaptive Query Optimization
22
Adaptive Statistics – SQL Plan Directives in the Data Dictionary
SQL> SELECT type, reason, count(*) 2 FROM dba_sql_plan_directives 3 GROUP BY type, reason;
TYPE REASON COUNT(*)---------------- ------------------------------------ ----------DYNAMIC_SAMPLING SINGLE TABLE CARDINALITY MISESTIMATE 81DYNAMIC_SAMPLING JOIN CARDINALITY MISESTIMATE 180DYNAMIC_SAMPLING GROUP BY CARDINALITY MISESTIMATE 6
2014 © TrivadisAdaptive Query Optimization
Evolution of cardinality feedback
Used for single-table cardinalities as well as join cardinalities
Information about misestimates might be persisted through SQL plan directives For misestimates due to table functions no information is stored
V$SQL.IS_REOPTIMIZABLE Y means that the next execution will trigger a reoptimization N means that no reoptimization is necessary R means that reoptimization information is available, but reporting mode was enabled
28 September 2014
23
Adaptive Statistics – Statistics Feedbackstatistics_feedback1.sqlstatistics_feedback2.sql
2014 © TrivadisAdaptive Query Optimization
Enabled when PARALLEL_DEGREE_POLICY = ADAPTIVE
Assessment of the auto DOP after the first execution
If auto DOP is suboptimal, the next execution will trigger a reoptimization
28 September 2014
24
Adaptive Statistics – Performance Feedback
2014 © TrivadisAdaptive Query Optimization
Evolution of dynamic sampling
Used for single-table cardinalities as well as join and group-by cardinalities
The query optimizer decides when and how to use dynamic statistics
The data collected through dynamic statistics is shared through the result cache
The sampling can’t exceed a given amount of time The maximum is determined based on past executions (cursor cache, AWR)
To use the new features OPTIMIZER_DYNAMIC_SAMPLING has to be set to 11
28 September 2014
25
Adaptive Statistics – Dynamic Statisticsdynamic_statistics.sql
2014 © Trivadis
Text Text Adaptive query optimization introduced a number of very interesting features
The query optimizer is getting more and more dynamic
28 September 2014Adaptive Query Optimization
26
Summary
2014 © Trivadis
Questions and answers ...
2013 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
Christian Antognini
Senior Principal Consultant
28 September 2014Adaptive Query Optimization
27