1. Adaptive Query Optimization Oracle OpenWorld, 28 September 2014 Christian Antognini BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA 2014 Trivadis Adaptive Query Optimization 28 September 2014 1

2. 2014 Trivadis Adaptive Query Optimization 28 September 2014 2 @ChrisAntognini Senior principal consultant, trainer and partner at Trivadis in Zurich (CH) christian.antognini@trivadis.com http://antognini.ch Focus: get the most out of Oracle Database Logical and physical database design Query optimizer Application performance management Author of Troubleshooting Oracle Performance (Apress, 2008/2014) OakTable Network, Oracle ACE Director

3. Over the years Oracle has extremely improved the capabilities of the query 2014 Trivadis optimizer Most of the improvements fall into one of the following areas Enhance the quality of the inputs (e.g. objects statistics) Make the gathering and management of object statistics easier and more efficient Use new data structures Implement or enhance query transformations Improve plan stability Cope with poor estimations that leads to poor execution plans 12c is no exception, every one of these areas were improved Adaptive Query Optimization 28 September 2014 3 Introduction

4. Adaptive plans (Enterprise Edition only) Join methods Parallel distribution methods Star transformation Adaptive statistics SQL plan directives Automatic reoptimization - Statistics feedback (evolution of cardinality feedback) - Performance feedback Dynamic statistics (evolution of dynamic sampling) 2014 Trivadis Adaptive Query Optimization 28 September 2014 4 Adaptive Query Optimization Features

5. Enables or disables adaptive query optimization features Adaptive plans SQL plan directives Automatic reoptimization - It isnt the case in 12.1.0.1 (bug 16824474) The default value is TRUE OPTIMIZER_DYNAMIC_SAMPLING controls dynamic statistics 2014 Trivadis Adaptive Query Optimization 28 September 2014 5 OPTIMIZER_ADAPTIVE_FEATURES

6. Enables or disables reporting mode for adaptive query optimization features Useful to assess how an execution plan would change Use DBMS_XPLAN to get detail information about the analysis Might fail with an ORA-1001 (bug 17270605) The default value is FALSE 2014 Trivadis Adaptive Query Optimization 28 September 2014 6 OPTIMIZER_ADAPTIVE_REPORTING_ONLY SELECT * FROM table(dbms_xplan.display_cursor(format=>'report'))

7. Object statistics dont always provide sufficient information to find an optimal execution plan To get additional insights, the query optimizer can take advantage of features like dynamic sampling and cardinality feedback Dont solve all issues, though 2014 Trivadis Adaptive Query Optimization 28 September 2014 7 Adaptive Plans Challenge

8. As of 12c, the query optimizer can postpone some decisions until the execution 2014 Trivadis phase The idea is to leverage information collected while executing part of an execution plan to determine how another part should be carried out The query optimizer uses adaptive plans in three situations: To switch the join method from a nested loops join to a hash join and vice versa To switch the PX distribution method from hash to broadcast/round-robin To disable the access to a dimension for execution plans using the star transformation Adaptive Query Optimization 28 September 2014 8 Adaptive Plans Concept

9. The query optimizer adds subplans to execution plans One represents the nested loops join, the other the hash join One of the alternatives is the default plan One of the subplans is chosen during the first execution The choice is based on the number of rows actually processed The query optimizer computes an inflection point A new row source operation is used to partially buffer and count the rows STATISTICS COLLECTOR Limitation: if a too large buffer is required, no adaptive plan is used The execution plan that is actually executed is called the final plan 2014 Trivadis Adaptive Query Optimization 28 September 2014 9 Adaptive Plans Join Method Switch

10. Adaptive Plans Join Method Switch Example 2014 Trivadis Adaptive Query Optimization 28 September 2014 10 SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666 ----------------------------------------------- | Id | Operation | Name | ----------------------------------------------- | 0 | SELECT STATEMENT | | | 1 | HASH JOIN | | | 2 | NESTED LOOPS | | | 3 | NESTED LOOPS | | | 4 | STATISTICS COLLECTOR | | | 5 | TABLE ACCESS FULL | T1 | | 6 | INDEX UNIQUE SCAN | T2_PK | | 7 | TABLE ACCESS BY INDEX ROWID| T2 | | 8 | TABLE ACCESS FULL | T2 | ----------------------------------------------- adaptive_plan.sql

11. 2014 Trivadis Adaptive Query Optimization 28 September 2014 11 Adaptive Plans Join Method Switch Example SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666 ----------------------------------------------- | Id | Operation | Name | ----------------------------------------------- | 0 | SELECT STATEMENT | | | 1 | HASH JOIN | | | 2 | NESTED LOOPS | | | 3 | NESTED LOOPS | | | 4 | STATISTICS COLLECTOR | | | 5 | TABLE ACCESS FULL | T1 | | 6 | INDEX UNIQUE SCAN | T2_PK | | 7 | TABLE ACCESS BY INDEX ROWID| T2 | | 8 | TABLE ACCESS FULL | T2 | ----------------------------------------------- adaptive_plan.sql

12. Adaptive Plans Join Method Switch Example 2014 Trivadis Adaptive Query Optimization 28 September 2014 12 SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666 ----------------------------------------------- | Id | Operation | Name | ----------------------------------------------- | 0 | SELECT STATEMENT | | | 1 | HASH JOIN | | | 2 | NESTED LOOPS | | | 3 | NESTED LOOPS | | | 4 | STATISTICS COLLECTOR | | | 5 | TABLE ACCESS FULL | T1 | | 6 | INDEX UNIQUE SCAN | T2_PK | | 7 | TABLE ACCESS BY INDEX ROWID| T2 | | 8 | TABLE ACCESS FULL | T2 | ----------------------------------------------- adaptive_plan.sql

13. Adaptive Plans Join Method Switch Inflection Point For both join methods, the cost associated to different cardinalities is estimated The cardinality of the outer table varies The cardinality of the inner table remains fixed The query optimizer uses a binary search The search takes place between a minimum and maximum cardinality 2014 Trivadis Adaptive Query Optimization 28 September 2014 13

14. Adaptive Plans Join Method Switch Inflection Point Example 2014 Trivadis Adaptive Query Optimization 28 September 2014 14 20,000 2,000 200 20 Cost Cardinality (In Order of Execution) Nested Loops Join Hash Join

15. Adaptive Plans Parallel Distribution Method New distribution method HYBRID HASH It helps avoiding some data skewing problems The actual distribution method is chosen at execution time STATISTICS COLLECTOR The decision takes place during each execution Number of rows < 2 * DOP Left input: BROADCAST / Right input: ROUND-ROBIN Number of rows 2 * DOP Left input: HASH / Right input: HASH 2014 Trivadis Adaptive Query Optimization 28 September 2014 15

16. Adaptive Plans Parallel Distribution Method Example 2014 Trivadis Adaptive Query Optimization 28 September 2014 16 hybrid_hash.sql ------------------------------------------------------------------------ | Operation | Name | TQ |IN-OUT| PQ Distrib | ------------------------------------------------------------------------ | SELECT STATEMENT | | | | | | SORT AGGREGATE | | | | | | PX COORDINATOR | | | | | | PX SEND QC (RANDOM) | :TQ10002 | Q1,02 | P->S | QC (RAND) | | SORT AGGREGATE | | Q1,02 | PCWP | | | HASH JOIN | | Q1,02 | PCWP | | | PX RECEIVE | | Q1,02 | PCWP | | | PX SEND HYBRID HASH | :TQ10000 | Q1,00 | P->P | HYBRID HASH| | STATISTICS COLLECTOR | | Q1,00 | PCWC | | | PX BLOCK ITERATOR | | Q1,00 | PCWC | | | TABLE ACCESS FULL | T1 | Q1,00 | PCWP | | | PX RECEIVE | | Q1,02 | PCWP | | | PX SEND HYBRID HASH | :TQ10001 | Q1,01 | P->P | HYBRID HASH| | PX BLOCK ITERATOR | | Q1,01 | PCWC | | | TABLE ACCESS FULL | T2 | Q1,01 | PCWP | | ------------------------------------------------------------------------

17. Adaptive Plans Star Transformation With the star transformation, the data of each dimension that has a restriction applied to it might be joined to the corresponding bitmap index of the fact If the number of rowids returned by such a join is underestimated, applying the filter can be detrimental to the performance With an adaptive plan the access to some dimensions can be disabled Decision takes place during the first execution only 2014 Trivadis Adaptive Query Optimization 28 September 2014 17

18. Adaptive Plans Star Transformation Example 2014 Trivadis Adaptive Query Optimization 28 September 2014 18 ----------------------------------------------------- | Operation | Name | ----------------------------------------------------- | | | | VIEW | VW_ST_5497B905 | | NESTED LOOPS | | | BITMAP CONVERSION TO ROWIDS | | | BITMAP AND | | | BITMAP MERGE | | | BITMAP KEY ITERATION | | | TABLE ACCESS FULL | COLORS | | BITMAP INDEX RANGE SCAN | CAR_COLOR_IDX | | STATISTICS COLLECTOR | | | BITMAP MERGE | | | BITMAP KEY ITERATION | | | TABLE ACCESS FULL | MODELS | | BITMAP INDEX RANGE SCAN| CAR_MODEL_IDX | | | | | TABLE ACCESS BY USER ROWID | CARS | -----------------------------------------------------

19. Adaptive Plans V$SQL.IS_RESOLVED_ADAPTIVE_PLAN NULL means that the execution plan associated to the cursor isnt adaptive N means that the final execution plan hasnt been determined Y means that the final execution plan was determined Also set if reporting mode is enabled Set for join method switches and star transformation only 2014 Trivadis Adaptive Query Optimization 28 September 2014 19

20. Adaptive Statistics SQL Plan Directives SQL plan directives are automatically created when misestimates occur They are temporarily stored in the SGA and flushed to disk every 15 min by a background process They arent tied to a specific SQL statement, but to specific tables/columns Several of them can be used for a single SQL statement They instruct the database engine to automatically create extended statistics Only column groups are considered The limit of 20 extensions applies to them as well If creating extended statistics isnt supported/possible/sensible, they instruct the query optimizer to use dynamic sampling 2014 Trivadis Adaptive Query Optimization 28 September 2014 20

21. Adaptive Statistics Management of SQL Plan Directives The database engine automatically maintains SQL plan directives They are automatically created They are automatically purged if not used (by default after 53 weeks) They can be manually managed through DBMS_SPD Flush to disk SQL plan directives temporarily stored in the SGA Change retention period Immediate purge Alter attributes (ENABLED, AUTO_DROP) Pack/Unpack Requirement: ADMINISTER SQL MANAGEMENT OBJECT privilege 2014 Trivadis Adaptive Query Optimization 28 September 2014 21

22. Adaptive Statistics SQL Plan Directives in the Data Dictionary DBA_SQL_PLAN_DIRECTIVES DBA_SQL_PLAN_DIR_OBJECTS 2014 Trivadis Adaptive Query Optimization 28 September 2014 22 SQL> SELECT type, reason, count(*) 2 FROM dba_sql_plan_directives 3 GROUP BY type, reason; TYPE REASON COUNT(*) ---------------- ------------------------------------ ---------- DYNAMIC_SAMPLING SINGLE TABLE CARDINALITY MISESTIMATE 81 DYNAMIC_SAMPLING JOIN CARDINALITY MISESTIMATE 180 DYNAMIC_SAMPLING GROUP BY CARDINALITY MISESTIMATE 6

23. Adaptive Statistics Statistics Feedback Evolution of cardinality feedback Used for single-table cardinalities as well as join cardinalities Information about misestimates might be persisted through SQL plan directives For misestimates due to table functions no information is stored V$SQL.IS_REOPTIMIZABLE Y means that the next execution will trigger a reoptimization N means that no reoptimization is necessary R means that reoptimization information is available, but reporting mode was enabled 2014 Trivadis Adaptive Query Optimization 28 September 2014 23 statistics_feedback1.sql statistics_feedback2.sql

24. Adaptive Statistics Performance Feedback Enabled when PARALLEL_DEGREE_POLICY = ADAPTIVE Assessment of the auto DOP after the first execution If auto DOP is suboptimal, the next execution will trigger a reoptimization 2014 Trivadis Adaptive Query Optimization 28 September 2014 24

25. Adaptive Statistics Dynamic Statistics Evolution of dynamic sampling Used for single-table cardinalities as well as join and group-by cardinalities The query optimizer decides when and how to use dynamic statistics The data collected through dynamic statistics is shared through the result cache The sampling cant exceed a given amount of time The maximum is determined based on past executions (cursor cache, AWR) To use the new features OPTIMIZER_DYNAMIC_SAMPLING has to be set to 11 2014 Trivadis Adaptive Query Optimization 28 September 2014 25 dynamic_statistics.sql

26. Summary 2014 Trivadis Adaptive query optimization introduced a number of very interesting features The query optimizer is getting more and more dynamic Adaptive Query Optimization 28 September 2014 26

27. Questions and answers ... Christian Antognini Senior Principal Consultant christian.antognini@trivadis.com BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA 2014 2013 Trivadis Adaptive Query Optimization 28 September 2014 27