Dsi404 Query Optimizer

7/27/2019 Dsi404 Query Optimizer

1/503

DSI404e: QueryOptimization

eStudy PDF version

D13284GC10

Edition 1.0

August 2003

Not orderable


2/503

Copyright 2003, Oracle. All rights reserved.

This documentation contains proprietary information of Oracle Corporation. It is provided under a license agreementcontaining restrictions on use and disclosure and is also protected by copyright law. Reverse engineering of thesoftware is prohibited. If this documentation is delivered to a U.S. Government Agency of the Department of Defense,then it is delivered with Restricted Rights and the following legend is applicable:

Restricted Rights Legend

Use, duplication or disclosure by the Government is subject to restrictions for commercial computer software and shallbe deemed to be Restricted Rights software under Federal law, as set forth in subparagraph (c)(1)(ii) of DFARS252.227-7013, Rights in Technical Data and Computer Software (October 1988).

This material or any portion of it may not be copied in any form or by any means without the express prior writtenpermission of the Education Products group of Oracle Corporation. Any other copying is a violation of copyright law andmay result in civil and/or criminal penalties.

If this documentation is delivered to a U.S. Government Agency not within the Department of Defense, then it isdelivered with Restricted Rights, as defined in FAR 52.227-14, Rights in Data-General, including Alternate III (June1987).

The information in this document is subject to change without notice. If you find any problems in the documentation,please report them in writing to Worldwide Education Services, Oracle Corporation, 500 Oracle Parkway, Box SB-6,Redwood Shores, CA 94065. Oracle Corporation does not warrant that this document is error-free.

Oracle and all references to Oracle Products are trademarks or registered trademarks of Oracle Corporation.

All other products or company names are used for identification purposes only, and may be trademarks of theirrespective owners.

Authors

Lex de HaanAndrew HambeltonAlex MelidisJames CremoniniSteve DixonMuriel Fry

Andreas Walter

Technical Contributors and Reviewers

Christine JealJose Marco-DominguezRoderick ManalacPrabhaker GongloorSijmen de RooHermann BaerPatrick Amor

Hakan JakobssonRafi AhmedStephen VivianXuan-Cuong Bui

This book was published using: oracletutor


3/503


DSI404e Query Optimization Table of Contentsi

Table of Contents

Introduction..................................................................................................................................................... 1-1Introduction...................................................................................................................................................1-2Objectives .....................................................................................................................................................1-3

Relational Theory..........................................................................................................................................1-4Translation of Relational Terms.................................................................................................................... 1-6Introduction to Query Optimization..............................................................................................................1-7The Optimizer ............................................................................................................................................... 1-8Query Optimization....................................................................................................................................... 1-9Rule-Based Optimization (RBO).................................................................................................................. 1-10RBO Query Tuning.......................................................................................................................................1-12Cost-Based Optimization (CBO) .................................................................................................................. 1-14Optimizer Steps.............................................................................................................................................1-15Cardinality.....................................................................................................................................................1-17Selectivity .....................................................................................................................................................1-18Transitivity....................................................................................................................................................1-20Transitivity Example..................................................................................................................................... 1-21Summary.......................................................................................................................................................1-22References..................................................................................................................................................... 1-23

Generic Optimizer Concepts.......................................................................................................................... 2-1Generic Optimizer Concepts .........................................................................................................................2-2Objectives .....................................................................................................................................................2-3Optimizer Modes ..........................................................................................................................................2-4Row Sources ................................................................................................................................................. 2-5Table Access Operations............................................................................................................................... 2-6Index Access Operations............................................................................................................................... 2-7Join Terminology.......................................................................................................................................... 2-9Join Types..................................................................................................................................................... 2-10Join Methods.................................................................................................................................................2-13Antijoins and Semijoins ................................................................................................................................ 2-15

Star Joins.......................................................................................................................................................2-17Summary.......................................................................................................................................................2-18References..................................................................................................................................................... 2-19

SQL Processing Overview .............................................................................................................................. 3-1SQL Processing Overview............................................................................................................................3-2Objectives .....................................................................................................................................................3-3Cursors in Oracle ..........................................................................................................................................3-4Shared Cursor Information............................................................................................................................3-5Sharing of Cursors ........................................................................................................................................3-6Soft Parsing...................................................................................................................................................3-7Hard Parsing..................................................................................................................................................3-9Library Cache Lookup .................................................................................................................................. 3-10Library Cache Function ................................................................................................................................ 3-12

Importance of Bind Variables .......................................................................................................................3-14Cursor Sharing .............................................................................................................................................. 3-15Lifetime of a SQL Cursor ............................................................................................................................. 3-19Overview of SQL Execution......................................................................................................................... 3-21Step 1: Open Cursor...................................................................................................................................... 3-22Step 2: Parse Statement................................................................................................................................. 3-24Steps 3 and 4: Describe and Define .............................................................................................................. 3-26Steps 5 and 6: Bind and Parallelize............................................................................................................... 3-27Steps 7, 8, and 9: Execute, Fetch, and Close................................................................................................. 3-28


4/503


DSI404e Query Optimization Table of Contentsii

Four Application Types ................................................................................................................................ 3-29Type 1 ........................................................................................................................................................... 3-30Type 2 ........................................................................................................................................................... 3-31Type 3 ........................................................................................................................................................... 3-32Type 4 ........................................................................................................................................................... 3-33Row Level Security.......................................................................................................................................3-35Query Rewrite............................................................................................................................................... 3-37Summary.......................................................................................................................................................3-38

Query Transformations..................................................................................................................................4-1Query Transformations ................................................................................................................................. 4-2Objectives .....................................................................................................................................................4-3Query Processing ..........................................................................................................................................4-4Subquery Unnesting......................................................................................................................................4-6Subquery Unnesting Example.......................................................................................................................4-7When Is Subquery Unnesting Performed?....................................................................................................4-9Subqueries That Can Be Unnested................................................................................................................ 4-10Unnesting Restrictions.................................................................................................................................. 4-13Unnesting: Examples ....................................................................................................................................4-15View Merging ............................................................................................................................................... 4-21View Merging: Example............................................................................................................................... 4-22View Merging Restrictions...........................................................................................................................4-24Nonmergeable View Examples: ROWNUM Reference ............................................................................... 4-25Nonmergeable View Examples: Outer Join .................................................................................................. 4-26Complex View Merging................................................................................................................................ 4-27Pushing Predicates ........................................................................................................................................ 4-29IN-List and OR Evaluation ........................................................................................................................... 4-31IN-List Iterator .............................................................................................................................................. 4-33IN-List Expansion......................................................................................................................................... 4-35IN-List and OR as a Filter............................................................................................................................. 4-37Star Transformation ......................................................................................................................................4-38Star Transformation Using Temporary Tables.............................................................................................. 4-42Restrictions on Star Transformation ............................................................................................................. 4-43Optimizer Predicates..................................................................................................................................... 4-45Event 10060 ..................................................................................................................................................4-47Summary.......................................................................................................................................................4-49References..................................................................................................................................................... 4-50

Statistics ........................................................................................................................................................... 5-1Statistics ........................................................................................................................................................5-2Objectives .....................................................................................................................................................5-3Collecting Statistics....................................................................................................................................... 5-4Table Statistics .............................................................................................................................................. 5-6Index Statistics .............................................................................................................................................. 5-8Column Statistics ..........................................................................................................................................5-10Default Statistics ........................................................................................................................................... 5-11Histograms ....................................................................................................................................................5-12Histograms: Example.................................................................................................................................... 5-14# Buckets = # Distinct Values.......................................................................................................................5-15# Buckets < # Distinct Values.......................................................................................................................5-17Gather Histogram Statistics........................................................................................................................... 5-20When to Use Histograms .............................................................................................................................. 5-22Histogram Properties..................................................................................................................................... 5-23Histograms and Selectivity ...........................................................................................................................5-24Dynamic Sampling........................................................................................................................................5-26System Statistics ........................................................................................................................................... 5-29System Statistics: Overview..........................................................................................................................5-31System Statistics: AUX_STATS$.................................................................................................................5-34


5/503


DSI404e Query Optimization Table of Contentsiii

System Statistics: _optimizer_cost_model....................................................................................................5-37DBMS_STATS Package...............................................................................................................................5-40DBMS_STATS: Gathering Statistics............................................................................................................5-41Automated Statistics Gathering.....................................................................................................................5-43Global Statistics ............................................................................................................................................ 5-45Copy Statistics Between Databases .............................................................................................................. 5-46Copying Statistics: Example .........................................................................................................................5-47Gathering Statistics: Example .......................................................................................................................5-48Setting Column Statistics: Example.............................................................................................................. 5-49Summary.......................................................................................................................................................5-50

Execution Plans ............................................................................................................................................... 6-1Execution Plans............................................................................................................................................. 6-2Objectives .....................................................................................................................................................6-3Tuning Methodology..................................................................................................................................... 6-4Row Sources and Execution Plans................................................................................................................6-6Investigate Execution Plans ..........................................................................................................................6-7EXPLAIN Command Syntax........................................................................................................................6-10Some PLAN_TABLE Columns....................................................................................................................6-11Formatting PLAN_TABLE Output...............................................................................................................6-13PLAN_TABLE: Optimization ......................................................................................................................6-15PLAN_TABLE: Parallelism .........................................................................................................................6-16PLAN_TABLE: Remote Operations ............................................................................................................ 6-17PLAN_TABLE: Partitioning ........................................................................................................................6-18PLAN_TABLE: Partitioning Example .........................................................................................................6-19Interpreting EXPLAIN Output: Example ..................................................................................................... 6-20AUTOTRACE in SQL*Plus.........................................................................................................................6-22AUTOTRACE Examples..............................................................................................................................6-25Table Access Operations............................................................................................................................... 6-28Index Access Operations............................................................................................................................... 6-29Join Operations ............................................................................................................................................. 6-30Sort-Merge Join: Example ............................................................................................................................ 6-31Other Typical Operations..............................................................................................................................6-32Index Joins ....................................................................................................................................................6-35Bitmap Operations ........................................................................................................................................ 6-36Event 10060: Dump Optimizer Predicates.................................................................................................... 6-38Summary.......................................................................................................................................................6-41

SQL Trace and TKPROF...............................................................................................................................7-1SQL Trace and TKPROF..............................................................................................................................7-2Objectives .....................................................................................................................................................7-3Using SQL Trace ..........................................................................................................................................7-4Activating SQL Trace...................................................................................................................................7-5TKPROF Utility............................................................................................................................................7-7TKPROF: Example .......................................................................................................................................7-9Event 10046 ..................................................................................................................................................7-12Using Event 10046........................................................................................................................................ 7-13Event 10046: Level 1 Output........................................................................................................................ 7-14Event 10046: PARSING IN CURSOR.........................................................................................................7-15Event 10046: Operations............................................................................................................................... 7-16Event 10046: XCTEND................................................................................................................................ 7-17Event 10046: Statistics.................................................................................................................................. 7-18Event 10046 Level 4: Bind Variables ...........................................................................................................7-2010046 Level 8: Waits .................................................................................................................................... 7-21DBMS_SUPPORT Package.......................................................................................................................... 7-23TRCSUMMARY Tool..................................................................................................................................7-25Summary.......................................................................................................................................................7-26


6/503


DSI404e Query Optimization Table of Contentsiv

Hints .................................................................................................................................................................8-1Optimizer Hints............................................................................................................................................. 8-2Objectives .....................................................................................................................................................8-3Hints: Introduction........................................................................................................................................ 8-4Hints: Usage..................................................................................................................................................8-5Hints and Optimizer Mode............................................................................................................................ 8-6When Are Hints Ignored? .............................................................................................................................8-7Optimization Approach Hints .......................................................................................................................8-8Join Order Hints............................................................................................................................................8-10Join Operation Hints .....................................................................................................................................8-11Access Path Hints..........................................................................................................................................8-12Index Access Path Hints ............................................................................................................................... 8-13Query Transformation Hints .........................................................................................................................8-15Subquery Hints..............................................................................................................................................8-17Parallel Execution Hints................................................................................................................................8-19Remote and Distributed Query Hints............................................................................................................ 8-22Additional Hints............................................................................................................................................8-24Global and Local Hints ................................................................................................................................. 8-26Some Undocumented Hints...........................................................................................................................8-27Summary.......................................................................................................................................................8-29

Plan Stability ...................................................................................................................................................9-1Plan Stability.................................................................................................................................................9-2Objectives .....................................................................................................................................................9-3Plan Stability.................................................................................................................................................9-4Stored Outlines.............................................................................................................................................. 9-5Stored Outlines: Example ............................................................................................................................. 9-6Data Dictionary Tables and Views ............................................................................................................... 9-8Hints for Stored Outlines .............................................................................................................................. 9-11Stored Outline Structure................................................................................................................................9-13QBC Labeling When Hard Parsing............................................................................................................... 9-18Stored Outline Structure: Examples..............................................................................................................9-19Using Stored Outlines...................................................................................................................................9-22Execution Plan Logic.................................................................................................................................... 9-24Maintaining Stored Outlines .........................................................................................................................9-26Editing Stored Outlines ................................................................................................................................. 9-28Stored Outline Cloning ................................................................................................................................. 9-31Administration and Security .........................................................................................................................9-33CREATE OUTLINE Syntax Changes ..........................................................................................................9-35CREATE OUTLINE Syntax: Examples ....................................................................................................... 9-37Using Private Stored Outlines.......................................................................................................................9-39Exporting and Importing Outlines ................................................................................................................ 9-41Enterprise Manager.......................................................................................................................................9-42Summary.......................................................................................................................................................9-43

Parallel Execution and Partition Pruning............... .......... ........... ........... .......... ........... ........... ........... .......... . 10-1Parallel Execution and Partition Pruning ...................................................................................................... 10-2Objectives .....................................................................................................................................................10-3Partition Pruning ........................................................................................................................................... 10-4Partition Pruning Examples ..........................................................................................................................10-5Partition Pruning Rules ................................................................................................................................. 10-7How to Identify Pruning ............................................................................................................................... 10-8Purpose of Parallel Execution .......................................................................................................................10-12Operations That Can Run Parallel................................................................................................................. 10-14Parallel Execution Terminology ................................................................................................................... 10-16Parallel Execution Example.......................................................................................................................... 10-17Query Coordinator (QC) ............................................................................................................................... 10-18Query Coordinator ........................................................................................................................................10-19


7/503


DSI404e Query Optimization Table of Contentsv

Parallel Query Slaves.................................................................................................................................... 10-21Table Queues ................................................................................................................................................ 10-23Distribution Methods .................................................................................................................................... 10-25Parallelization Granules ................................................................................................................................ 10-27Block Range Granules .................................................................................................................................. 10-28ROWID Range Granules .............................................................................................................................. 10-29Partition Granules ......................................................................................................................................... 10-30Physical Partitions: Example.........................................................................................................................10-31Granule Iterator............................................................................................................................................. 10-32Direct Reads..................................................................................................................................................10-34PX Initialization Parameters .........................................................................................................................10-36Degree of Parallelism.................................................................................................................................... 10-38Parallel Execution Hints................................................................................................................................10-40Example ........................................................................................................................................................ 10-41Example (continued)..................................................................................................................................... 10-43Full Partition-Wise Join ................................................................................................................................ 10-44Partial Partition-Wise Join ............................................................................................................................ 10-46Parallel Execution Tuning Tips.....................................................................................................................10-48Source Code References ............................................................................................................................... 10-49Other References........................................................................................................................................... 10-50Summary.......................................................................................................................................................10-51

Materialized Views and Query Rewrites ...................................................................................................... 11-1Materialized Views and Query Rewrites ...................................................................................................... 11-2Objectives .....................................................................................................................................................11-3MV Query Rewrite........................................................................................................................................11-4Cost-Based Query Rewrite............................................................................................................................11-5SQL Statements Eligible for Rewrite............................................................................................................ 11-7CBO Hints for Query Rewrite.......................................................................................................................11-8Conditions for MV Query Rewrite................................................................................................................ 11-9Enabling Query Rewrite in Session .............................................................................................................. 11-10Enabling Query Rewrite in MV.................................................................................................................... 11-11Rewrite Integrity Levels................................................................................................................................ 11-13ENFORCED Integrity Level.........................................................................................................................11-14TRUSTED Integrity Level............................................................................................................................11-15STALE_TOLERATED Integrity Level........................................................................................................11-16System Privileges for Query Rewrite............................................................................................................ 11-17Query Rewrite Methods ................................................................................................................................ 11-18Text Match Rewrite ......................................................................................................................................11-19General Query Rewrite ................................................................................................................................. 11-20MV Types Used for Query Rewrite.............................................................................................................. 11-21Join Compatibility......................................................................................................................................... 11-23Join Compatibility Graphs ............................................................................................................................ 11-24Join Compatibility: Common Joins............................................................................................................... 11-26Join Compatibility......................................................................................................................................... 11-27Join Compatibility: Special Case .................................................................................................................. 11-29Join Compatibility: Query Delta Joins.......................................................................................................... 11-31Join Compatibility: MV Delta Joins ............................................................................................................. 11-33

Data Sufficiency Check ................................................................................................................................ 11-35Grouping Compatibility Check.....................................................................................................................11-40Aggregate Computability Check................................................................................................................... 11-44Selection Compatibility Check ..................................................................................................................... 11-46Expression Matching..................................................................................................................................... 11-49Date Folding..................................................................................................................................................11-50Views and Query Rewrite............................................................................................................................. 11-52View Constraints........................................................................................................................................... 11-53Six Special Cases for MV Query Rewrite..................................................................................................... 11-56


8/503


DSI404e Query Optimization Table of Contentsvi

1. Inline Views.............................................................................................................................................. 11-572. Self Joins................................................................................................................................................... 11-593. Partially Stale MVs ................................................................................................................................... 11-624. Complex MVs........................................................................................................................................... 11-685. Nested MVs .............................................................................................................................................. 11-696. GROUP BY Extensions............................................................................................................................ 11-71Diagnostics and Troubleshooting.................................................................................................................. 11-82Diagnostics and Troubleshooting 1...............................................................................................................11-83Diagnostics and Troubleshooting 2...............................................................................................................11-86Diagnostics and Troubleshooting 3...............................................................................................................11-91Source Code References ............................................................................................................................... 11-93Summary.......................................................................................................................................................11-94

Appendix.......................................................................................................................................................... 12-1Appendix.......................................................................................................................................................12-2Topics............................................................................................................................................................ 12-3Histograms and Event 10053........................................................................................................................12-4Dynamic Sampling and Event 10053............................................................................................................12-9CBO Operations and Costs: Old Model........................................................................................................12-16CBO Operations and Costs: New Model ...................................................................................................... 12-18Initialization Parameters ............................................................................................................................... 12-21Oracle SQL Engine Architecture .................................................................................................................. 12-22Case Studies ..................................................................................................................................................12-23


9/503


IntroductionChapter 1 - Page 1

Introduction

Chapter 1


10/503



Introduction

Introduction


11/503



Objectives

Objectives

After completing this lesson, you should be able to dothe following:

Describe the origins and implications ofrelational theory

Realize that theory says nothing aboutperformance and optimal execution

Identify some basic Cost Based Optimizer (CBO)terminology

Describe the basics of the Rule Based Optimizer(RBO)


12/503



Relational Theory

Relational Theory

R S

T

U

Relational databases have their origins based on some very basic mathematicalprinciples.

John Venn (1834-1923)[http://www-history.mcs.st-andrews.ac.uk/history/Mathematicians/Venn.html]popularized a technique formodelling sets and relationships in his paper On the Diagrammatic and MechanicalRepresentation of Propositions and Reasonings in 1880. It denotes a way ofrepresenting sets and their unions and intersections. Venn considered three discsR, S,and Tas typical subsets of a set U. The intersections of these discs and theircomplements divide Uinto eight non-overlapping regions, the unions of which give256 different Boolean combinations of the original setsR, S, and T.


13/503



Relational Theory

Relational Theory

R

U

T

S

It is not a huge leap from sets to tables. The work of Ted Codd and Chris Date (andothers) has been to define how to model sets of data and their relationships in a

computerized environment using data in a tabulated format. Much of the originalvocabulary used in Venns days and built upon by Tedd Codd and Chris Date hasremained, but much has been added.

The end result of all this academic activity has been a definition of a relationaldatabase model and a standard manipulation language that combine set theory andpredicate logic, neither of which providing implementation details.

Click here[http://www-st.us.oracle.com/sql_standards/index.html]for moreinformation about the SQL standard on the Oracle intranet.

The challenge for Oracle is to comply with these standards while providing maximumperformance. Occasionally, this leads to compromises that can expose (and be

exposed by) flaws in database designs that are not normalized. For more details, seeBug:2235753[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=bug:2235753]and WebIVNote:232243.1[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:232243.1].


14/503



Translation of Relational Terms

Translation of Relational Terms

PredicateRestrict

Number of rows in arow set

Extension/Cardinality

Select Column listProject

ColumnAttribute

RowTuple

TableRelation2

Similar to a user-defined type1

Domain

RDBMSRelational

In the transition from the logical model of relational theory, many terms used havebeen changed or have colloquial alternatives. They are summarized above for

reference but are not considered in any detail in this course.Consider the following query:

select first_name, last_name

from employees

where employee_id=n

An example of syntax using relational algebralike terms to satisfy this query wouldbe the following:

project employees over first_name, last_name

restrict employee_id=n

1The purpose of a domain is to ensure that all columns that hold similar data aredefined to be of the same type. This is more than just a user-defined type. The

definition would also state the allowable values. For example, a domain ofLAST_NAMESwould disallow the use of certain characters (numbers, punctuation,

and so on) as well as specify the maximum length. Any subsequent definition of arelation with an attribute of this type would automatically inherit its properties.2A relation is a set and, as such, is not necessarily a table. It could be a result set or anintermediate result set. This concept is similar to a row source (covered later in thiscourse).


15/503



Introduction to Query Optimization

Introduction to Query Optimization

What is query optimization trying to achieve?

Optimization decides how to:

Access the data

Return the correct result

Access the data efficiently

The optimizers function is to identify the most cost-effective way of satisfying agiven query. In other words, translate the set-level language of SQL into a series of

intermediate steps that are implemented using a procedural language: the Oraclesoftware is written in C.

Optimization is based on either hard-coded best practices (for the rule-basedoptimizer) or costs derived from gathered statistical information (for the cost-basedoptimizer). There is more information on this later in the course.

Returning the result as quickly as possible is clearly the most important factor inoptimization. The result must of course be accurate.

The steps used to satisfy a query may involve objects (such as indexes) that are notdefined as part of the relational database theory or the SQL standard, or they mayinvolve intermediate data sets created temporarily just to satisfy the query (row

sources, sort segments, and so on.)


16/503



The Optimizer

The Optimizer

Objectdefinitions

OptimizerSemantic

information

Rulesand

costs

Query execution plans

Inputqueryblocks

The optimizer has a query block (or blocks) passed to it. It uses this along with otherinformation to produce the optimal access path for the query. The following pages and

the following lessons examine the contents of the optimizer box.


17/503



Query Optimization

Query Optimization

Rule-based optimizer (RBO):

Based on fixed ranking of possible access paths

Will be desupported with Oracle Database 10g

Cost-based optimizer (CBO):

Introduced with Oracle7

Based on object statistics

Using system statistics (since Oracle9i)

Oracle8i and Oracle9ioffer two query optimizers: the rule-based optimizer (RBO)and the cost-based optimizer (CBO).

As the RBO is not supported in Oracle Database10g, it is not covered in any greatdepth in this eStudy. Details here are of historical interest only. Although RBO is notsupported, it will nevertheless not be removed from the code, so you may come acrossits use from time to time.


18/503



Rule-Based Optimization (RBO)

Rule-Based Optimization (RBO)

RBO uses a documented set of rules to determineaccess paths for queries.

Rules are optimized for OLTP-type queries.

Problems:

New features not available

Difficult to tune queries

Query structure may affect access path

The RBO chooses an execution plan based on the access paths available and the ranksof these access paths.

The RBO access path ranking is heuristic. If there is more than one way to execute aSQL statement, the rule-based approach always uses the operation with the lowerrank. Usually, operations of lower rank execute faster than those associated withconstructs of higher rank.

Rule-Based Access Path Ranking

1. Single row by rowid

2. Single row by cluster join

3. Single row by hash cluster key with unique or primary key

4. Single row by unique or primary key

5. Cluster join

6. Hash cluster key7. Indexed cluster key

8. Composite key (entire key)

9. Single-column indexes (single index or index merge)

10. Bounded range search on indexed columns (including key prefix)

11. Unbounded range search on indexed columns (including key prefix)

12. Sort-merge join


19/503



13. Maximum or minimum of an indexed column

14. ORDERBYon indexed columns

15. Full table scan

For more details, see the Oracle8i online documentation[http://st-doc.us.oracle.com/8.0/817/server.817/a76992/optimops.htm#38896].

Several features cannot be used with the rule-based optimizer, and some will force the

use of the CBO if they are referenced in a query:

- Parallelism- Reverse index scans- Partition view elimination- Star joins- Hash joins- Fast full index scans- Function-based indexes- Materialized views- Temporary tables- Domains

Details of what factors affect whether the RBO or CBO is used are maintained inWebIV Note:66484.1[http://webiv.oraclecorp.com/cgi-

bin/webiv/do.pl/Get?WwwID=note: 66484.1].


20/503



RBO Query Tuning

RBO Query Tuning

To change the access path of RBO:

Change the table order in the FROMclause

Disable index usage by applying dummy functions

Add 0 toNUMBERand DATE expressions

Append empty string to [VAR]CHARexpressions

If two access methods have the same rank, then

the RBO makes an arbitrary decision: Row cache order

Order of the FROMclause (right to left)

When optimizing joins, the RBO evaluates all possible join orders (not includingCartesian products) and searches for the join order with the most single-row

predicates, fewest full table scans, and fewest sort-merge join operations.The WHEREpredicates will be worked on from bottom to top, but this will only come

into effect if an AND-EQUALoperation can use more than five indexes. In this case,

the bottom five predicates are used for selecting the five indexes for the AND-EQUAL

operation.

Functions applied to columns are applied to every row returned by the query. This canhave a major effect on performance, and care should be taken to use lightweightfunctions (like ||'' for character columns and +0 for numeric and data columns).

By placing the function on the column, you disable index usage or you can effectivelyinfluence the join order according to the preceding comment about joins.

Equal RanksIf, after evaluating all possible join orders, there is still a tie between two or morejoin

orders, the RBO will select a driving table working from righttoleftin the FROMlist.

If there is a conflict between the ranks of two access paths, the RBO has no choicebut to choose the first equally ranked access method in the row cache. This means thatthe access path of a query potentially can be affected by dropping and re-creatingindexes or other object reorganizations.


21/503



To reproduce with the RBO:

1. Create two single-column indexes.

2. Write a query that specifies both columns in the WHEREclause.

3. Recreate the indexes in the reversed sequence.4. Reexecute the query; the latest created index will be used.

indexes on table employees:

TABLE_NAME INDEX_TYPE INDEX_NAME-------------------- ---------- ------------------

EMPLOYEES UNIQUE EMP_EMAIL_UK

EMP_EMP_ID_PK

NONUNIQUE EMP_DEPARTMENT_IX

EMP_JOB_IX

EMP_MANAGER_IX

EMP_NAME_IX

SQL> select * from employees

2 where manager_id > 100

3 and department_id > 100;

Execution Plan

---------------------------------------------------

0 SELECT STATEMENT Optimizer=RULE1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMPLOYEES'

2 1 INDEX (RANGE SCAN) OF 'EMP_MANAGER_IX'

SQL> drop index EMP_DEPARTMENT_IX;

SQL> create index EMP_DEPARTMENT_IX on employees(department_id);

SQL> select * from employees

2 where manager_id > 100

3 and department_id > 100;

Execution Plan

---------------------------------------------------

0 SELECT STATEMENT Optimizer=RULE

1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMPLOYEES'

2 1 INDEX (RANGE SCAN) OF 'EMP_DEPARTMENT_IX'

Note:Ties can also happen in the CBO if identical costs are found. This, however,does not have much of an impact. First, it is fairly rare for two different plans to haveidentical cost; moreover, if they do have the same cost, then it is likely that both planswill give roughly the same performance.


22/503



Cost-Based Optimization (CBO)

Cost-Based Optimization (CBO)

CBO uses statistical information about the objectsbeing queried to determine the most cost-effective:

Access paths

Join methods

Join orders

By knowing how big a table is, how many distinct values it contains, and whether thedata is skewed, the CBO can decide the best method for processing the query. This

invariably comes up with a superior plan of execution than the RBO. Each of thesesubjects will be covered later.


23/503



Optimizer Steps

Optimizer Steps

The steps in cost-based optimization are:

1. SQL transformation

2. Cost evaluation

3. Plan selection

4. Runtime optimization

The optimizer uses both heuristic and cost-based transformations to find new ways ofwriting the query that has been submitted. This can open up new ways to join the

tables in the query.It then evaluates the cost of accessing the tables and their indexes. During thisprocess, it attempts to generate multiple execution plans. As it generates each plan, itcomputes the cost of each step, and at each step compares the cost with the cheapestcomplete plan that has already been computed. As soon as the cost exceeds that of thecheapest plan, it stops further computation of the current plan and starts the next one.This process stops when either all possible join orders have been evaluated or the

OPTIMIZER_MAX_PERMUTATIONSparameter value is exceeded.

Once the plan has been selected and the query starts running, there are stilloptimizations that can be performed.

The number of parallel execution slaves that are allocated to the query aredetermined based on the availability of slave processes.

The query has a memory profile attached to it that indicates the ideal memoryprofile required to complete the operation entirely in memory. If

PGA_AGGREGATE_TARGETis set, the Oracle server examines the amount of

memory available on the system and allocates memory to the query to provideoptimal performance in light of the current system workload. The memory will


24/503



continue to be adjusted during the life of the query as demands of the query andloads on the system change.

The database resource manager can adjust resources used by the query to stop aresource-intensive execution plan from dominating the system and reducingresponse times for other users.


25/503



Cardinality

Cardinality

SQL> select table_name, num_rows

2 from user_tables;

TABLE_NAME NUM_ROWS

----------------------- --------

COUNTRIES 25

DEPARTMENTS 27

EMPLOYEES 107

JOBS 19

JOB_HISTORY 10

LOCATIONS 23

REGIONS 4

In the remainder of this lesson, we introduce some basic terminology that is used laterin the course.

Cardinality tells you the number of rows in a row source. If a select from a table isperformed, the table is the row source and the cardinality is the number of rows in thattable.

If the result of the select from the table is fed into the next node in a query tree, thenthat too is a row source and its cardinality is the number of rows it contains. SeeWebIVNote:68992.1[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:68992.1]for more information on the relationshipbetween selectivity and cardinality.


26/503



Selectivity

Selectivity

...

SH_CLERKChungKelly188

SH_CLERKTaylorWinstone180

SA_REPTaylorJonathon176

SA_REPSmithWilliam171

SA_REPSmithLindsey159

JOB_IDLAST_NAMEFIRST_NAMEEMPLOYEE_ID

By knowing how many times values occur in a table or column, you can tell if thedata is skewed, has a regular distribution of values, or contains only unique values.

This is important as it helps the Oracle server to know whether it is better to do aserial read of a table or to use an index lookup instead.

In the example table, you can see that employee_idis highly selective, while

last_nameand job_idare not.

Selectivity is more properly defined as follows: a measure of the proportion of a rowsource retrieved by application of a particular predicate or combination of predicates.Selectivity can be expressed by the following formula:

# rows satisfying a condition

Selectivity = -----------------------------

Total # of rows


27/503



Selectivity

Selectivity

SQL> select job_id, count(*)

2 from employees

3 group by job_id

4 having count(*) > 1;

JOB_ID COUNT(*)

---------- --------

AD_VP 2

FI_ACCOUNT 5

IT_PROG 5

PU_CLERK 5SA_MAN 5

SA_REP 30

SH_CLERK 20

ST_CLERK 20

ST_MAN 5

Using the above example, a query with a predicate on job_idwould have a varying

selectivity depending on the value of the predicate.

Suppose you have an index on job_id. This index would be preferable for

predicates like job_id='AD_VP'but a full table scan may be cheaper than an

index range scan for predicates like job_id='SA_REP'.

For more details see WebIV Note:68992.1[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:68992.1].


28/503



Transitivity

Transitivity

CBO performs transitivity as the first step inoptimization.

Transitivity means generating additionalpredicates based on existing predicates.

Main advantage is the inclusion of extra accessmethods.

Transitivity is not done for join predicates.

Assume A, B, and C are columns. If A=5 and A=B, you can conclude that B=5. Thiscan be applied to queries as shown on the following page. Note that A=5 contains a

constant.However, in terms of joins, if A=B and B=C, it is notconcluded that A=C; in otherwords, the Oracle server does not generate transitive closures for join predicates.

For more details about transitivity, see WebIV Note 68979.1: Transitivity andTransitive Closure[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:68979.1].

Transitivity is one of the heuristic transformations discussed previously in this lesson.


29/503



Transitivity Example

SQL> select ... from T1, T2

2 where T1.col1 = T2.col1

3 and T1.col1 = 100

4 and T2.col1 = 100;

SQL> select ... from T1, T2

2 where T1.col1 = T2.col1

3 and T1.col1 = 100;

Transitivity Example

In the above query, assume there is an index on T2(Col1).

In the first SELECTstatement, the Oracle server is unable to drive off this index on

T2unless it performs a full index scan because it does not have a value for the leading

column. Once the transitive predicate has been introduced, it makes driving off T2

feasible because the leading column is now filled.


30/503



Summary

Summary

In this lesson, you should have learned about:

The origins and implications of relational theory

The basics of query optimization

Cost-Based Optimizer (CBO)

Rule-Based Optimizer (RBO)

Cardinality, selectivity, and transitivity


31/503



References

References

This page summarizes all references to WebIV notesand bugs used in this lesson:

Bug 2235753:ORA-1722 from valid query against a view

Note 68979.1:Transitivity and Transitive Closure

Note 68992.1:Predicate Selectivity

Note 232243.1:ORA-01722 ORA-01847 or ORA-01858 from querieswith dependent predicates

Bugs

Bug:2235753[http://webiv.oraclecorp.com/cgi-

bin/webiv/do.pl/Get?WwwID=bug:2235753].

WebIV Notes

232243.1: ORA-01722 ORA-01847 or ORA-01858 From Queries with DependentPredicates[http://webiv.oraclecorp.com/cgi-

bin/webiv/do.pl/Get?WwwID=note:232243.1].68992.1: Predicate Selectivity[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:68992.1].68979.1: Transitivity and Transitive Closure[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:68979.1].


32/503




33/503


Generic Optimizer ConceptsChapter 2 - Page 1

Generic OptimizerConcepts

Chapter 2


34/503



Generic Optimizer Concepts

Generic Optimizer Concepts


35/503



Objectives

Objectives

After completing this lesson, you should be able to dothe following:

Identify the various optimizer modes

Describe data and index access methods

Describe all join types and methods


36/503



Optimizer Modes

Optimizer Modes

CHOOSE

RULE

ALL_ROWS

FIRST_ROWS

FIRST_ROWS_{1|10|100|1000}

These parameters are all dynamic, so they can be set at the session level as well as theinstance level with an initialization parameter. Note, though, that there are other factors

that can affect which optimizer is used. Objects referenced in the query that have beenimplemented in Oracle since RBO development ceased will cause the CBO to be

invoked regardless of whether RULEhas been set at session, instance, or query level

using a hint. Hints are covered in a later lesson of this eStudy.

For definitions of these parameter values, see WebIV Note 30784.1: Init.ora Parameter"OPTIMIZER_MODE" Reference Note[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:30784.1].

For more details of what affects the behavior of the optimizer, see WebIV Note 35934.1:Cost Based Optimizer - Common Misconceptions and

Issues[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:35934.1]and WebIV Note 66484.1: Which Optimizer is BeingUsed?[http://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:66484.1].


37/503



Row Sources

Row Sources

The execution of a SQL statement is composed ofsmall building blocks called row sources.

The execution plan is the set of operations whichproduces the final result for the statement.

SQL> select t1.col1, t2.col2

2 from t1, t2, t33 where t1.col1 = t2.col1

4 ant t2.col1 = t3.col1

5 and t3.col2 = '';

Row sources are pieces of software that implement specific operations, like a tablescan or a hash join. Row sources operate on rows and produce rows. The execution

plan is the set of steps or operations which produces the final statement result.There are two types of row source operations: row operationsandset operations. Allof these operations act on a set of rows and produce a set of rows, which is fed to thenext operation in the process. The final row source output is the result of the query orSQL statement.

Consider the above query. The result of applying a filter against t3to remove rows

that do not comply with the condition t3.col2 = ''produces a

result set that can then be the input to the join condition t2.col1 = t3.col1.


38/503



Table Access Operations

Table Access Operations

There are two principal ways of retrieving rows from atable:

TABLE ACCESS FULLThe Oracle server reads all blocksup to the high-water mark (HWM).

TABLE ACCESS BY ROWIDThe Oracle server gets blocks that are identifiedby an index lookup, or by a specified ROWID.

Full Table Scans

This access method uses multiblock reads. You can set the

db_file_multiblock_read_countparameter to regulate the number of blocks ina single read. When the Oracle server reads blocks in a serial full table scan, they goto the least recently used (LRU) end of the buffer cache LRU list. For parallel scans,where the blocks do not already exist in the buffer cache, the blocks are read andwritten using direct I/O into the query slaves PGAs.

Full table scans can be efficient if they retrieve a large number of rows (more thanabout 10% of the total) or if they are executed in parallel. Otherwise, index access isusually quicker.

The high water mark is the last block ever used by the table. If rows have beendeleted, many of the tables blocks may be empty or partly empty, but they are stillread.

Table Access by ROWID

Usually, the step before a ROWIDaccess shows the index that is used. Applications

occasionally supply the ROWID, so there is no index access step.


39/503



Index Access Operations

Index Access Operations

There are five ways of retrieving data from an index:

INDEX UNIQUE SCAN Returns a single key value

Only if index is unique, for equality searches

INDEX RANGE SCAN Lookups that can return more than one row

INDEX FULL SCAN Retrieves all values

INDEX FAST FULL SCAN Scans all index blocks; can be parallel

INDEX SKIP SCAN Splits a composite index logically into smaller subindexes

and scans only the subindex of interest to the query

Index Unique Scan

This is an index lookup where the result is guaranteed to produce only one result (or

no result at all) based on knowledge about constraints or unique indexes.Index Range Scan

This is used for all index lookups using a nonunique index (as more than one row

might be returned) and range predicates (for example: BETWEEN, < or >).

Index Full Scan

Index full scans return the data in sorted order, and the CBO may decide that this is acheaper access method than a full table scan followed by a sort. But these scanscannot use multiblock reads.

The Oracle server uses an index full scan only when at least one column in the indexis not null (otherwise there is no guarantee that all rows will be present in the index).

Index Fast Full ScanIndex fast full scans are available only when using the cost-based optimizer. Indexfast full scans use multiblock reads and can be parallelized, but will not necessarilyreturn the data in sorted order. Note that index fast full scans can be used onlywhenthe index satisfies all columns needed. Index fast full scans do notreturn row IDs fora subsequent table access; this is not implemented because it does not make sense.

Index Skip Scan

Index skip scans improve index scans on nonprefix columns. Skip scanning lets a


40/503



composite index be split logically into smaller subindexes. In skip scanning, the initialcolumn of the composite index is not specified in the query. In other words, it isskipped.

The number of logical subindexes is determined by the number of distinct values inthe initial column. Skip scanning is advantageous if there are few distinct values in theleading column of the composite index and many distinct values in the nonleading key

of the index. For example, suppose you have the following (GENDER,EMPLOYEE_ID)index data:

('F',98)

('F',100)

('F',102)

('F',104)

('M',101)

('M',103)

('M',105)

The index is split logically into the following two subindexes:

-The first subindex has the keys with the value F.

-The second subindex has the keys with the value M.

The GENDERcolumn is skipped in the following query:

SQL> select *

2 from employees

3 where employee_id = 101;

A complete scan of the index is not performed, but the subindex with the value Fis

searched first, followed by a search of the subindex with the value M.

Note: Occasionally, execution plans show index access paths such as the following:

INDEX (FULL SCAN (MIN/MAX))

INDEX (RANGE SCAN (MIN/MAX))

INDEX (FULL SCAN DESCENDING), These are simple variations of the five basic paths discussed here. Note also thatbitmapped indexes are not covered here; they come with their own operations andshow very differently in execution plans.


41/503



Join Terminology

Join Terminology

Join conditionThe predicate that defines the join

Join typesThere are a number of logical join types supportedby the ISO/ANSI SQL standard.

Join methodsThe physical implementations of joins

Here is an example of a join condition:where t1.c1=t2.c1

Join Methods

Join methods are algorithms that physically implement the join types. Different joinmethods can still achieve the same results. The method chosen by the optimizerdepends on the relative cost of each method for the particular query and statisticsavailable to the optimizer.


42/503



Join Types

Join Types

Logical, abstract, and implementation-independentterms to determine the results of joining two tables

They appear in the SQL standard as:

CROSS JOIN

{LEFT|RIGHT|FULL} OUTER JOIN

NATURAL JOINJOIN ... ON

JOIN ... USING ()

Cross Join (also known as Cartesian product)

Returns all rows of the first table joined with all rows of the second table (returns

rows in t1 * rows in t2.) This is typically caused when (mistakenly) no join conditionis supplied between the two tables. In some cases, a common filter condition betweenthe two tables could be picked up by the optimizer as a possible join condition.Obviously, this is very expensive, and is rarely what was intended by the personwriting the SQL statement.

Be careful when using the ORDEREDhint. By specifying a table before its join table is

specified, you force the optimizer to perform a Cartesian join. This is covered later inmore detail.

Outer JoinAlso returns results that do not comply with the join condition (the Oracle operator(+) predates the ISO/ANSI standard) and augments the missing values with nulls. The

prefix LEFTor RIGHTdetermines which of the joined tables is null augmented. A

left outer join of T1 and T2 preserves rows from T2 with no matching row in T1. Aright outer join does the opposite, and a full outer join does both. For example, thefollowing query:

SQL> select city, country_name

2 from locations natural right outer join countries;


43/503



also returns countries that have no matching locations, with a NULL value in the

citycolumn.

Natural Join (special case of Equijoin)Returns all rows from both tables joined on columns that are common to both tables.For example, the following query:

SQL> select employee_id

2 from employees natural join departments3 where location_id = 1700;

is equivalent to:

SQL> select employee_id

2 from employees e,departments d

3 where d.department_id = e.department_id

4 and d.manager_id = e.manager_id

5 and d.location_id = 1700;

This example shows the danger of using natural joins; you probably don't want thehighlighted predicate on the fourth line above, because although both tables happen to

have a manager_idcolumn, they have a totally different meaning and should not beused to join these two tables.

Join On

This syntax offers you the possibility to separate the join predicates from the nonjoinpredicates to enhance the readability of your statements. For example, the followingquery:

SQL> select e.first_name, d.department_name

2 from employees e JOIN departments d

3 ON (e.department_id = d.department_id)

3 where d.department_id < 30;

is equivalent to:


2 from employees e, departments d

3 where e.department_id = d.department_id4 and d.department_id < 30;

Here is an example of a three-way join using the JOINONsyntax:

SQL> select e.first_name, j.job_title, d.department_name

2 from employees e

3 JOIN departments d

4 ON (e.department_id = d.department_id)

5 JOIN jobs j

6 ON (j.job_id = e.job_id)

7 where d.department_id < 30;

The preceding syntax is equivalent to:

SQL> select e.first_name, j.job_title, d.department_name

2 from employees e, departments d, jobs j3 where e.department_id = d.department_id

4 and j.job_id = e.job_id

5 and d.department_id < 30;

Join Using (Equijoin)

This is similar to a natural join, but if the tables can be joined in more than one

column, the USINGclause can specify the columns on which you want to join the

tables.


44/503



Note:The column referenced in the USINGclause may not have a qualifier (alias or

table name) anywhere in the statement; if you use such a qualifier, you get an ORA-25154.


2 from employees e JOIN departments d USING (department_id)

3 where department_id < 30;

is equivalent to:SQL> select e.first_name, d.department_name2 from employees e, departments d

3 where e.department_id = d.department_id

4 and d.department_id < 30;


45/503



Join Methods

Join Methods

SORT-MERGE JOIN Sorts tables on the join key and then merges them

together

Sorts are expensive

NESTED LOOPS Retrieves a row from one table and finds the

corresponding rows in the other table

Usually best for small numbers of rows

HASH JOIN Build hash table in memory for smaller row source

Hash larger row source

Probe in-memory hash table for matches

Hash joins are only considered by the CBO

All join operations take place on two row sources.These are usually tables but mayalso be rows produced by a previous operation in the plan.

Merge JoinsThese are often described as sort-merge joins. The table data must be sorted beforemerging. There will usually be two sort steps just before the merge join in the plan,one for each row source. If the rows have been sorted earlier in the plan, or have beenretrieved from an index, one sort step can be avoided.

Until hash joins were introduced in Oracle7.3, sort-merge joins were the onlypractical way of joining large row sources. They are expensive because sorts areexpensive.

Nested Loops Joins

The first operation in a nested loop join is a row retrieval from the first row source,

often called the outer, or driving, table. Then the Oracle server looks up thecorresponding rows in the second table. The outer row source should contain as fewrows as possible. The retrieval method for the second row source should usually be anindex lookup. If these conditions are true, nested loops can be very efficient.

Hash Joins

The idea of hash joins is to build a hash table in memory based on the smallest inputrow source, followed by a probe of the larger input row source against that hash table.Sometimes the smallest row source is too big to fit into memory and must be


46/503



Dsi404 Query Optimizer

Documents

Transcript of Dsi404 Query Optimizer