How to Use the PowerPoint Template€¦ · SQL> SELECT table_name, num_rows, last_analyzed FROM...
Transcript of How to Use the PowerPoint Template€¦ · SQL> SELECT table_name, num_rows, last_analyzed FROM...
Turbocharge the Performance of your Data Warehouse Real World Performance
Andrew Holdsworth Vice President Real-World Performance Server Technologies
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
John Clarke Director Real-World Performance Server Technologies
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The database is not being used as it was designed to be used
11/4/2015
The application architecture/code design is suboptimal
There is a suboptimal algorithm in the database
Root Causes of Suboptimal Database Performance
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Perception
DB Usage and Application Code
DB Internals
Focus in on exploring what is going on inside the database
11/4/2015
Reality
DB Usage and Application Code
DB Internals
Focus should be spent on how the database is being used
Where is the Performance Problem ?
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
We got the Plan Wrong
11/4/2015
Performance Excellence Going Beyond Excellence
Query Performance
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Fundamentals
Demo: Fundamentals
Demo: Turbocharging Star Queries
Multi-User Star Query Race Demos
1
2
3
4
7
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Fundamentals
Demo: Fundamentals
Demo: Turbocharging Star Queries
Multi-User Star Query Race Demos
1
2
3
4
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Fundamentals Star Query Fundamentals
9
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 10
Race Demo Results Star Query Fundamentals
Throughput 27x slower when not following prescription
Spending less time doing “productive” work when not
following prescription
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 11
Race Demo Results Star Query Fundamentals
“Bad” schema Good schema
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Fundamentals: The Dimensional Model and Star Queries
12
• Dimensional models are deployed in dimensional schemas
• Dimensional schemas comprised of fact and dimension tables
• Fact tables represent business events; i.e., sales, orders, logins, etc. These are referred to as measures
• Dimensions contain reference information about facts; i.e., who, what, when, how, why, etc.
• Fact tables generally contain numerical values and dimension tables contain descriptive information
Fact
Dimension
Dimension
Dimension Dimension
Dimension
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Fundamentals: The Dimensional Model and Star Queries
13
Many Vendors Work With Dimensional Data Models
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Fundamentals: The Star Schema
14
• Fact table provides the “intersection” of business event data for each dimension
• Dimension tables store reference information for facts, or attributes that describe facts
• Fact tables are tables that store data for multiple dimensions
• All measures in fact table are related to all dimensions; i.e., they have the same level of granularity
Fact
Dimension
Dimension
Dimension Dimension
Dimension
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Fundamentals: Star Queries
15
• Star queries access data in star schemas
• Star schemas comprised of fact tables and dimension tables
• Fact tables store measures
• Dimension tables store attributes to describe facts
• Tables are joined using keys
• Filtering is performed on dimension table attributes
• Dimension table attributes are used for aggregation and sorting
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Get the rows we want, efficiently
Index
16
Discard the rows we don’t want, quickly
Scans and Filtering
Fundamentals: Star Mechanics
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 17
Fundamentals: Oracle's Methods to Tackle Star Query
B*Tree Indexes with Nested Loop Joins
Full Scans with Intelligent Filtering
Bitmap Indexes and the Star Transformation Full Scans and In-Memory Aggregation
In-Memory Aggregation
Exadata
Database In-Memory
Bloom Filters
Database In-Memory
Star Transformation
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Performance Fundamentals
Statistics
Constraints
Partitioning
Data Types
18
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Performance Fundamentals
Think about skew
Think about correlation
Think about how and when to gather
Think about sampling, dynamic features, adaptive features
19
Statistics and Optimizer
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Performance Fundamentals
NOT NULL constraints on Join Keys
Primary Key constraints on Dimension Join Keys
Foreign Key Constraints on Fact Table Join Keys
20
Constraints
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Performance Fundamentals
• The state of the constraint depends on trust in the ETL process and volume of data
• Constraints must be in RELY state
• It is not necessary to enforce constraints on the fact table
• You need to tell the optimizer you can trust constraints in the RELY state
21
Constraints
alter table customer
add constraint customer_pk
primary key (c_custkey)
RELY;
alter table lineorder
add constraint lo_customer_pk
foreign key (lo_custkey)
references
customer (c_custkey)
RELY
DISABLE NOVALIDATE;
alter system
set query_rewrite_integrity=TRUSTED;
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Performance Fundamentals
Partition fact table on time dimension
Typically RANGE or INTERVAL
Reduces number of rows from fact table
Can improve manageability
22
Partitioning
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Performance Fundamentals
Data types need to be the same for dimension and fact table join keys
Data precision should also be the same
Avoids runtime data type conversion and potential optimizer challenges
23
Data Types
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Fundamentals
Demo: Fundamentals
Demo: Turbocharging Star Queries
Multi-User Star Query Race Demos
1
2
3
4
24
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Demo Fundamentals
25
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 26
Fundamentals SQL Query
SELECT d_sellingseason,
p_category,
s_region,
SUM(lo_extendedprice)
FROM lineorder
JOIN customer ON lo_custkey = c_custkey
JOIN date_dim ON lo_orderdate = d_datekey
JOIN part ON lo_partkey = p_partkey
JOIN supplier ON lo_suppkey = s_suppkey
WHERE d_year IN (1993)
AND d_monthnuminyear IN (12,1)
AND p_mfgr = 'MFGR#1'
AND p_category = 'MFGR#12'
AND s_region = 'ASIA'
AND p_color IN ('red','orange','yellow','green','purple','violet')
GROUP BY d_sellingseason, p_category, s_region
ORDER BY d_sellingseason, p_category, s_region;
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 27
Baseline Fundamentals
Time spent doing random
I/O
Nested Loops join between
PART and LINEORDER
Cardinality misestimate on
PART
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 28
Fundamentals Statistics
SQL> SELECT table_name, num_rows, last_analyzed
FROM user_tables
WHERE table_name = 'PART';
TABLE_NAME NUM_ROWS LAST_ANALYZED
========== ======== =============
PART 1 04-DEC-14 SQL> SELECT COUNT(*) FROM part;
COUNT(*)
========
12000000
SQL> EXEC DBMS_STATS.GATHER_TABLE_STATS(user,'PART');
Question: What is the first thing Support will ask you to do? Gather Statistics
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 29
Updated Statistics on Part Fundamentals
Time still spent doing random
I/O
Still doing NL join between
PART and LINEORDER
Cardinality estimate is
better, but still low
Query still took almost 2 minutes
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 30
Fundamentals Correlated Predicates and Column Group with Extended Statistics
SQL> SELECT p_mfgr, p_category
FROM part;
P_MFGR P_CATEGORY
========== ======== MFGR#3 MFGR#34 MFGR#2 MFGR#24
MFGR#5 MFGR#55
MFGR#5 MFGR#52
MFGR#2 MFGR#24
MFGR#2 MFGR#21
MFGR#3 MFGR#31
MFGR#4 MFGR#42
…
Observation: We have two predicates on PART Question: Are the column values correlated?
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 31
Fundamentals Correlated Predicates and Column Group with Extended Statistics
SQL> SELECT dbms_stats.create_extended_stats(user,'PART','(P_MFGR,P_CATEGORY')') FROM dual;
SYS_STUFJJF00WI8IBSBOHMDAMQQOL
SQL> EXPLAIN PLAN FOR
SELECT 1
FROM part
WHERE p_mfgr = 'MFGR#1'
AND p_category = 'MFGR#12';
Explained.
SQL> EXEC dbms_stats.gather_table_stats(user,'PART');
PL/SQL procedure successfully completed.
Let’s create a column group and gather extended statistics
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 32
Fundamentals Auto Column Group Creation
SQL> EXEC dbms_stats.seed_col_usage;
SQL> SELECT dbms_stats.report_col_usage(user,'PART') FROM dual;
SQL> SELECT dbms_stats.create_extended_stats(user,'PART','(P_MFGR,P_CATEGORY')')
FROM dual;
SQL> EXEC dbms_stats.gather_table_stats(user,'PART');
• Monitor column usage • Run query • Show column groups identified during Seed Column Usage • Create extended statistics on identified column groups • Gather statistics
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 33
Extended Statistics on Part Fundamentals
Hash join between PART
and LINEORDER
Cardinality estimate
accurate for PART
Elapsed time reduced to 21
seconds
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 34
Extended Statistics on Part Fundamentals
Notice join to CUSTOMER …
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 35
Fundamentals Constraints
Question: For every row in LINEORDER, how many rows will be joined from the CUSTOMER table? Answer: Two ways to answer the question: 1. Complete the join 2. Provide this information to the optimizer (and use it cleverly)
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 36
Fundamentals Constraints
SELECT uc.constraint_name,ucc1.table_name,ucc1.column_name, ucc2.table_name reftable,ucc2.column_name refcolumn
FROM user_constraints uc
, user_cons_columns ucc1
, user_cons_columns ucc2
WHERE uc.constraint_name = ucc1.constraint_name
AND uc.r_constraint_name = ucc2.constraint_name
AND ucc1.position = ucc2.POSITION
AND uc.constraint_type = 'R'
AND uc.table_name in ('LINEORDER','DATE_DIM','PART','SUPPLIER','CUSTOMER')
ORDER BY 2,3;
no rows selected.
Observations: We’re neither projecting or filtering on any columns from CUSTOMER Suggests: Lack of primary key / foreign key / NOT NULL constraints
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 37
Fundamentals Constraints
ALTER TABLE date_dim MODIFY (d_datekey) NOT NULL;
ALTER TABLE supplier MODIFY (s_suppkey) NOT NULL;
ALTER TABLE part MODIFY (p_partkey) NOT NULL;
ALTER TABLE customer MODIFY (c_custkey) NOT NULL;
ALTER TABLE date_dim ADD CONSTRAINT date_dim_pk PRIMARY KEY (d_datekey) RELY ENABLE VALIDATE;
ALTER TABLE supplier ADD CONSTRAINT supplier_pk PRIMARY KEY (s_suppkey) RELY ENABLE VALIDATE;
ALTER TABLE part ADD CONSTRAINT part_pk PRIMARY KEY (p_partkey) RELY ENABLE VALIDATE;
ALTER TABLE customer ADD CONSTRAINT customer_pk PRIMARY KEY (c_custkey) RELY ENABLE VALIDATE;
Action: Create constraints on dimension tables
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 38
Fundamentals Constraints
ALTER TABLE lineorder MODIFY lo_orderdate NOT NULL;
ALTER TABLE lineorder MODIFY lo_suppkey NOT NULL;
ALTER TABLE lineorder MODIFY lo_partkey NOT NULL;
ALTER TABLE lineorder MODIFY lo_custkey NOT NULL;
ALTER TABLE lineorder ADD CONSTRAINT lo_orderdate_fk FOREIGN KEY (lo_orderdate)
REFERENCES date_dim (d_datekey) RELY DISABLE NOVALIDATE;
ALTER TABLE lineorder ADD CONSTRAINT lo_supplier_fk FOREIGN KEY (lo_suppkey)
REFERENCES supplier (s_suppkey) RELY DISABLE NOVALIDATE;
ALTER TABLE lineorder ADD CONSTRAINT lo_part_fk FOREIGN KEY (lo_partkey)
REFERENCES part (p_partkey) RELY DISABLE NOVALIDATE;
ALTER TABLE lineorder ADD CONSTRAINT lo_customer_fk FOREIGN KEY (lo_custkey)
REFERENCES customer (c_custkey) RELY DISABLE NOVALIDATE;
Constraint Table Column Reference Tab Reference Col
========== ===== ======= ============= =============
LO_CUSTOMER_FK LINEORDER LO_CUSTKEY CUSTOMER C_CUSTKEY
LO_PART_FK LINEORDER LO_PARTKEY PART P_PARTKEY
LO_SUPPLIER_FK LINEORDER LO_SUPPKEY SUPPLIER S_SUPPKEY
Action: Create constraints on Fact Table
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 39
Fundamentals Constraints
When you create Primary Key / Foreign Key Constraints, you're providing more information to the optimizer: • For each row in the fact table, one row will be joined from dimension • Knowing this, the optimizer can perform Join Elimination • JE avoids having to join to dimension tables with no filters/projections • Can also lead to other "opportunities" in the plan
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 40
Fundamentals Constraints
Question: Can't I just re-code the reports and leave the unnecessary joins out? Answer: Sure, but often you've got tools "completing the star" on your behalf and generating code for you, so build your constraints.
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 41
Constraints Fundamentals
Now using a Bloom Filter for both DATE_DIM
and PART
Join to CUSTOMER is now gone – Join
Elimination
Elapsed time reduced to 3
seconds
Are we done?
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 42
Constraints & Partitioning Fundamentals
Scanning all partitions – indicates we’re Partitioned on
the wrong column
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 43
Fundamentals Partitioning
SELECT dbms_metadata.get_ddl('TABLE','LINEORDER') FROM dual;
CREATE TABLE XDW_BAD.LINEORDER
( LO_ORDERKEY NUMBER,
… remaining columns )
PARTITION BY RANGE ("LO_COMMITDATE")
INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
(PARTITION "R199201" VALUES LESS THAN (TO_DATE('1992-02-01 00:00:00', …
)
…
Observation: Let’s take a look at how LINEORDER is partitioned
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 44
Fundamentals Partitioning
CREATE TABLE XDW_BAD.LINEORDER
PARTITION BY RANGE ("LO_ORDERDATE")
INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
(PARTITION "R199201" VALUES LESS THAN (TO_DATE('19920201','YYYYMMDD'))
)
AS SELECT * FROM …
Action: Partition LINEORDER on LO_ORDERDATE, the join key to our date/time dimension
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 45
Partitioning Fundamentals
Nested Loops join between
DATE_DIM and LINEORDER
Elapsed time now almost 3
minutes!
What happened?
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 46
Fundamentals Data Types
SELECT table_name, column_name, data_type, data_length
FROM user_tab_columns
WHERE table_name IN
('LINEORDER','DATE_DIM')
AND column_name IN
('LO_ORDERDATE','D_DATEKEY');
TABLE COLUMN Data Type Length
========== =========== ========= =======
DATE_DIM D_DATEKEY VARCHAR2 20
LINEORDER LO_ORDERDATE DATE 7
Observation: Data type mismatch between LINEORDER and DATE_DIM join keys
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 47
Data Types Fundamentals
Bloom Filtering and Bloom
Pruning
Sub-second elapsed time!
Early filtering
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Fundamentals: Conclusion
48
Elapsed time
Baseline 74 seconds
Gathering statistics on PART 116 seconds
Creating column group on PART and gathering extended statistics 21 seconds
Adding PK, FK, and NOT NULL constraints 3 seconds
Partitioning LINEORDER on date/time join key 173 seconds
Fixing data type on date dimension 0.3 seconds
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Query Fundamentals: Conclusion
Statistics, Constraints, Partitioning, and Data Types are all important
You need to get them all right
Now let’s turbocharge our queries!
49
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Turbocharging your Star Queries
50
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Fundamentals
Demo: Fundamentals
Demo: Turbocharging Star Queries
Multi-User Star Query Race Demos
1
2
3
4
51
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Turbocharging Star Queries
Exadata
Database In-Memory
Compression
Parallel Query
Clustering and Zonemaps
In-Memory Aggregation
Materialized Views
52
Options
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 53
Turbocharging Star Queries SQL Query
SELECT d_year,
d_sellingseason,
c_region,
SUM(lo_extendedprice),
SUM(lo_supplycost)
FROM lineorder
JOIN customer ON lo_custkey = c_custkey
JOIN date_dim ON lo_orderdate = d_datekey
JOIN part ON lo_partkey = p_partkey
JOIN supplier ON lo_suppkey = s_suppkey
WHERE d_month IN ('May','June','July','August')
AND p_mfgr IN ('MFGR#1','MFGR#2')
AND s_nation IN ('CHINA')
GROUP BY d_year, d_sellingseason, c_region
ORDER BY d_year, d_sellingseason, c_region;
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 54
Exadata with Fundamentals in Place Turbocharging Star Queries
Nice, tidy right-deep hash join plan with Bloom Filtering and Bloom Pruning
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 55
DBIM with Fundamentals in Place Turbocharging Star Queries
Elapsed time similar – why?
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 56
DBIM with Zonemap & Clustering Turbocharging Star Queries
Note ZONEMAP and more rows being filtered from fact table
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 57
Turbocharging Star Queries Zonemaps
ALTER TABLE lineorder
ADD CLUSTERING lineorder
JOIN supplier ON (lineorder.lo_suppkey = supplier.s_suppkey)
BY LINEAR ORDER (supplier.s_nation)
YES ON LOAD YES ON DATA MOVEMENT WITH MATERIALIZED ZONEMAP (lineorder_zmap);
ALTER TABLE lineorder MOVE NOLOGGING;
• Add clustering to lineorder • Clustering is based on join between supplier and lineorder on the join key • Clustering by linear order means that the data in lineorder will be ordered by lo_suppkey based on the supplier.s_nation column. This reduces amount of rows scanned (or CUs scanned) when predicates exist on suppler.s_nation • Create a Zonemap called lineorder_zmap • Move the table to establish clustering
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 58
Database In-Memory with In-Memory Aggregation Turbocharging Star Queries
With In-Memory Aggregation we’re able to aggregate as we scan, reducing time spent on
aggregation code path
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 59
Database In-Memory with Materialized Join View Turbocharging Star Queries
If we eliminate all joins by using a Materialized Join View, elapsed time reduces to 2 seconds
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 60
Database In-Memory with Attribute Clustered Materialized Join View Turbocharging Star Queries
If we then attribute cluster our MJV on one of our query predicates, DBIM Min/Max pruning
yields additional benefits
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 61
Database In-Memory with Partitioned, Attribute Clustered Materialized Join View Turbocharging Star Queries
If we partition our MJV by date and also attribute cluster it on s_nation, we see
additional gains
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 62
Turbocharging Star Queries Partitioned Materialized View with Attribute Clustering
CREATE MATERIALIZED VIEW xdw_mv
PARTITION BY LIST (d_month) (
PARTITION p_jan VALUES ('January'),
PARTITION p_feb VALUES ('February'),
PARTITION p_mar VALUES ('March'),
…
CLUSTERING BY LINEAR ORDER (suppnation)
[ MV build options ]
AS SELECT s.s_name suppname, s.s_city suppcity, s.s_nation suppnation,...
FROM lineorder lo, part p, supplier s, customer c, date_dim d
WHERE lo.lo_partkey = p.p_partkey
AND lo.lo_suppkey = s.s_suppkey
…
• Create a partitioned materialized join view, list partitioned by d_month • Add clustering clause • Select columns to project in MJV • Complete joins to all dimensions
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multiuser Star Query Race Demos
63
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Fundamentals
Demo: Fundamentals
Demo: Turbocharging Star Queries
Multi-User Star Query Race Demos
1
2
3
4
64
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 65
Race Demo Results, Race #1 Multiuser Star Query Race Demo DBIM without partitioning yields
4X improvement over Exadata with the complete recipe
applied
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 66
Race Demo Results, Race #1 Multiuser Star Query Race Demo
Race statistics
Exadata cell utilization high
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 67
Race Demo Results, Race #2 Multiuser Star Query Race Demo
IMA with fundamentals applied wins over partitioned DBIM,
partitioned DBIM with Zonemaps, and un-partitioned
DBIM with Zonemaps
Partitioning with Zonemaps provides best non-IMA
throughput
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 68
Race Demo Results, Race #3 Multiuser Star Query Race Demo
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multiuser Race Demo Summary
69
19 596
2424
2871
5937
7644
8387
9280
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Exadata without Recipe
Exadata with Recipe DBIM with Recipe DBIM with Recipe + Zonemaps
DBIM with IMA & Recipe
DBIM with MV DBIM with Clustered MV
DBIM with Partitioned
Clustered MV
Queries Completed
Queries Completed
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multiuser Race Demo Summary
Applying the Fundamentals is necessary to get the right execution plans
In multiuser tests, Exadata performs well but ultimately we’re constrained on IO
DBIM provides benefit under heavy workloads
In-Memory Aggregation improves time on queries limited by aggregation
Partitioning, Zonemaps and Attribute Clustering are means to reduce work and improve throughput
70
Queries completed, 48 user test Queries completed
Exadata without Recipe 19
Exadata with Recipe applied 596
DBIM with Partitioning 2,424
DBIM with Zonemaps and Partitioning 2,871
In-Memory Aggregation + Recipe 5,937
DBIM with MV 7,644
DBIM with Attribute Clustering 8,387
DBIM with Partitioned MV and Attribute Clustering 9,280
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Summary: Indications You've Got Something Wrong
NL joins and index access
Cardinality misestimates
Joins to tables in which we're not projecting columns from and have no filter predicates
Lack of early filtering/pruning; hash joins reducing rows as we proceed through the plan
Inability to scale with PX
Exadata &/or DBIM not helping performance
71
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Summary
You have to walk before you can run!
Your schema and use of the database are important to get the right execution plans
You need the right plans in order to turbocharge!
72
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Where to get more info
• Real-World Performance Engineers discussing and demonstrating performance issues, root causes and when to apply the correct techniques
– The Optimizer
– Core DB Performance
– Extreme OLTP
– Extreme DW
• http://www.oracle.com/goto/oll/rwp
11/4/2015
Real-World Performance Online Video Series
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
74
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Appendix: Bloom Filters
• Efficient way to filter data
• Bloom Filters created from dimension tables and applied to fact table during scan
• Utilizes swap join optimization and yields right-deep plans
• Filtered data is pipelined to hash joins
75
Conceived by Burton Howard Bloom in 1970
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 76
Creating the Bloom Filter
Appendix: Bloom Filters
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
For dimension tables, create an array of bits
Size estimated by NDV of join column in dimension table
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 77
Build
Appendix: Bloom Filters
0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
10
This example uses 3 hash functions
As we scan dimension table, apply hash function(s) and turn
bits on (0->1)
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 78
Build
Appendix: Bloom Filters
0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0
10 20
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 79
Build
Appendix: Bloom Filters
0 0 0 0 0 1 0 1 0 0 1 0 0 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0
10 20 30
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 80
Build
Appendix: Bloom Filters
0 0 0 0 0 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0
10 20 30 40
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 81
Bloom Filter pushed/passed down
Appendix: Bloom Filters
0 0 0 0 0 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0
Think of this as a bit signature of the join
Bits set = number of entries in bit array set to 1
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 82
Test
Appendix: Bloom Filters
0 0 0 0 0 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0
10 Probable Match
10 20 30 40
For each row in fact table, apply hash function to determine
match
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 83
Test
Appendix: Bloom Filters
0 0 0 0 0 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0
30 Probable Match
10 20 30 40
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 84
Test
Appendix: Bloom Filters
0 0 0 0 0 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0
60 Definite No Match
10 20 30 40
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 85
Test
Appendix: Bloom Filters
0 0 0 0 0 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0
70 Probable Match
In this case, the match is in fact a false positive
10 20 30 40
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Appendix: Bloom Filters
• Goal is to scan fact table and quickly eliminate rows
• Effective because computers are good at bitwise operations
• More efficient than hash joins alone because building and testing Bloom Filters is fast -> working with bits, not bytes
• Bloom Pruning uses same concepts and used to quickly prune partitions
86
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 87
Identifying in Plans
Appendix: Bloom Filters
Bloom Filter create
Bloom Filter use
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 88
Left Deep Tree
Appendix: Bloom Filters and Swap Join Optimization
PART
HASH JOIN
LINEORDER
HASH JOIN
DATEDIM
HASH JOIN
SUPPLIER
1 2
3
4
1
2
3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 89
… becomes a Right Deep Tree
Appendix: Bloom Filters and Swap Join Optimization
PART
HASH JOIN
LINEORDER
HASH JOIN
DATE_DIM
HASH JOIN
SUPPLIER
2
3
1
2
3
4
1
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 90
91