Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle...
Transcript of Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle...
![Page 1: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/1.jpg)
Aggregate Navigationusing Materialized Views and Query Rewrite
John P. McKennaCounterpoint Technologies, Inc.
![Page 2: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/2.jpg)
Agenda
Why Aggregate Navigation?What is Query Rewrite?Using Query RewriteDesign GuidelinesProduction ConsiderationsAdditional AssistanceDemonstration
![Page 3: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/3.jpg)
Why Aggregate Navigation?
Aggregate ChallengesWouldn’t it be nice if…The Aggregate NavigatorOracle’s Aggregate Navigator
![Page 4: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/4.jpg)
Aggregate Challenges
What aggregates should be created?Are current aggregates being utilized?Would intermediate summarizations be more efficient?Are our users and developers using the best available aggregate?Will changing aggregates cause saved queries or applications to fail?
![Page 5: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/5.jpg)
Wouldn’t it be nice if you could ...
utilize new aggregates without changing codeeliminate aggregates with low usagechange existing aggregations when neededeliminate the need for users and applications to understand aggregates insulate applications and users from the impact of changing aggregates
![Page 6: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/6.jpg)
The Aggregate Navigator
The aggregate navigator is a software component that intercepts SQL and transforms it to use the best available aggregate(s).According to Ralph Kimball
“An aggregate navigator is an essential component of a data warehouse because it insulates end user applications from the changing portfolio of aggregations, and allows the DBA to dynamically adjust the aggregations without having to roll over the applications base.”
![Page 7: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/7.jpg)
Oracle’s Aggregate Navigator
Since the introduction of summary management in Oracle8i, the RDBMS contains an aggregate navigation capability. This feature is known as query rewrite.
![Page 8: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/8.jpg)
What is Query Rewrite?
The DefinitionThe ProcessThe Methods
![Page 9: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/9.jpg)
The Definition
According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed in terms of tables or views into a statement accessing one or more materialized views that are defined on the detail tables.’In other words, your query is rewritten to take best advantage of summaries, joins or aggregations of your base tables that are found in materialized views.
![Page 10: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/10.jpg)
The Process
The cost based optimizer processes SQLExecution plans are generatedQuery is rewritten and additional execution plans are generatedPlan costs are compared and the least cost query is processed
![Page 11: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/11.jpg)
The Methods
SQL text match (full & partial)Selection compatibility (where clause)Join compatibility (equivalent joins)Data sufficiency (columns)Grouping compatibility (granularity)Aggregate compatibility (sum, avg)
![Page 12: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/12.jpg)
Using Query Rewrite
Configure ParametersCreate Materialized View(s)Define Constraints and DimensionsWrite & Execute SQL Statement(s)Verify Rewrite Occurred
![Page 13: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/13.jpg)
Parameter Requirements
Compatible = ‘8.1.0’ (or higher)Optimizer_mode = ‘cost’ or ‘choose’Optimizer_features_enable = ‘none’, ‘8.1.6’ (or higher)Query_rewrite_enabled = ‘true’Query_rewrite_integrity = ‘enforced’, ‘trusted’ or ‘stale_tolerated’
![Page 14: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/14.jpg)
Query Rewrite Integrity
EnforcedOnly mviews known to contain fresh dataOnly constraints that are ENABLED VALIDATEDDoes not use hierarchy information
TrustedTrusts that mviews are freshTrusts RELY and NOVALIDATE constraints Trusts hierarchy information
Stale_toleratedMaximum degree of rewrite, maximum risk
![Page 15: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/15.jpg)
Materialized View Requirements
Materialized view must contain the ‘enable query rewrite’ clauseQuery rewrite only works against materialized views, not custom built summary tablesYou may be able to register custom built summary tables as a materialized view using ‘create materialized view…on prebuilt table’
Requires query_rewrite_integrity = ‘trusted’ or ‘stale_tolerated’
![Page 16: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/16.jpg)
Constraint and Dimension Requirements
Not RequiredNot RequiredAggregate Computability
RequiredRequiredGrouping Compatibility
RequiredRequiredData Sufficiency
RequiredNot RequiredJoin Compatibility
Not RequiredNot RequiredMatching SQL Text
ConstraintsDimensionsRewrite Methods
![Page 17: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/17.jpg)
Writing SQL Statements
All SQL should be coded to access base fact and dimension tables directlyDo not reference any materialized viewsThis removes the SQL dependency on any given aggregate
![Page 18: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/18.jpg)
Did Query Rewrite Occur?
Check explain planOracle_home\rdbms\admin\utlxplan.sql
Use dbms_mview.explain_rewriteOracle_home\rdbms\admin\utlxrw.sql
![Page 19: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/19.jpg)
Design Guidelines
ConstraintsDimensionsAggregationDate FoldingHints
![Page 20: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/20.jpg)
Constraints
Define the constraints!for all primary and foreign keysinclude NOT NULL constraints
For performance (or on views)use NOVALIDATE and RELY on constraintsmust set query_rewrite_integrity = ‘stale_tolerated’ or ‘trusted’
![Page 21: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/21.jpg)
Dimensions
Define all dimensions & hierarchies!Ensure data in dimensions is accurate to hierarchies and levels, Oracle does not verify.Must set query_rewrite_integrity = ‘stale_tolerated’ or ‘trusted’
![Page 22: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/22.jpg)
Aggregation
Include aggregate functions in materialized views to support a greater number of rewrites.
Count, Sum, Avg, Etc.Include aggregate functions required for fast refreshes of the materialized views.
Count(*), count(expression), etc.
![Page 23: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/23.jpg)
Date Folding
Create aggregate columns that are eligible for date folding.
to_char(somedate, ‘YYYY-MM’) instead of to_char(somedate, ‘MM-YYYY’)
![Page 24: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/24.jpg)
Hints
Using hints defeats the purposeMake sure to implement all other techniques first (constraints, dimensions, date folding, etc.)If all else fails
/*+ norewrite *//*+ rewrite (materialviewname) */
![Page 25: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/25.jpg)
Production Considerations
Keep statistics current!They are key to evaluating SQL plans
Keep materialized views refreshed!Especially if you set query_rewrite_integrity = ‘trusted’ or ‘stale_tolerated’Failed refreshes may cause data accuracy issues.
![Page 26: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/26.jpg)
Additional Assistance
The summary advisor (dbms_olap or enterprise manager) can provide assistance with materialized view analysis and advise.Materialized views (dbms_mview)
Explain_rewrite
![Page 27: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/27.jpg)
Demonstration
Detail query w/o query rewrite enabledReview explain planEnable query rewriteRe-execute detail queryReview explain plan
![Page 28: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/28.jpg)
Questions & Answers
Feel free to contact me with additional questions:
John P. McKennaCounterpoint Technologies, [email protected](631) 757-7264
![Page 29: Aggregate Navigation using Materialized Views and … · The Definition ¾According to the Oracle data warehousing guide, query rewrite ‘...transforms a SQL statement expressed](https://reader031.fdocuments.in/reader031/viewer/2022011803/5b94d32a09d3f2205c8b85a5/html5/thumbnails/29.jpg)
References
Oracle 9.2 Data Warehousing GuideOracle 9.2 PL/SQL Packages GuideOracle 9.2 Sample Schemas GuideKimball, Ralph. The Data Warehouse Toolkit. John Wiley & Sons, Inc. 1996.