Meetup Oracle Database: 1.2 Oracle Database 18c (autonomous database)
Oracle Database Advanced Querying
-
Upload
zohar-elkayam -
Category
Technology
-
view
2.202 -
download
4
Transcript of Oracle Database Advanced Querying
![Page 1: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/1.jpg)
1
Oracle Database Advanced Querying
111
Zohar Elkayam CTO, Brillix
Twitter: @realmgic
![Page 2: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/2.jpg)
22
• Zohar Elkayam, CTO/COO at Brillix-DBAces
• Oracle ACE Associate
• DBA, team leader, Oracle University instructor, public speaker, and a senior consultant for over 18 years
• Editor of ilDBA – Israel Database Community website
• Blogger – www.realdbamagic.com
Who am I?
![Page 3: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/3.jpg)
3
We are committed to provide the highest quality of services delivered by our dedicated team of highly trained and experienced industry’s top data experts. We offer:
Complete integrated end-to-end solutions based on best-of-breed innovations in database, security and big data technologies
On-site professional customized trainings led by our team of Oracle ACEs and Oracle Certified Professionals
Comprehensive security solutions and services for leading database platforms and business applications, leveraging a world-class team of security experts
About Brillix
![Page 4: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/4.jpg)
44
• Aggregative and advanced grouping options
• Analytic functions, ranking and pagination
• Hierarchical and recursive queries
• Oracle 12c new rows pattern matching feature
• XML and JSON handling with SQL
• Regular Expressions
• SQLcl – a new replacement tool for SQL*Plus from Oracle
Agenda
![Page 5: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/5.jpg)
55
יכולות –Oracle SQL"הספר •
מדריך לשולף , מתקדמות
2011-פורסם ב" המהיר
הראשון SQL-זה ספר ה•
והיחיד שנכתב בעברית
מתחילתו ועד סופו
הספר נכתב על ידי עמיאל •דיוויס ועבר עריכה טכנית שלי
יכולות מתקדמות–Oracle SQLאודות
![Page 6: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/6.jpg)
6
The REAL Agenda
הפסקה10:30-10:45
י הכנסארוחת צהריים לכל משתתפ12:30-13:30
הפסקה מתוקה15:00-15:15
הולכים הביתה16:30
![Page 7: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/7.jpg)
77
• SQL was invented in 1970 by Dr. E. F. Codd
• Each vendor had its own flavor of SQL
• Standardized by ASNI since 1986
• Current stable standard is ANSI SQL:2011/2008
• Oracle 12c is fully compliant to CORE SQL:2011
• Oracle 11g is compliant to SQL:2008
ANSI SQL
![Page 8: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/8.jpg)
88
• In this seminar we will only talk about Queries
Queries
![Page 9: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/9.jpg)
Group Functions
More than just group by…
9
![Page 10: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/10.jpg)
10
• Using SQL for aggregation:
– Group functions basics
– The CUBE and ROLLUP extensions to the GROUP BY clause
– The GROUPING functions
– The GROUPING SETS expression
• Working with composite columns
• Using concatenated groupings
Group Function and SQL
![Page 11: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/11.jpg)
1111
• Group functions will return a single row for each group
• The group by clause groups rows together and allows group functions to be applied
• Common group functions: SUM, MIN, MAX, AVG, etc.
Basics
![Page 12: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/12.jpg)
12
Group Functions Syntax
SELECT [column,] group_function(column). . .FROM table[WHERE condition][GROUP BY group_by_expression][ORDER BY column];
SELECT AVG(salary), STDDEV(salary),COUNT(commission_pct),MAX(hire_date)
FROM hr.employeesWHERE job_id LIKE 'SA%';
![Page 13: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/13.jpg)
13
SELECT department_id, job_id, SUM(salary), COUNT(employee_id)
FROM hr.employeesGROUP BY department_id, job_idOrder by department_id;
The GROUP BY Clause
SELECT [column,] group_function(column)FROM table[WHERE condition][GROUP BY group_by_expression][ORDER BY column];
![Page 14: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/14.jpg)
14
• Use the HAVING clause to specify which groups are to be displayed
• You further restrict the groups on the basis of a limiting condition
The HAVING Clause
SELECT [column,] group_function(column)... FROM table[WHERE condition][GROUP BY group_by_expression][HAVING having_expression] [ORDER BY column];
![Page 15: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/15.jpg)
15
• Use ROLLUP or CUBE with GROUP BY to produce superaggregate rows by cross-referencing columns
• ROLLUP grouping produces a result set containing the regular grouped rows and the subtotal and grand total values
• CUBE grouping produces a result set containing the rows from ROLLUP and cross-tabulation rows
GROUP BY with the ROLLUP and CUBE Operators
![Page 16: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/16.jpg)
16
• ROLLUP is an extension of the GROUP BYclause
• Use the ROLLUP operation to produce cumulative aggregates, such as subtotals
Using the ROLLUP Operator
SELECT [column,] group_function(column). . .FROM table[WHERE condition][GROUP BY [ROLLUP] group_by_expression][HAVING having_expression];[ORDER BY column];
![Page 17: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/17.jpg)
17
Using the ROLLUP Operator: ExampleSELECT department_id, job_id, SUM(salary)FROM hr.employeesWHERE department_id < 60GROUP BY ROLLUP(department_id, job_id);
1
2
3
Total by DEPARTMENT_IDand JOB_ID
Total by DEPARTMENT_ID
Grand total
![Page 18: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/18.jpg)
18
• CUBE is an extension of the GROUP BY clause
• You can use the CUBE operator to produce cross-tabulation values with a single SELECTstatement
Using the CUBE Operator
SELECT [column,] group_function(column)...FROM table[WHERE condition][GROUP BY [CUBE] group_by_expression][HAVING having_expression][ORDER BY column];
![Page 19: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/19.jpg)
19
1
2
3
4
Grand total
Total by JOB_ID
Total by DEPARTMENT_IDand JOB_ID
Total by DEPARTMENT_ID
SELECT department_id, job_id, SUM(salary)FROM hr.employeesWHERE department_id < 60GROUP BY CUBE (department_id, job_id);
. . .
Using the CUBE Operator: Example
![Page 20: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/20.jpg)
20
SELECT [column,] group_function(column) .. ,GROUPING(expr)
FROM table[WHERE condition][GROUP BY [ROLLUP][CUBE] group_by_expression][HAVING having_expression][ORDER BY column];
• The GROUPING function: – Is used with the CUBE or ROLLUP operator– Is used to find the groups forming the subtotal in a row– Is used to differentiate stored NULL values from NULL
values created by ROLLUP or CUBE– Returns 0 or 1
Working with the GROUPING Function
![Page 21: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/21.jpg)
21
SELECT department_id DEPTID, job_id JOB, SUM(salary),GROUPING(department_id) GRP_DEPT,GROUPING(job_id) GRP_JOB
FROM hr.employeesWHERE department_id < 50GROUP BY ROLLUP(department_id, job_id);
Working with the GROUPING Function: Example
12
3
![Page 22: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/22.jpg)
2222
• Extension to the GROUPING function
• GROUPING_ID returns a number corresponding to the GROUPING bit vector associated with a row
• Useful for understanding what level the row is aggregated at and filtering those rows
Working with GROUPING_ID Function
![Page 23: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/23.jpg)
2323
GROUPING_ID Function ExampleSELECT department_id DEPTID, job_id JOB,
SUM(salary),GROUPING_ID(department_id,job_id) GRP_ID
FROM hr.employeesWHERE department_id < 40GROUP BY CUBE(department_id, job_id);
DEPTID JOB SUM(SALARY) GRP_ID---------- ---------- ----------- ----------
48300 3
MK_MAN 13000 2MK_REP 6000 2PU_MAN 11000 2AD_ASST 4400 2PU_CLERK 13900 2
10 4400 110 AD_ASST 4400 020 19000 120 MK_MAN 13000 020 MK_REP 6000 030 24900 130 PU_MAN 11000 030 PU_CLERK 13900 0
![Page 24: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/24.jpg)
2424
• GROUP_ID distinguishes duplicate groups resulting from a GROUP BY specification
• A Unique group will be assigned 0, the non unique will be assigned 1 to n-1 for n duplicate groups
• Useful in filtering out duplicate groupings from the query result
Working with GROUP_ID Function
![Page 25: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/25.jpg)
2525
GROUP_ID Function ExampleSELECT department_id DEPTID, job_id JOB,
SUM(salary),GROUP_ID() UNIQ_GRP_ID
FROM hr.employeesWHERE department_id < 40GROUP BY department_id, CUBE(department_id, job_id);
DEPTID JOB SUM(SALARY) UNIQ_GRP_ID---------- ---------- ----------- -----------
10 AD_ASST 4400 020 MK_MAN 13000 020 MK_REP 6000 030 PU_MAN 11000 030 PU_CLERK 13900 010 AD_ASST 4400 120 MK_MAN 13000 120 MK_REP 6000 130 PU_MAN 11000 130 PU_CLERK 13900 110 4400 020 19000 030 24900 010 4400 120 19000 130 24900 1
![Page 26: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/26.jpg)
26
• The GROUPING SETS syntax is used to define multiple groupings in the same query.
• All groupings specified in the GROUPING SETSclause are computed and the results of individual groupings are combined with a UNION ALLoperation.
• Grouping set efficiency:– Only one pass over the base table is required.– There is no need to write complex UNION statements.– The more elements GROUPING SETS has, the
greater the performance benefit.
GROUPING SETS
![Page 27: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/27.jpg)
28
SELECT department_id, job_id, manager_id, AVG(salary)
FROM hr.employeesGROUP BY GROUPING SETS
((department_id,job_id), (job_id,manager_id));
GROUPING SETS: Example
. . .
. . .
1
2
![Page 28: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/28.jpg)
30
• A composite column is a collection of columns that are treated as a unit.ROLLUP (a,(b,c), d)
• Use parentheses within the GROUP BY clause to group columns, so that they are treated as a unit while computing ROLLUP or CUBE operators.
• When used with ROLLUP or CUBE, composite columns require skipping aggregation across certain levels.
Composite Columns
![Page 29: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/29.jpg)
32
SELECT department_id, job_id, manager_id,
SUM(salary)
FROM hr.employees
GROUP BY ROLLUP( department_id,(job_id, manager_id));
Composite Columns: Example
…
1
2
3
4
![Page 30: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/30.jpg)
34
• Concatenated groupings offer a concise way to generate useful combinations of groupings.
• To specify concatenated grouping sets, you separate multiple grouping sets, ROLLUP, and CUBE operations with commas so that the Oracle server combines them into a single GROUP BY clause.
• The result is a cross-product of groupings from each GROUPING SET.
Concatenated Groupings
GROUP BY GROUPING SETS(a, b), GROUPING SETS(c, d)
![Page 31: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/31.jpg)
35
SELECT department_id, job_id, manager_id, SUM(salary)
FROM hr.employeesGROUP BY department_id,
ROLLUP(job_id),CUBE(manager_id);
Concatenated Groupings: Example
…
…
…
1
3
4
5
6
2
7
…
…
![Page 32: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/32.jpg)
Pivot and Unpivot
Turning things around!
36
![Page 33: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/33.jpg)
PIVOT and UNPIVOT Clauses of the SELECT Statement
• You can use the PIVOT operator of the SELECT statement to write cross-tabulation queries that rotate the column values into new columns, aggregating data in the process.
• You can use the UNPIVOT operator of the SELECT statement to rotate columns into values of a column.
PIVOT UNPIVOT
![Page 34: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/34.jpg)
38
Pivoting on the QUARTERColumn: Conceptual Example
30,000
40,000
60,000
30,000
40,000
20,000
AMOUNT_SOLD
2,500Q1IUSAKids Jeans
2,000Q2CJapanKids Jeans
2,000Q3SUSAShorts
I
P
C
CHANNEL
Kids Jeans
Shorts
Shorts
PRODUCT
1,000Q2Germany
1,500Q4USA
Q2
QUARTER
2,500Poland
QUANTITY_SOLD
COUNTRY
2,000
Q3
Kids Jeans
Shorts
PRODUCT
3,500
2,000
Q2
1,5002,500
Q4Q1
![Page 35: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/35.jpg)
3939
• Pivoting the data before 11g was a complex query which required the use of the CASE or DECODE functions
Pivoting Before Oracle 11g
select product,sum(case when quarter = 'Q1' then amount_sold else null end) Q1,sum(case when quarter = 'Q2' then amount_sold else null end) Q2,sum(case when quarter = 'Q3' then amount_sold else null end) Q3,sum(case when quarter = 'Q4' then amount_sold else null end) Q4from sales
group by product;
![Page 36: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/36.jpg)
40
Pivot Clause Syntaxtable_reference PIVOT [ XML ]
( aggregate_function ( expr ) [[AS] alias ][, aggregate_function ( expr ) [[AS] alias ] ]...
pivot_for_clausepivot_in_clause )
-- Specify the column(s) to pivot whose values are to -- be pivoted into columns.pivot_for_clause = FOR { column |( column [, column]... ) }
-- Specify the pivot column values from the columns you -- specified in the pivot_for_clause.pivot_in_clause = IN ( { { { expr | ( expr [, expr]... ) } [ [ AS] alias] }...
| subquery | { ANY | ANY [, ANY]...} } )
![Page 37: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/37.jpg)
42
Creating a New View: Example
CREATE OR REPLACE VIEW sales_view ASSELECTprod_name AS product, country_name AS country, channel_id AS channel, SUBSTR(calendar_quarter_desc, 6,2) AS quarter,SUM(amount_sold) AS amount_sold, SUM(quantity_sold) AS quantity_sold
FROM sales, times, customers, countries, productsWHERE sales.time_id = times.time_id AND
sales.prod_id = products.prod_id ANDsales.cust_id = customers.cust_id ANDcustomers.country_id = countries.country_id
GROUP BY prod_name, country_name, channel_id,SUBSTR(calendar_quarter_desc, 6, 2);
![Page 38: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/38.jpg)
44
Selecting the SALES VIEW DataSELECT product, country, channel, quarter, quantity_soldFROM sales_view;
PRODUCT COUNTRY CHANNEL QUARTER QUANTITY_SOLD------------ ------------ ---------- -------- -------------Y Box Italy 4 01 21Y Box Italy 4 02 17Y Box Italy 4 03 20. . .Y Box Japan 2 01 35Y Box Japan 2 02 39Y Box Japan 2 03 36Y Box Japan 2 04 46Y Box Japan 3 01 65. . .Bounce Italy 2 01 34Bounce Italy 2 02 43. . .9502 rows selected.
![Page 39: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/39.jpg)
45
Pivoting the QUARTER Column in the SH Schema: Example
SELECT *FROM
(SELECT product, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR quarter IN ('01', '02', '03', '04'))
ORDER BY product DESC;
. . .
![Page 40: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/40.jpg)
47
Unpivoting the QUARTER Column: Conceptual Example
2,000
Q3
Kids Jeans
Shorts
PRODUCT
3,500
2,000
Q2
1,5002,500
Q4Q1
2,500Q1Kids Jeans
2,000Q2Kids Jeans
3,500Q2Shorts
1,500Q4Kids Jeans
Q3
QUARTER
2,000Shorts
SUM_OF_QUANTITYPRODUCT
![Page 41: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/41.jpg)
4848
• Univoting the data before 11g requires multiple queries on the table using the UNION ALL operator
Unpivoting Before Oracle 11g
SELECT *FROM (
SELECT product, '01' AS quarter, Q1_value FROM salesUNION ALLSELECT product, '02' AS quarter, Q2_value FROM salesUNION ALLSELECT product, '03' AS quarter, Q3_value FROM salesUNION ALLSELECT product, '04' AS quarter, Q4_value FROM sales);
![Page 42: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/42.jpg)
49
• An UNPIVOT operation does not reverse a PIVOT operation; instead, it rotates data found in multiple columns of a single row into multiple rows of a single column.
• If you are working with pivoted data, UNPIVOTcannot reverse any aggregations that have been made by PIVOT or any other means.
Using the UNPIVOT Operator
UNPIVOT
![Page 43: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/43.jpg)
50
• The UNPIVOT clause rotates columns from a previously pivoted table or a regular table into rows. You specify:– The measure column or columns to be unpivoted
– The name or names for the columns that result from the UNPIVOT operation
– The columns that are unpivoted back into values of the column specified in pivot_for_clause
• You can use an alias to map the column name to another value.
Using the UNPIVOT Clause
![Page 44: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/44.jpg)
51
UNPIVOT Clause Syntax
table_reference UNPIVOT [{INCLUDE|EXCLUDE} NULLS]-- specify the measure column(s) to be unpivoted.( { column | ( column [, column]... ) }
unpivot_for_clauseunpivot_in_clause )
-- Specify one or more names for the columns that will-- result from the unpivot operation.
unpivot_for_clause = FOR { column | ( column [, column]... ) }
-- Specify the columns that will be unpivoted into values of -- the column specified in the unpivot_for_clause.
unpivot_in_clause = ( { column | ( column [, column]... ) }
[ AS { constant | ( constant [, constant]... ) } ][, { column | ( column [, column]... ) }[ AS { constant | ( constant [, constant]...) } ] ]...)
![Page 45: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/45.jpg)
52
Creating a New Pivot Table: Example
. . .
CREATE TABLE pivotedtable ASSELECT *FROM
(SELECT product, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR quarter IN ('01' AS Q1, '02' AS Q2,
'03' AS Q3, '04' AS Q4));
SELECT * FROM pivotedtable ORDER BY product DESC;
![Page 46: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/46.jpg)
53
Unpivoting the QUARTER Column in the SH Schema: Example
SELECT *FROM pivotedtableUNPIVOT (quantity_sold For Quarter IN (Q1, Q2, Q3, Q4))ORDER BY product DESC, quarter;
. . .
![Page 47: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/47.jpg)
5454
• More information and examples could be found on my Blog:
http://www.realdbamagic.com/he/pivot-a-table/
More Examples…
![Page 48: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/48.jpg)
Analytic FunctionsLet’s analyze our data!
55
![Page 49: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/49.jpg)
56
• Oracle has enhanced SQL's analytical processing capabilities by introducing a new family of analytic SQL functions.
• These analytic functions enable you to calculate and perform:– Rankings and percentiles– Pivoting operations– Moving window calculations– LAG/LEAD analysis– FIRST/LAST analysis– Linear regression statistics
Overview of SQL for Analysis and Reporting
![Page 50: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/50.jpg)
5757
• Ability to see one row from another row in the results
• Avoid self-join queries
• Summary data in detail rows
• Slice and dice within the results
Why Use Analytic Functions?
![Page 51: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/51.jpg)
58
Using the Analytic Functions
Function type Used for
Ranking Calculating ranks, percentiles, and n-tiles of the values in a
result set
Windowing Calculating cumulative and moving aggregates, works with functions such as SUM, AVG, MIN, and so on
Reporting Calculating shares such as market share, works with functions such as SUM, AVG, MIN, MAX, COUNT, VARIANCE,
STDDEV, RATIO_TO_REPORT, and so on
LAG/LEAD Finding a value in a row or a specified number of rows
from a current row
FIRST/LAST First or last value in an ordered group
Linear Regression Calculating linear regression and other statistics
![Page 52: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/52.jpg)
59
• Result set partitions: These are created and available to any aggregate results such as sums and averages. The term “partitions” is unrelated to the table partitions feature.
• Window: For each row in a partition, you can define a sliding window of data, which determines the range of rows used to perform the calculations for the current row.
• Current row: Each calculation performed with an analytic function is based on a current row within a partition. It serves as the reference point determining the start and end of the window.
Concepts Used in Analytic Functions
![Page 53: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/53.jpg)
61
• We can use aggregative functions as analytic functions (i.e. SUM, AVG, MIN, MAX, COUNT
etc.)
• Each row will get the aggregative value for a given partition without the need for group by clause
Reporting Functions
![Page 54: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/54.jpg)
62
• We can have multiple group by on the same row
• Getting the raw data along with the aggregated value
Reporting Example
SELECT last_name, salary,ROUND(AVG(salary) OVER (PARTITION BY department_id),2),COUNT(*) OVER (PARTITION BY manager_id),SUM(salary) OVER (PARTITION BY department_id ORDER BY salary),MAX(salary) OVER ()
FROM hr.employees;
![Page 55: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/55.jpg)
Ranking Functions
![Page 56: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/56.jpg)
64
• A ranking function computes the rank of a record compared to other records in the data set based on the values of a set of measures. The types of ranking function are:– RANK and DENSE_RANK functions
– PERCENT_RANK function
– ROW_NUMBER function
– NTILE function
– CUME_DIST function
Using the Ranking Functions
![Page 57: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/57.jpg)
65
• The RANK function calculates the rank of a value in a group of values, which is useful for top-N and bottom-N reporting.
• For example, you can use the RANK function to find the top ten products sold in Boston last year.
• When using the RANK function, ascending is the default sort order, which you can change to descending.
• Rows with equal values for the ranking criteria receive the same rank.
• Oracle Database then adds the number of tied rows to the tied rank to calculate the next rank.
Working with the RANK Function
RANK ( ) OVER ( [query_partition_clause] order_by_clause )
![Page 58: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/58.jpg)
66
Using the RANK Function: Example
SELECT department_id, last_name, salary,RANK() OVER (PARTITION BY department_idORDER BY salary DESC) "Rank"
FROM employees WHERE department_id = 60ORDER BY department_id, "Rank", salary;
![Page 59: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/59.jpg)
67
• The RANK function can be made to operate within groups—that is, the rank gets reset whenever the group changes.
• This is accomplished with the PARTITION BYclause.
• The group expressions in the PARTITION BYsubclause divide the data set into groups within which RANK operates.
• For example, to rank products within each channel by their dollar sales, you could issue a statement similar to the one in the next slide.
Per-Group Ranking
![Page 60: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/60.jpg)
68
Per-Group Ranking: ExampleSELECT channel_desc, calendar_month_desc, TO_CHAR(SUM(amount_sold),'9,999,999,999') SALES$, RANK() OVER (PARTITION BY channel_descORDER BY SUM(amount_sold) DESC) AS RANK_BY_CHANNELFROM sales, products, customers, times, channelsWHERE sales.prod_id = products.prod_idAND sales.cust_id = customers.cust_idAND sales.time_id = times.time_id AND sales.channel_Id = channels.channel_idAND times.calendar_month_desc IN ('2000-08', '2000-09', '2000-
10', '2000-11')AND channels.channel_desc IN ('Direct Sales', 'Internet')
GROUP BY channel_desc, calendar_month_desc;
![Page 61: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/61.jpg)
69
RANK and DENSE_RANK Functions: Example
SELECT department_id, last_name, salary,RANK() OVER (PARTITION BY department_idORDER BY salary DESC) "Rank",
DENSE_RANK() over (partition by department_idORDER BY salary DESC) "Drank"
FROM employees WHERE department_id = 60ORDER BY department_id, last_name, salary DESC, "Rank" DESC;
DENSE_RANK ( ) OVER ([query_partition_clause] order_by_clause)
![Page 62: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/62.jpg)
70
Per-Cube and Rollup Group Ranking
SELECT channel_desc, country_iso_code, TO_CHAR(SUM(amount_sold), '9,999,999,999')SALES$, RANK() OVER
(PARTITION BY GROUPING_ID(channel_desc, country_iso_code)ORDER BY SUM(amount_sold) DESC) AS RANK_PER_GROUP
FROM sales, customers, times, channels, countriesWHERE sales.time_id = times.time_id AND
sales.cust_id=customers.cust_id AND sales.channel_id = channels.channel_id AND channels.channel_desc IN ('Direct Sales', 'Internet') AND
times.calendar_month_desc='2000-09' AND country_iso_code IN ('GB', 'US', 'JP')
GROUP BY CUBE(channel_desc, country_iso_code);
![Page 63: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/63.jpg)
71
• Uses rank values in its numerator and returns the percent rank of a value relative to a group of values
• PERCENT_RANK of a row is calculated as follows:
• The range of values returned by PERCENT_RANK is 0 to 1, inclusive. The first row in any set has a PERCENT_RANK of 0. The return value is NUMBER. Its syntax is:
Using the PERCENT_RANK Function
(rank of row in its partition - 1) / (number of rows in the partition - 1)
PERCENT_RANK () OVER ([query_partition_clause] order_by_clause)
![Page 64: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/64.jpg)
72
Using the PERCENT_RANK Function: Example
SELECT department_id, last_name, salary, PERCENT_RANK() OVER (PARTITION BY department_id ORDER BY salary DESC) AS pr
FROM hr.employeesORDER BY department_id, pr, salary;
. . .
![Page 65: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/65.jpg)
73
• The ROW_NUMBER function calculates a sequential number of a value in a group of values.
• When using the ROW_NUMBER function, ascending is the default sort order, which you can change to descending.
• Rows with equal values for the ranking criteria receive a different number.
Working with the ROW_NUMBER Function
ROW_NUMBER ( ) OVER ( [query_partition_clause] order_by_clause )
![Page 66: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/66.jpg)
7474
• ROWNUM is a pseudo column, ROW_NUMBERis an actual function
• ROWNUM requires sorting of the entire dataset in order to return ordered list
• ROW_NUMBER will only sort the required rows thus giving better performance
ROW_NUMBER vs. ROWNUM
![Page 67: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/67.jpg)
7575
• Not really a rank function
• Divides an ordered data set into a number of buckets indicated by expr and assigns the appropriate bucket number to each row
• The buckets are numbered 1 through expr
Working with the NTILE Function
NTILE ( expr ) OVER ([query_partition_clause] order_by_clause)
![Page 68: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/68.jpg)
7676
• Different ranking functions may return different results if the data has ties
• For example:
Summary of Ranking Functions
SELECT last_name, salary,ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC),RANK() OVER (PARTITION BY department_id ORDER BY salary DESC),DENSE_RANK() OVER (PARTITION BY department_id ORDER BY salary DESC),PERCENT_RANK() OVER (PARTITION BY department_id ORDER BY salary DESC),NTILE(4) OVER (PARTITION BY department_id ORDER BY salary DESC)
FROM hr.employees;
![Page 69: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/69.jpg)
Inter-row Analytic Functions
77
![Page 70: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/70.jpg)
78
• LAG provides access to more than one row of a table at the same time without a self-join.
• Given a series of rows returned from a query and a position of the cursor, LAG provides access to a row at a given physical offset before that position.
• If you do not specify the offset, its default is 1. • If the offset goes beyond the scope of the window, the
optional default value is returned. If you do not specify the default, its value is NULL.
Using the LAG and LEAD Analytic Functions
{LAG | LEAD}(value_expr [, offset ] [, default ])OVER ([ query_partition_clause ] order_by_clause)
![Page 71: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/71.jpg)
79
Using the LAG and LEAD Analytic Functions: Example
SELECT time_id, TO_CHAR(SUM(amount_sold),'9,999,999') AS SALES,
TO_CHAR(LAG(SUM(amount_sold),1) OVER (ORDER BY time_id),'9,999,999') AS LAG1,
TO_CHAR(LEAD(SUM(amount_sold),1) OVER (ORDER BY time_id),'9,999,999') AS LEAD1
FROM salesWHERE time_id >= TO_DATE('10-OCT-2000') AND
time_id <= TO_DATE('14-OCT-2000')GROUP BY time_id;
![Page 72: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/72.jpg)
8080
• For a specified measure, LISTAGG orders data within each group specified in the ORDER BY clause and then concatenates the values of the measure column
• Limited to output of 4000 chars
Using the LISTAGG FunctionLISTAGG(measure_expr [, 'delimiter'])
WITHIN GROUP (order_by_clause) [OVER query_partition_clause]
![Page 73: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/73.jpg)
8181
Using the LISTAGG Function Example
SELECT department_id "Dept", hire_date "Date",
last_name "Name",
LISTAGG(last_name, ', ') WITHIN GROUP (ORDER BY
hire_date, last_name)
OVER (PARTITION BY department_id) as "Emp_list"
FROM hr.employees
WHERE hire_date < '01-SEP-2003'
ORDER BY "Dept", "Date", "Name";
![Page 74: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/74.jpg)
8282
• Both are aggregate and analytic functions • Used to retrieve a value from the first or last row
of a sorted group, but the needed value is not the sort key
• FIRST and LAST functions eliminate the need for self-joins or views and enable better performance
Using the FIRST and LAST Functionsaggregate_function KEEP
(DENSE_RANK FIRST ORDER BYexpr [ DESC | ASC ][ NULLS { FIRST | LAST } ][, expr [ DESC | ASC ] [ NULLS { FIRST | LAST } ]]...
)[ OVER query_partition_clause ]
![Page 75: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/75.jpg)
8383
FIRST and LAST Aggregate Example
SELECT department_id,
MIN(salary) KEEP (DENSE_RANK FIRST ORDER BY commission_pct)
"Worst",
MAX(salary) KEEP (DENSE_RANK LAST ORDER BY commission_pct)
"Best"
FROM employees
GROUP BY department_id
ORDER BY department_id;
![Page 76: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/76.jpg)
8484
FIRST and LAST Analytic Example
SELECT last_name, department_id, salary,
MIN(salary) KEEP (DENSE_RANK FIRST ORDER BY
commission_pct) OVER (PARTITION BY department_id)
“Worst",
MAX(salary) KEEP (DENSE_RANK LAST ORDER BY commission_pct)
OVER (PARTITION BY department_id) "Best"
FROM employees
ORDER BY department_id, salary, last_name;
![Page 77: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/77.jpg)
8585
• Returns the first value in an ordered set of values
• If the first value in the set is null, then the function returns NULL unless you specify IGNORE NULLS. This setting is useful for data densification.
Using FIRST_VALUE Analytic Function
FIRST_VALUE (expr [ IGNORE NULLS ]) OVER (analytic_clause)
![Page 78: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/78.jpg)
8686
Using FIRST_VALUE Analytic FunctionExample
SELECT department_id, last_name, salary,
FIRST_VALUE(last_name) OVER (ORDER BY salary ASC ROWS
UNBOUNDED PRECEDING) AS lowest_sal
FROM (SELECT * FROM employees WHERE department_id = 30
ORDER BY employee_id)
ORDER BY department_id, last_name, salary, lowest_sal;
![Page 79: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/79.jpg)
8787
• Returns the last value in an ordered set of values.
Using LAST_VALUE Analytic Function
LAST_VALUE (expr [ IGNORE NULLS ]) OVER (analytic_clause)
![Page 80: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/80.jpg)
8888
• Returns the N-th values in an ordered set of values
• Different default window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
ROW
Using NTH_VALUE Analytic Function
NTH_VALUE (measure_expr, n)[ FROM { FIRST | LAST } ][ { RESPECT | IGNORE } NULLS ] OVER (analytic_clause)
![Page 81: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/81.jpg)
8989
Using NTH_VALUE Analytic FunctionExample
SELECT prod_id, channel_id, MIN(amount_sold),
NTH_VALUE ( MIN(amount_sold), 2) OVER (PARTITION BY
prod_id ORDER BY channel_id
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED
FOLLOWING) nv
FROM sh.sales
WHERE prod_id BETWEEN 13 and 16
GROUP BY prod_id, channel_id;
![Page 82: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/82.jpg)
Window Functions
![Page 83: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/83.jpg)
9191
• The windowing_clause gives some analytic functions a further degree of control over this window within the current partition
• The windowing_clause can only be used if an order_by_clause is present
Window Functions
![Page 84: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/84.jpg)
9292
Windows can be by RANGE or ROWS
Possible values for start_point and end_point
UNBOUNDED PRECEDING The window starts at the first row of the partition. Only available for start points.
UNBOUNDED FOLLOWING The window ends at the last row of the partition. Only available for end points.
CURRENT ROW The window starts or ends at the current row
value_expr PRECEDING A physical or logical offset before the current row.When used with RANGE, can also be an interval literal
value_expr FOLLOWING As above, but an offset after the current row
RANGE BETWEEN start_point AND end_point
ROWS BETWEEN start_point AND end_point
![Page 85: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/85.jpg)
9393
• The windows are limited to the current partition
• Generally, the default window is the entire work set unless said otherwise
Window Limitations
![Page 86: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/86.jpg)
9494
Shortcuts
• Useful shortcuts for the windowing clause:
ROWS UNBOUNDED PRECEDING ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
ROWS 10 PRECEDING ROWS BETWEEN 10 PRECEDING AND CURRENT ROW
ROWS CURRENT ROW ROWS BETWEEN CURRENT ROW AND CURRENT ROW
![Page 87: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/87.jpg)
9595
• Cumulative aggregation
• Sliding average over proceeding and/or following rows
• Using the RANGE parameter to filter aggregation records
Windowing Clause Useful Usages
![Page 88: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/88.jpg)
Top-N and Paging QueriesIn Oracle 12c
96
![Page 89: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/89.jpg)
97
Top-N Queries
• A Top-N query is used to retrieve the top or bottom N rows from an ordered set
• Combining two Top-N queries gives you the ability to page through an ordered set
• Oracle 12c has introduced the row limiting clause to simplify Top-N queries
![Page 90: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/90.jpg)
9898
• This is ANSI syntax
• The default offset is 0
• Null values in offset, rowcount or percentwill return no rows
Top-N in 12c
[ OFFSET offset { ROW | ROWS } ][ FETCH { FIRST | NEXT } [ { rowcount | percent PERCENT } ]
{ ROW | ROWS } { ONLY | WITH TIES } ]
![Page 91: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/91.jpg)
9999
Top-N Examples
SELECT last_name, salaryFROM hr.employeesORDER BY salaryFETCH FIRST 4 ROWS ONLY;
SELECT last_name, salaryFROM hr.employeesORDER BY salaryFETCH FIRST 4 ROWS WITH TIES;
SELECT last_name, salaryFROM hr.employeesORDER BY salary DESCFETCH FIRST 10 PERCENT ROWS ONLY;
![Page 92: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/92.jpg)
100100
• Before 12c we had to use the rownum pseudo column to filter out rows
• That will require sorting the entire rowset
Paging Before 12c
SELECT valFROM (SELECT val, rownum AS rnum
FROM (SELECT valFROM rownum_order_testORDER BY val)
WHERE rownum <= 10)WHERE rnum >= 5;
![Page 93: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/93.jpg)
101101
• After 12c we have a syntax improvement for paging using the Top-N queries
• This will use ROW_NUMBER and RANK in the background – there is no real optimization improvements
Paging in Oracle 12c
SELECT valFROM rownum_order_testORDER BY valOFFSET 4 ROWS FETCH NEXT 5 ROWS ONLY;
![Page 94: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/94.jpg)
102102
• More information and examples could be found on my blog:
http://www.realdbamagic.com/he/12c-top-n-query/
More Examples
![Page 95: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/95.jpg)
103103
• Analytic functions has positive impact on performance for the most part
• Using analytic functions can reduce the number of table scans and reduce IO consumption
• The query might use more CPU and/or memory but it will usually run faster than the same result without analytic functions
• Top-N queries might struggle with cardinality evaluation when using the “With Ties” option
Analytic Functions and Performance
![Page 96: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/96.jpg)
Hierarchical Queries
104
![Page 97: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/97.jpg)
105
• You can use hierarchical queries to retrieve data based on a natural hierarchical relationship between rows in a table.
• A relational database does not store records in a hierarchical way; therefore, a hierarchical query is possible only when a relationship exists between rows in a table.
• However, where a hierarchical relationship exists between the rows of a single table, a process called “tree walking” enables the hierarchy to be constructed.
• A hierarchical query is a method of reporting, with the branches of a tree in a specific order.
Using Hierarchical Queries
![Page 98: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/98.jpg)
106106
• Getting all employees that report directly or indirectly to a manager
• Managing documents and folders
• Managing privileges
• Aggregating levels on the same row
Business Challenges
![Page 99: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/99.jpg)
Using Hierarchical Queries: ExampleSample Data from the EMPLOYEES Table
• In the EMPLOYEES table in the HR schema, Kochhar, De Haan, and Hartstein report to the MANAGER_ID 100, which is King’s EMPLOYEE_ID.
…
![Page 100: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/100.jpg)
108
Natural Tree Structure
De Haan
HunoldWhalen
Kochhar
Higgins
Mourgos Zlotkey
Rajs Davies Matos
Gietz Ernst Lorentz
Hartstein
Fay
Abel Taylor Grant
Vargas
MANAGER_ID = 100 (Child)
EMPLOYEE_ID = 100 (Parent)
. . . . . .
. . .
. . .
. . .
King
![Page 101: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/101.jpg)
109
condition:
Hierarchical Queries: Syntax
expr comparison_operator expr
SELECT [LEVEL], column, expr...FROM table
[WHERE condition(s)]
[START WITH condition(s)]
[CONNECT BY PRIOR condition(s)] ;
![Page 102: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/102.jpg)
110
– Use the START WITH clause to specify the starting point, that is, the row or rows to be used as the root of the tree:
• Specifies the condition that must be met
• Accepts any condition that is valid in a WHERE clause
– For example, using the HR.EMPLOYEES table, start with the employee whose last name is Kochhar.
Walking the Tree: Specifying the Starting Point
. . .START WITH last_name = 'Kochhar'
START WITH column1 = value
![Page 103: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/103.jpg)
111
• The direction of the query is determined by the CONNECT BY PRIOR column placement.
• The PRIOR operator refers to the parent row.
• For example, you can walk the tree from top down using the EMPLOYEES table as follows:
Walking the Tree:Specifying the Direction of the Query
CONNECT BY PRIOR column1 = column2
. . . CONNECT BY PRIOR employee_id = manager_id. . .
Parent key Child key
![Page 104: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/104.jpg)
112
Hierarchical Query Example: Using the CONNECT BY Clause
SELECT employee_id, last_name, manager_idFROM hr.employeesCONNECT BY PRIOR employee_id = manager_id;
. . .
![Page 105: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/105.jpg)
113
Specifying the Direction of the Query: From the Top Down
SELECT last_name||' reports to '|| PRIOR last_name "Walk Top Down"
FROM hr.employees
START WITH last_name = 'King'
CONNECT BY PRIOR employee_id = manager_id ;
. . .
![Page 106: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/106.jpg)
114
Specifying the Direction of the Query: From the Bottom Up
SELECT employee_id, last_name, job_id, manager_id
FROM hr.employees
START WITH employee_id = 101
CONNECT BY PRIOR manager_id = employee_id ;
![Page 107: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/107.jpg)
115
Using the LEVEL Pseudocolumn
Level 1root/
parent
Level 3parent/
child/leaf
Level 4leaf
De Haan
King
HunoldWhalen
Kochhar
Higgins
Mourgos Zlotkey
Rajs Davies Matos
Gietz Ernst Lorentz
Hartstein
Fay
Abel Taylor Grant
Vargas
Level 2parent/child
![Page 108: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/108.jpg)
116
Using the LEVEL Pseudocolumn: ExampleSELECT employee_id, last_name, manager_id, LEVELFROM hr.employeesSTART WITH employee_id = 100CONNECT BY PRIOR employee_id = manager_idORDER siblings BY last_name;
. . .
![Page 109: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/109.jpg)
117
• Create a report displaying company management levels beginning with the highest level and indenting each of the following levels.
Formatting Hierarchical Reports Using LEVEL and LPAD
SELECT LPAD(last_name, LENGTH(last_name)+(LEVEL*2)-2,'_') AS org_chartFROM hr.employeesSTART WITH first_name = 'Steven' AND last_name = 'King' CONNECT BY PRIOR employee_id = manager_idORDER SIBLINGS BY last_name;
![Page 110: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/110.jpg)
118
– Use the WHERE clause to eliminate a node.
– Use the CONNECT BY clause to eliminate a branch.
Pruning Nodes and Branches
Kochhar
Higgins
Gietz
Whalen
Kochhar
HigginsWhalen
Gietz
. . .WHERE last_name != 'Higgins'
. . .CONNECT BY PRIOR employee_id = manager_idAND last_name != 'Higgins'
1 2
![Page 111: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/111.jpg)
119
Pruning Branches Example 1:Eliminating a Node
SELECT department_id, employee_id,last_name, job_id, salary
FROM hr.employeesWHERE last_name != 'Higgins'START WITH manager_id IS NULLCONNECT BY PRIOR employee_id = manager_id;
. . .
. . .
. . .
![Page 112: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/112.jpg)
120
Pruning Branches Example 2:Eliminating a Branch
SELECT department_id, employee_id,last_name, job_id, salaryFROM hr.employeesSTART WITH manager_id IS NULLCONNECT BY PRIOR employee_id = manager_idAND last_name != 'Higgins';
. . .
![Page 113: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/113.jpg)
121121
• Join happens before connect by
• Where is happening after connect by
• Regular order by will rearrange the returning rows
• Sibling order by will rearrange the returning rows for each level
Order of Precedence
![Page 114: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/114.jpg)
122122
Other Connect By Functions
• CONNECT_BY_ISCYCLE
• CONNECT_BY_ISLEAF
• CONNECT_BY_ROOT
• SYS_CONNECT_BY_PATH
![Page 115: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/115.jpg)
123123
• ANSI SQL:2008 (Oracle 11g) introduced a new way to run hierarchical queries: Recursive Subquery Factoring (RSF)
• Using the with clause, making queries easier to write
New Syntax: Recursive Subquery Factoring
![Page 116: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/116.jpg)
124124
Recursive Subquery Factoring Example
with mytree(id, parent_id, "level")as (
select id, parent_id, 1 as "level"from temp_vwhere id = 1
union allselect temp_v.id, temp_v.parent_id,
mytree."level" + 1from temp_v, mytreewhere temp_v.parent_id = mytree.id
)Select * from mytree;
Stop Condition
Actual Recursion
![Page 117: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/117.jpg)
125125
• Recursion and Hierarchies might have bad impact on performance
• Watch out for mega-trees – it has CPU and memory impact
• Using recursion might lead for multiple IO reads of the same blocks
Warning: Recursion might be Bad for Performance
![Page 118: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/118.jpg)
Pattern Matching inOracle 12c
126
![Page 119: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/119.jpg)
127127
• Identify and group rows with consecutive values
• Consecutive in this regards – row after row
• Uses regular expression like syntax to find patterns
What is Pattern Matching
![Page 120: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/120.jpg)
128128
• Finding sequences of events in security applications
• Locating dropped calls in a CDR listing
• Financial price behaviors (V-shape, W-shape U-shape, etc.)
• Fraud detection and sensor data analysis
Common Business Challenges
![Page 121: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/121.jpg)
129129
MATCH_RECOGNIZE Syntax
SELECTFROM [row pattern input table]MATCH_RECOGNIZE( [ PARTITION BY <cols> ][ ORDER BY <cols> ][ MEASURES <cols> ][ ONE ROW PER MATCH | ALL ROWS PER MATCH ][ SKIP_TO_option]PATTERN ( <row pattern> )DEFINE <definition list>)
![Page 122: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/122.jpg)
130130
• PARTITION BY divides the data in to logical groups
• ORDER BY orders the data in each logical group
• MEASURES define the data measures of the pattern
• ONE/ALL ROW PER MATCH defines what to do with the pattern – return one row or all rows
• PATTERN says what the pattern actually is
• DEFINE gives us the condition that must be met for a row to map to the pattern variables
Basix Syntax Legend
![Page 123: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/123.jpg)
131131
• Find Simple V-Shape with 1 row output per match
MATCH_RECOGNIZE Example
SELECT *FROM Ticker MATCH_RECOGNIZE (
PARTITION BY symbolORDER BY tstampMEASURES STRT.tstamp AS start_tstamp,
LAST(DOWN.tstamp) AS bottom_tstamp,LAST(UP.tstamp) AS end_tstamp
ONE ROW PER MATCHAFTER MATCH SKIP TO LAST UPPATTERN (STRT DOWN+ UP+)DEFINE
DOWN AS DOWN.price < PREV(DOWN.price),UP AS UP.price > PREV(UP.price)
) MRORDER BY MR.symbol, MR.start_tstamp;
![Page 124: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/124.jpg)
132132
What Will Be Matched?
![Page 125: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/125.jpg)
133133
• Our goal: find uninterrupted sequences in a book
• This can be useful for detecting missing records or sequential behavior
Pages in a Book Example
![Page 126: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/126.jpg)
134
Building Our Query
1. Define input
2. Pattern Matching
3. Order input
4. Process pattern
5. using defined conditions
6. Output: rows per match
7. Output: columns per row
8. Where to go after match?
SELECT *
FROM book_pages
MATCH_RECOGNIZE (
ORDER BY page
PATTERN (A B*)
DEFINE B AS page = PREV(page)+1
ONE ROW PER MATCH
MEASURES
A.page firstpage,
LAST(page) lastpage,
COUNT(*) cnt
AFTER MATCH SKIP PAST LAST ROW
);
SELECT *
FROM book_pages
MATCH_RECOGNIZE (
ORDER BY page
MEASURES
A.page firstpage,
LAST(page) lastpage,
COUNT(*) cnt
ONE ROW PER MATCH
AFTER MATCH SKIP PAST LAST ROW
PATTERN (A B*)
DEFINE B AS page = PREV(page)+1
);
![Page 127: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/127.jpg)
135135135135
And The Output…
FIRSTPAGE LASTPAGE CNT---------- ---------- ----------
1 3 35 7 310 15 642 42 1
![Page 128: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/128.jpg)
136136136136
• Concatenation: No operator between elements.• Quantifiers:
– * 0 or more matches.– + 1 or more matches– ? 0 or 1 match.– {n} Exactly n matches.– {n,} n or more matches.– {n, m} Between n and m (inclusive) matches.– {, m} Between 0 an m (inclusive) matches.
• Alternation: |• Grouping: ()
Supported Regular Expression Patterns
![Page 129: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/129.jpg)
137137
• CLASSIFIER(): Which pattern variable applies to which row
• MATCH_NUMBER(): Which rows are members of which match
• PREV(): Access to a column/expression in a previous row
• NEXT(): Access to a column/expression in the next row
• LAST(): Last value within the pattern match
• FIRST(): First value within the pattern match
• COUNT(), AVG(), MAX(), MIN(), SUM()
Functions
![Page 130: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/130.jpg)
138138
• Find suspicious transfers – a large transfer after 3 small ones
Example: All Rows Per Match
SELECT userid, match_id, pattern_variable, time, amountFROM (SELECT * FROM event_logWHERE event = 'transfer')MATCH_RECOGNIZE(PARTITION BY userid ORDER BY timeMEASURESMATCH_NUMBER() match_id,CLASSIFIER() pattern_variableALL ROWS PER MATCHPATTERN ( x{3,} y)DEFINEx AS (amount < 2000 AND LAST(x.time) -FIRST(x.time) < 30),y AS (amount >= 1000000 AND y.time-LAST(x.time) < 10));
![Page 131: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/131.jpg)
139139
• MATCH_ID shows current match sequence
• PATTERN_VARIABLE show which variable was applied
• USERID is the partition key
The Output
USERID MATCH_ID PATTERN_VA TIME AMOUNT-------- ---------- ---------- --------- ----------john 1 X 06-JAN-12 1000john 1 X 15-JAN-12 1500john 1 X 20-JAN-12 1500john 1 X 23-JAN-12 1000john 1 Y 26-JAN-12 1000000
![Page 132: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/132.jpg)
140140
• Same as before – show one row per match
Example: One Row Per Match
SELECT userid, first_trx, last_trx, amountFROM (SELECT * FROM event_log WHERE event = 'transfer')MATCH_RECOGNIZE(PARTITION BY userid ORDER BY timeMEASURESFIRST(x.time) first_trx,y.time last_trx,y.amount amountONE ROW PER MATCHPATTERN ( x{3,} y )DEFINEx AS (amount < 2000 AND LAST(x.time) -FIRST(x.time) < 30),y AS (amount >= 1000000 AND y.time-LAST(x.time) < 10));
![Page 133: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/133.jpg)
141141
• USERID is the partition key
• FIRST_TRX is a calculated measure
• AMOUNT and LAST_TRX are measures
The Output
USERID FIRST_TRX LAST_TRX AMOUNT-------- --------- --------- ----------john 06-JAN-12 26-JAN-12 1000000
![Page 134: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/134.jpg)
142142
• Test all cases: pattern matching can be very tricky
• Don’t forget to test your data with no matches
• There is no LISTAGG and no DISTINCT when using match recognition
• Pattern variables cannot be used as bind variables
Few Last Tips
![Page 135: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/135.jpg)
Using XML with SQL
143
![Page 136: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/136.jpg)
144144
• XML stand for Extensible Markup Language
• Defines a set of rules for encoding documents in a format which is both human-readable and machine-readable
• Data is unstructured and can be transferred easily to other system
What is XML
![Page 137: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/137.jpg)
145145
• Root
• Element
• Attribute
• Forest
• XML Fragment
• XML Document
XML Terminology
![Page 138: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/138.jpg)
146146
What Does XML Look Like?
<?xml version="1.0"?><ROWSET><ROW><USERNAME>SYS</USERNAME><USER_ID>0</USER_ID><CREATED>28-JAN-08</CREATED>
</ROW><ROW>
<USERNAME>SYSTEM</USERNAME><USER_ID>5</USER_ID><CREATED>28-JAN-08</CREATED>
</ROW></ROWSET>
![Page 139: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/139.jpg)
147147
• Concatenating strings – building the XML manually. This is highly not recommended
• Using DBMS_XMLGEN
• Using ANSI SQL:2003 XML functions
Generating XML From Oracle
![Page 140: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/140.jpg)
148148
• The DBMS_XMLGEN package converts the results of a SQL query to a canonical XML format
• The package takes an arbitrary SQL query as input, converts it to XML format, and returns the result as a CLOB
• Using the DBMS_XMLGEN we can create contexts and use it to build XML documents
• Old package – exists since Oracle 9i
Using DBMS_XMLGEN
![Page 141: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/141.jpg)
149149
Example of Using DBMS_XMLGENselect dbms_xmlgen.getxml(q'{select column_name, data_typefrom all_tab_columnswhere table_name = 'EMPLOYEES' and owner = 'HR'}')from dual/
<?xml version="1.0"?><ROWSET><ROW><COLUMN_NAME>EMPLOYEE_ID</COLUMN_NAME><DATA_TYPE>NUMBER</DATA_TYPE>
</ROW><ROW><COLUMN_NAME>FIRST_NAME</COLUMN_NAME><DATA_TYPE>VARCHAR2</DATA_TYPE>
</ROW>[...]</ROWSET>
![Page 142: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/142.jpg)
150150
• DBMS_XMLGEN is an old package (9.0 and 9i)
• Any context change requires complex PL/SQL
• There are improved ways to use XML in queries
• Use DBMS_XMLGEN for the “quick and dirty” solution only
Why Not Use DBMS_XMLGEN
![Page 143: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/143.jpg)
151151
• Introduced in ANSI SQL:2003 – Oracle 9iR2 and 10gR2
• Standard functions that can be integrated into queries
• Removes the need for PL/SQL code to create XML documents
Standard XML Functions
![Page 144: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/144.jpg)
152152
XMLELEMENT The basic unit for turning column data into XML fragments
XMLATTRIBUTES Converts column data into attributes of the parent element
XMLFOREST Allows us to process multiple columns at once
XMLAGG Aggregate separate Fragments into a single fragment
XMLROOT Allows us to place an XML tag at the start of our XML document
XML Functions
![Page 145: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/145.jpg)
153153
XMLELEMENT
SELECT XMLELEMENT("name", e.last_name) AS employeeFROM employees eWHERE e.employee_id = 202;
EMPLOYEE------------------------------<name>Fay</name>
![Page 146: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/146.jpg)
154154
XMLELEMENT (2)
SELECT XMLELEMENT("employee",XMLELEMENT("works_number", e.employee_id),XMLELEMENT("name", e.last_name)
) AS employeeFROM employees eWHERE e.employee_id = 202;
EMPLOYEE----------------------------------------------------------<employee><works_number>202</works_number><name>Fay</name></employee>
![Page 147: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/147.jpg)
155155
XMLATTRIBUTES
SELECT XMLELEMENT("employee",XMLATTRIBUTES(e.employee_id AS "works_number",e.last_name AS "name")
) AS employeeFROM employees eWHERE e.employee_id = 202;
EMPLOYEE----------------------------------------------------------<employee works_number="202" name="Fay"></employee>
![Page 148: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/148.jpg)
156156
XMLFOREST
SELECT XMLELEMENT("employee",XMLFOREST(e.employee_id AS "works_number",e.last_name AS "name",e.phone_number AS "phone_number")
) AS employeeFROM employees eWHERE e.employee_id = 202;
EMPLOYEE----------------------------------------------------------<employee><works_number>202</works_number><name>Fay</name><phone_number>603.123.6666</phone_number></employee>
![Page 149: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/149.jpg)
157157
XMLFOREST Problem
SELECT XMLELEMENT("employee",XMLFOREST(e.employee_id AS "works_number",e.last_name AS "name",e.phone_number AS "phone_number")
) AS employeeFROM employees eWHERE e.employee_id in (202, 203);
EMPLOYEE----------------------------------------------------------<employee><works_number>202</works_number><name>Fay</name><phone_number>603.123.6666</phone_number></employee><employee><works_number>203</works_number><name>Mavris</name><phone_number>515.123.7777</phone_number></employee>
2 row selected.
![Page 150: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/150.jpg)
158158
XMLAGG
SELECT XMLAGG(XMLELEMENT("employee",
XMLFOREST(e.employee_id AS "works_number",e.last_name AS "name",e.phone_number AS "phone_number")
)) AS employeeFROM employees eWHERE e.employee_id in (202, 203);
EMPLOYEE----------------------------------------------------------<employee><works_number>202</works_number><name>Fay</name><phone_number>603.123.6666</phone_number></employee><employee><works_number>203</works_number><name>Mavris</name><phone_number>515.123.7777</phone_number></employee>
1 row selected.
![Page 151: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/151.jpg)
159159
• Creating a well formed XML document
XMLROOT
SELECT XMLROOT (XMLELEMENT("employees",XMLAGG(XMLELEMENT("employee",
XMLFOREST(e.employee_id AS "works_number",e.last_name AS "name",e.phone_number AS "phone_number")
))), VERSION '1.0') AS employeeFROM employees eWHERE e.employee_id in (202, 203);
![Page 152: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/152.jpg)
160160
• Well formed, version bound, beatified XML:
XMLROOT
EMPLOYEE------------------------------------------<?xml version="1.0"?><employees><employee><works_number>202</works_number><name>Fay</name><phone_number>603.123.6666</phone_number>
</employee><employee><works_number>203</works_number><name>Mavris</name><phone_number>515.123.7777</phone_number>
</employee></employees>
![Page 153: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/153.jpg)
161161
• Using the XQuery language we can create, read and manipulate XML documents
• Two main functions: XMLQuery and XMLTable
• XQuery is about sequences - XQuery is a general sequence-manipulation language
• Each sequence can contain numbers, strings, Booleans, dates, or other XML fragments
Using XQuery
![Page 154: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/154.jpg)
162
Creating XML Document using XQuery
SELECT warehouse_name,EXTRACTVALUE(warehouse_spec, '/Warehouse/Area'),XMLQuery(
'for $i in /Warehousewhere $i/Area > 50000return <Details>
<Docks num="{$i/Docks}"/><Rail>
{if ($i/RailAccess = "Y") then "true" else
"false"}
</Rail></Details>' PASSING warehouse_spec RETURNING CONTENT)
"Big_warehouses"FROM warehouses;
![Page 155: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/155.jpg)
163
Creating XML Document using XQuery
WAREHOUSE_ID Area Big_warehouses------------ --------- --------------------------------------------------------
1 250002 500003 85700 <Details><Docks></Docks><Rail>false</Rail></Details>4 103000 <Details><Docks num="3"></Docks><Rail>true</Rail></Details>
. . .
![Page 156: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/156.jpg)
164164
Example: Using XMLTable to Read XML
SELECT lines.lineitem, lines.description, lines.partid,lines.unitprice, lines.quantity
FROM purchaseorder,XMLTable('for $i in /PurchaseOrder/LineItems/LineItem
where $i/@ItemNumber >= 8and $i/Part/@UnitPrice > 50and $i/Part/@Quantity > 2
return $i'PASSING OBJECT_VALUECOLUMNS lineitem NUMBER PATH '@ItemNumber',
description VARCHAR2(30) PATH 'Description',partid NUMBER PATH 'Part/@Id',unitprice NUMBER PATH 'Part/@UnitPrice',quantity NUMBER PATH 'Part/@Quantity')
lines;
![Page 157: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/157.jpg)
Oracle 12c JSON Support
165
![Page 158: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/158.jpg)
166166
• Javascript Object Notation
• Converts database tables to a readable document – just like XML but simplier
• Very common in NoSQL and Big Data solutions
What is JSON
{"FirstName" : "Zohar","LastName" : "Elkayam","Age" : 36,"Connection" :[
{"Type" : “Email", "Value" : "[email protected]"},{"Type" : “Twitter", "Value" : “@realmgic"},{"Type" : "Site", "Value" : "www.realdbamagic.com"},
]}
![Page 159: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/159.jpg)
167167
• Ability to store data without requiring a Schema
– Store semi-structured data in its native (aggregated) form
• Ability to query data without knowledge of Schema
• Ability to index data with knowledge of Schema
JSON Benefits
![Page 160: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/160.jpg)
168168
• Oracle supports JSON since version 12.1.0.2
• JSON documents stored in the database using existing data types: VARCHAR2, CLOB or BLOB
• External JSON data sources accessible through external tables including HDFS
• Data accessible via REST API
Oracle JSON Support
![Page 161: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/161.jpg)
169169
• Simple well understood model
• CRUD operations are mapped to HTTP Verbs
– Create / Update : PUT / POST
– Retrieve : GET
– Delete : DELETE
– QBE, Bulk Update, Utilitiy functions : POST
• Stateless
REST based API for JSON documents
![Page 162: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/162.jpg)
170170
• Similar role to XPATH in XML
• Syntactically similar to Java Script (. and [ ])
• Compatible with Java Script
JSON Path Expression
![Page 163: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/163.jpg)
171171
• There are few common JSON Operators:
Common JSON SQL Functions
JSON_EXISTS Checks if a value exists in the JSON
JSON_VALUE Retrieve a scalar value from JSON
JSON_QUERY Query a string from JSON Document
JSON_TABLE Query data from JSON Document (like XMLTable)
![Page 164: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/164.jpg)
172172
• Extract JSON fragment from JSON document
JSON_QUERY
select count(*)from J_PURCHASEORDER
where JSON_EXISTS(PO_DOCUMENT, '$.ShippingInstructions.Address.state‘)
/
![Page 165: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/165.jpg)
173173
• Generate rows from a JSON Array
• Pivot properties / key values into columns
• Use Nested Path clause to process multi-level collections with a single JSON_TABLE operator.
Using JSON_TABLE
![Page 166: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/166.jpg)
174174
• 1 Row of output for each row in table
Example: JSON_TABLE
select M.*from J_PURCHASEORDER p,
JSON_TABLE(p.PO_DOCUMENT,'$'columnsPO_NUMBER NUMBER(10) path '$.PONumber',REFERENCE VARCHAR2(30 CHAR) path '$.Reference',REQUESTOR VARCHAR2(32 CHAR) path '$.Requestor',USERID VARCHAR2(10 CHAR) path '$.User',COSTCENTER VARCHAR2(16) path '$.CostCenter'
) Mwhere PO_NUMBER > 1600 and PO_Number < 1605/
![Page 167: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/167.jpg)
175175
• 1 row output for each member of LineItems array ``
Example: JSON_TABLE (2)
select D.*from J_PURCHASEORDER p,
JSON_TABLE(p.PO_DOCUMENT,'$'columns(PO_NUMBER NUMBER(10) path '$.PONumber',NESTED PATH '$.LineItems[*]'columns(ITEMNO NUMBER(16) path '$.ItemNumber',UPCCODE VARCHAR2(14 CHAR) path '$.Part.UPCCode‘ ))
) Dwhere PO_NUMBER = 1600 or PO_NUMBER = 1601/
![Page 168: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/168.jpg)
176176
• Known Query Patterns : JSON Path expression
– Functional indexes using JSON_VALUE and, JSON_EXISTS
– Materialized View using JSON_TABLE()
• Ad-hoc Query Strategy
– Based on Oracle’s full text index (Oracle Text)
– Support ad-hoc path, value and keyword query search using JSON Path expressions
JSON Indexing
![Page 169: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/169.jpg)
Regular Expression
177
![Page 170: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/170.jpg)
178178
• Regular expression (regexp) is a sequence of characters that define a search pattern
• Commonly used for smart “Search and Replace” of patterns and for input validations of text
• Widely introduced in Oracle 10g (and it even existed even before that)
Regular Expression
![Page 171: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/171.jpg)
179179
Common REGEXP Functions and Operators
REGEXP_LIKE Perform regular expression matching
REGEXP_REPLACE Extends the functionality of the REPLACEfunction by using patterns
REGEXP_SUBSTR Extends the functionality of the SUBSTRfunction by using patterns
REGEXP_COUNT Count the number of matches of the pattern in a given string
REGEXP_INSTR Extends the functionality of the INSTRfunction by using patterns
![Page 172: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/172.jpg)
180180180180
• Concatenation: No operator between elements.• Quantifiers:
– . Matches any character in the database character set– * 0 or more matches– + 1 or more matches– ? 0 or 1 match– {n} Exactly n matches– {n,} n or more matches– {n, m} Between n and m (inclusive) matches– {, m} Between 0 an m (inclusive) matches
• Alternation: [|]• Grouping: ()
Supported Regular Expression Patterns
![Page 173: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/173.jpg)
181181
Supported Regular Expression Patterns
Value Description
^Matches the beginning of a string. If used with a match_parameter of 'm', it matches the start of a line anywhere within expression.
$Matches the end of a string. If used with a match_parameter of 'm', it matches the end of a line anywhere withinexpression.
\W Matches a nonword character.
\s Matches a whitespace character.
\S matches a non-whitespace character.
\AMatches the beginning of a string or matches at the end of a string before a newline character.
\Z Matches at the end of a string.
![Page 174: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/174.jpg)
182182
Character Class Description
[:alnum:] Alphanumeric characters
[:alpha:] Alphabetic characters
[:blank:] Blank Space Characters
[:cntrl:] Control characters (nonprinting)
[:digit:] Numeric digits
[:graph:] Any [:punct:], [:upper:], [:lower:], and [:digit:] chars
[:lower:] Lowercase alphabetic characters
[:print:] Printable characters
[:punct:] Punctuation characters
[:space:]Space characters (nonprinting), such as carriage return, newline, vertical tab, and form feed
[:upper:] Uppercase alphabetic characters
[:xdigit:] Hexidecimal characters
Character Classes
![Page 175: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/175.jpg)
Regular Expression Demo
183
![Page 176: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/176.jpg)
184184
• Regular expressions might be slow when used on large amount of data
• Writing regular expression can be very tricky –make sure your pattern is correct
• Oracle REGEXP syntax is not standard, regular expression might not work or partially work causing wrong results
• There can only be up to 9 placeholders in a given quantifier
Pitfalls
![Page 177: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/177.jpg)
SQLcl Introduction
The Next Generation of SQL*Plus?
185
![Page 178: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/178.jpg)
186186
• Introduced in Oracle 5 (1985)
• Looks very simple but has tight integration with other Oracle infrastructure and tools
• Very good for reporting, scripting, and automation
• Replaced old CLI tool called …UFI (“User Friendly Interface”)
SQL*Plus
![Page 179: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/179.jpg)
187187
• Nothing really wrong with SQL*Plus – it is being updated constantly but it is missing a lot of functionality
• SQL*Plus forces us to use GUI tools to complete some basic tasks
• Easy to understand, a bit hard to use
• Not easy for new users or developers
What’s Wrong With SQL*Plus?
![Page 180: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/180.jpg)
188188
• SQLcl is a new command line interface (CLI) for SQL users and DBAs
• It is part of the SQL Developer suite – developed by the same team: Oracle Database Development Tools Team
• Can do most of what SQL*Plus does and much more
• Minimal installation, minimal requirements
Introducing SQLcl
![Page 181: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/181.jpg)
189189
• It’s still in the early adopter version (current version: 4.2.0.15.295.1605 RC, October 23, 2015)
• Uses Java, one version for Windows, Linux and OS X
• Planned to be shipped out with Oracle Database 12cR2 (“within the next 12 month”)
Introducing SQLcl (cont.)
![Page 182: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/182.jpg)
190190
• Early Adopter version/RC
• QA still logging bugs from SQL*Plus regression tests
• Adding support for existing SQL*Plus commands and syntax
• Adding new commands
• But can it do...?– Yes
– Not yet
– No
Current Status
![Page 183: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/183.jpg)
191191
• Download from: SQL Developer Download page
• Unzip the file
• Run it
Installing
![Page 184: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/184.jpg)
What Can It Do?
192
![Page 185: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/185.jpg)
193193
• Use the tab key to complete commands
• Can be used to list tables, views or other queriableobjects
• Can be used to replace the * with actual column names
• Use the arrow keys to move around the command
• Use CTRL+W and CTRL+S to jump to the beginning/end of commands
Object Completion and Easy Edit
![Page 186: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/186.jpg)
194194
• 100 command history buffer• Commands are persistent between sessions (watch out for
security!)• Use UP and DOWN arrow keys to access old commands• Usage:
historyhistory usageHistory scripthistory fullHistory clear [session?]
• Load from history into command buffer:history <number>
Command History
![Page 187: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/187.jpg)
195195
• Describe lists the column of the tables just like SQL*Plus
• Information shows column names, default values, indexes and constraints.
• In 12c database information shows table statistics and In memory status
• Works for table, views, sequences, and code objects
• Info+ shows additional information regarding column statistics and column histograms
Describe, Information and info+
![Page 188: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/188.jpg)
196196
• Outputting query results becomes easier with the “set sqlformat” command (also available in SQL Developer)
• We can create a query in the “regular” way and then switch between the different output styles:– ANSIConsole– Fixed column size output– XML or JSON output
– HTML output generates a built in search field and a responsive html output for the result only
Generating Pretty Output
![Page 189: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/189.jpg)
197197
• We can generate loader ready output (with “|” as a delimiter)
• We can generate insert commands
• We can easily generate CSV output
• Usage:set sqlformat { csv,html,xml,json,ansiconsole,insert,loader,fixed,default}
Generating Other Useful Outputs
![Page 190: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/190.jpg)
198198
• Loads a comma separated value (csv) file into a table
• The first row of the file must be a header row and the file must be encoded UTF8
• The load is processed with 50 rows per batch
• Usage:LOAD [schema.]table_name[@db_link] file_name
Load Data From CSV File
![Page 191: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/191.jpg)
199199
• There is a lot in SQL than meets the eye
• Wise use of analytic queries can be good for readability and performance
• Recursive queries are good replacement for the old connect by prior but a little dangerous
• Oracle 12c features are really cool!
• Look out for SQLcl: it’s cool and it’s going places!
Summary
![Page 192: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/192.jpg)
200200
• The Model clause
• Using the WITH clause
• Adding PL/SQL to our SQL (Oracle 12c)
• Hints and other tuning considerations
• The SQL reference book is 1906 pages long. We didn’t talk about most of it…
What Did We Not Talk About?
![Page 193: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/193.jpg)
Q&AAny Questions? Now will be the time!
201
![Page 195: Oracle Database Advanced Querying](https://reader030.fdocuments.in/reader030/viewer/2022020203/587587781a28ab901c8b50a7/html5/thumbnails/195.jpg)
203203203203