Oracle Database 12c: Heat Map, Automatic Data …Days... · Platform Technology Solutions Oracle...
Transcript of Oracle Database 12c: Heat Map, Automatic Data …Days... · Platform Technology Solutions Oracle...
Platform Technology Solutions
Oracle Database Server Technologies
Oracle Database 12c Heat Map, Automatic Data Optimization & In-Database Archiving
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4
Growth in Data Diversity and Usage 1,800 Exabytes of Data in 2011, 20x Growth by 2020
Mobile #1 Internet access device in 2013
Big Data Large customers top 50PB
Enterprise 45% per year growth
in database data
Cloud 80% of new applications
and their data
Regulation 300 exabytes in archives by 2015
Social Business $30B/year in commerce by 2015
Today’s Drivers Emerging Growth Factors
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5
Managing Storage Challenges
Compress data,
without impacting
performance
Manage more data
without incurring
additional cost
Tier and
compress data
based on usage
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6
Information Lifecycle Management Managing Data Over its Lifetime
“The policies, processes,
practices, and tools used to align
the business value of information
with the most appropriate and
cost effective IT infrastructure
from the time information is
conceived through its final
disposition.”
Storage Networking Industry Association
(SNIA) Data Management Forum
$ $$
Total Cost of Ownership (TCO)
$$$
High
Value
Medium
Value
Low
Value
Va
lue
at
Ris
k
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7
Smart Compression
Heat Map
Automated Tiering
In Database Archiving
Network Compression
Automatic Data Optimization Optimize data storage based on usage
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8
Data Compression Reduce storage footprint, read compressed data faster
Hot Data
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 8
111010101010101001101010101011010001011011000110100101000001001110001010101101001011010010110001010010011111001001000010001010101101000
10101010111010100110101
11000010100010110111010
10100101001001000010001
01010110100101101001110
00010100100101000010010
00010001010101110011010
Warm Data
101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001
10101010111010100110101110000101000101
10111010101001010010010000100010101011
01001011010011100001010010010100001001
00001000101010111001101110011000111010
Archive Data
101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001
10101010111010100110101110000101000101101110101
01001010010010000100010101011010010110100111000
01010010010100001001000010001010101110011011100
3X Advanced Row Compression
10X
Columnar Query Compression
15X
Columnar Archive Compression
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9
Oracle Advanced Compression Transparent, Smaller, Faster
100% Application Transparent
End-to-end Cost/Performance Benefits across CPU, DRAM, Flash,
Disk & Network
Runs Faster: OLTP Apps (Transactional & Analytics) & DW
Reduces Database Footprint
– CapEx & OpEx savings
– Increases Cloud ROI through Database Footprint reduction in
DRAM Memory
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10
Oracle Advanced Compression New Features, New Feature Names
Ora
cle
A
dva
nce
d C
om
pre
ssio
n
Oracle Database 11g Oracle Database 12c
OLTP Compression Advanced Row Compression
Secure Files Compression Advanced LOB Compression
Secure Files De-duplication Advanced LOB Deduplication
Hybrid Columnar Compression Hybrid Columnar Compression
NEW Heat Map (Object and Row Level)
NEW Automatic Data Optimization
NEW Temporal (Advancements)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11
Compression New features in Oracle Database 12c
Logminer and GoldenGate
support Capture side changes completed in
11.1 logminer
Apply side changes in 11.2
Faster queries on advanced
row (OLTP) compression
Wide tables (>255 columns) for
advanced row (OLTP) compression
Network Compression
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12
Automatic Data Optimization
An in memory heat map tracks access to segments and
blocks
– Data is periodically written to disk
– Information is accessible by views or stored procedures
Uses can attach policies to tables to compress or tier data
based on access to data
– Tables or Partitions can be moved between compression levels
whilst data is still being accessed
Simplifying the life cycle of data
Po licy 1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13
Heat Map What it tracks “Heat Map” tracking
– Database level Heat Map shows which tables and
partitions are being used
– Block level Heat Map shows last modification at the
block level
Comprehensive
– Segment level shows both reads and writes
– Distinguishes index lookups from full scans
– Automatically excludes stats gathering, DDLs or
table redefinitions
High Performance – Object level at no cost
– Block level < 5% cost
Active
Frequent
Access
Occasional
Access
Dormant
Actively
updated
Infrequently
updated,
Frequently
Queried
Infrequent
access for
query and
updates
Long term
analytics &
compliance
HOT
COLD
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14
Heat Map How to enable
Active
Frequent Access
Occasional Access
Dormant
SQL> alter system set heat_map =‘ON’ scope=both;
Enabling Heat Map
SQL> alter system set heat_map =‘OFF’ scope=both;
Disabling Heat Map
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 15
Understanding Data Usage Patterns Database ‘heat map’
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0
0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 1
1 1
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16
Understanding Data Usage Patterns Database ‘heat map’
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0
0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 1
1 1
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 17
Heat Map for Tables and Partitions “segment” level tracking
ORDERS
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18
Heat Map for Blocks “row” level tracking
ORDERS
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19
Viewing Heat Map Data Data Dictionary Views
V$HEAT_MAP_SEGMENTS
DBA_HEAT_MAP_SEGMENTS
OBJECT_NAME SEGMENT_READ_TIME SEGMENT_WRITE_TIME FULL_SCAN LOOKUP_SCAN
-------------------- ---------------------- ---------------------- ---------------------- ----------------------
DEPT 20/MAR/2013 10:09:30 19/MAR/2013 10:09:30 21/MAR/2013 04:09:30
EMP 20/MAR/2013 22:00:36 20/MAR/2013 22:09:30 21/MAR/2013 09:49:47
BONUS 21/MAR/2013 10:09:30 21/MAR/2013 10:09:30
SALGRADE 20/MAR/2013 10:09:30
EMPLOYEE 18/FEB/2013 09:33:41 21/MAR/2013 09:49:47 18/FEB/2013 09:33:41
ORDERS 21/MAR/2013 15:00:00 19/MAR/2013 10:10:20
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21
0101101110101010010100100100001000
1010101101001011010011100001010010
Archive Data
011100001010001011011
101010100101001001000
010001010101101001011
010101001010010010001
Automatic Data Optimization Usage Based Data Compression
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 21
Hot Data
3X
Advanced Row Compression
Warm Data
1010101011101010011010111000010100
0101101110101010010100100100001000
1010101101001011010011100001010010
0101000010010000100010101011010010
10X
Columnar Query Compression
1000010100100101001010110111000010
101010101110101001101011100001010001011011
101010100101001001000010001010101101001011
010011100001010010010100001001000010001010
101010101110101001101011100001010001011011
15X
Columnar Archive Compression
01110101010010
10000100010101
01011100001010
10101010111010100110101
11000010100010110111010
10100101001001000010001
01010110100101101001110
00010100100101000010010
00010001010101110011010
10100101001001000010001
1110010100100101001010110111011010
101010101110101001101011100001011101011001
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 22
Automatic Data Optimization Add compression and tiering policies to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 22
Oldest Data Most Recent Data
Po licy 1
Po licy 2
POLICY 1:
Compress Partitions with
row compression if they haven’t
been modified in 30 days
POLICY 2:
Compress Partitions with
columnar compression if they
haven’t been modified in 180
days
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 23
Automatic Data Optimization A heat map tracks the activity of segments and blocks
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 23
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 24
Automatic Data Optimization Policies are automatically applied to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 24
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 25
Automatic Data Optimization Policies are automatically applied to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 25
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 26
Automatic Data Optimization Policies are automatically applied to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 26
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 27
Automatic Data Optimization Reduce storage footprint, read compressed data faster
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 27
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 28
Automatic Data Optimization Automatically tier data to lower cost storage
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 28
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Po licy 3
POLICY 3:
If the tablespace is nearly full
compress the oldest partition
with archive compression and
move it to Tier 2 Storage
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 29
Automatic Data Optimization
This Quarter This Year Prior Years
Row Store
for fast OLTP
Compressed Column Store for fast analytics
10x compressed 15x compressed As data cools
down, Advanced
Data Optimization
automatically
converts data to
columnar
compressed
Online
Archive Compressed Column Store for max compression
Reporting Compliance & Reporting OLTP
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 30
Up to 15x Smaller Footprint & Faster Queries
Both Columnar & Archive Compression now complement Advanced
Row Compression
Best Practice:
– Step 1: Use Advanced Row Compression for entire DB and then
– Step 2: ADO automatically converts into columnar compressed once
the updates cool down, and is used mainly for reporting
=> Query speed of Columnar & 10x smaller footprint
– Step 3: ADO automatically converts into archive compressed once
data cools down further and is no longer frequently queried
=> 15-50x smaller footprint
Automatic Data Optimization for OLTP
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 31
Optimizes Data Based on Heat Map Automatic Data Optimization for DW
Data generally comes in via Bulk Loading
Workload dominated by queries, even during loading
Step 1: Bulk Load directly into Columnar Compressed
– 10x smaller footprint, Query speed of Columnar
Step 2: ADO automatically converts to Archive Compressed
and moves to Lower Cost Storage once its queried infrequently
– Data remains online, with 15-50x smaller footprint, & lower
storage cost
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 32
Fast, Flexible Loads & Queries on Columnar Fastest Load with uncompressed & Fastest Queries with columnar
– Mixed workloads often use Java app or 3rd party tools to insert and update data
that does not use Bulk Loads, so cannot use Columnar
Step 1: Load into uncompressed, conventional inserts & updates
– Fast loading, & flexibility of using a regular OLTP app for loading
Step 2: ADO moves to Row Compressed or Columnar
Compressed or Low Cost Storage once updates cool down
– Faster Queries, 3-10x smaller footprint
Automatic Data Optimization – Mixed Use
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 33
Powerful Policy Specification Automatic Data Optimization Declarative Policy Specification: Condition Action
– alter table orders ilm add policy row store compress advanced row after 3 days
of no modification;
– Conditions are time period after creation, no access or no modification of data
– Actions can be Compression Tiering or Tablespace Tiering
Policies are inherited from the tablespace or table
– New tables inherit from tablespace; can also be applied to existing tables
– New partitions (including interval partitions) inherit from table
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 34
Simple Declarative SQL extension Automatic Data Optimization ALTER TABLE orders
ILM add
Active
Frequent
Access
Occasional
Access
Dormant
OLTP Compressed (2-4x)
Affects ONLY Candidate Rows
Cached in DRAM & FLASH
row store compress advanced row
after 2 days of no update
Warehouse Compressed (10x)
High Performance Storage
compress for query low
after 1 week of no update
Warehouse Compressed (10x)
Low Cost Storage
tier to SATA Tablespace
Archive Compressed (15-50X)
Archival Storage
compress for archive high
after 6 months no access
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 35
Scheduled Policy Execution Automatic Data Optimization Immediate and background policy execution
– Row level policies are executed periodically
(Users can configure the frequency of execution)
– Segment level policies are executed in maintenance windows
Policies can be extended to incorporate Business Rules
– Users can add custom conditions to control placement
(e.g. 3 months after the ship date of an order)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 36
Automatic Data Optimization Optimized Back up
Read / Write
Tablespace Read-mostly data is moved to a READONLY Tablespace
10x compressed 15x compressed
READONLY
TBS
As data cools down,
Automatic Data
Optimization
automatically moves
it to a READONLY
TBS, it’s backed up
only once after that
Reporting Compliance & Reporting OLTP
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 37
Automatic Data Optimization Optimized Backups with Automated READONLY data movement
ORDERS 1. As tables grow in size ILM
policies compress data
2. Tablespace containing
partitions reaches ILM
tiering threshold
3. Partitions are moved to
new read only tablespace
on lower spec disk group
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 38
Automatic Data Optimization Optimized Backups with Automated READONLY data movement
SQL> ALTER TABLE ORDERS ILM ADD POLICY
TIER TO DATA2 READ ONLY
AFTER 180 DAYS OF NO MODIFICATION
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 39
Automatic Data Optimization
Data
Classification Automatic Detection
WHAT IF and WHEN Then DO
Scope
• Tablespace level
• Group level
• Segment level:
– Table/Partition/
Subpartition
– Clustered table
• Row level:
– Table
Then Actions
• Compression – Types: OLTP …
• Move to other storage
• Both compress + move
If conditions met
• Which operation
to track?
– Creation
– Low access
– No data modification
– Validity expired
• When?
– After 3 days
– After 1 year
– Tablespace full
Automatic Action
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 40
Oracle Storage Vendors
Disk Saving Extensive Partial
Performance Speeds Queries Adds Overhead
Integration with
Database
Deep integration
excludes maintenance tasks
includes memory access
Integrated with RMAN and Active Data
Guard
Zero integration
maintenance tasks considered
real access
Automatic Data Optimization Why Oracle?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 41
Active
• Recently inserted, actively updated
• OLTP compressed (2-4x)
• Cached in DRAM & Flash
Frequent Access
• Infrequent updates, frequent reports
• High compression (10x)
• High performance storage
Occasional Access
• Infrequent access
• High compression (10x)
• Low cost storage
Dormant
• Retained for long-term analytics and compliance with corporate policies
• Archive compressed (15-50x)
• Archival storage (database or tape)
Automatic Data Optimization
Automatic Data Optimization Summary
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 42
In-Database Archiving Speed up upgrade and reports
• Applications typically work with recent data
• But often need to retain data for 5 to 10 years
• In-Database Archiving provides the ability to archive infrequently used data
within the database
• Archived data is invisible by default
• Works with partition pruning and Exadata storage indexes to eliminate I/O for
archived data
• Archived data remains online for SQL Query & DMLs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 43
In-Database Archiving How to enable
Easily enabled for a table:
Application can marks rows as archived:
Sessions can set default visibility to see all data or active data only (default)
alter table
… row archival
update SALES_ORDERS …
set ORA_ARCHIVE_STATE = 1
alter session set
row archival visibility = [all | active]
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 44
Oracle Others
Application
Knowledge Required
Supply Knowledge Packs for
various Oracle Apps. Supports
custom rules. Consultant cost
usually required.
Cost No cost for functionality
(included in EE) Typical deal $$$
Schema Changes Minimal Required (typically shadow table)
Operational Impact Minimal Access to archive data requires
special support, app dev effort
In-Database Archiving Why Oracle?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 45
Flashback Data Archive provides the ability to track and store
transactional changes to a table over its lifetime. A Flashback Data
Archive is useful for compliance with record stage policies and audit
reports. It archives previous states of rows, the current state of a
record is always visible in the table.
In-Database Archiving only keeps the current state of a record but
allows the application to hide infrequently used data within the
database.
Vs Flashback Archive
In-Database Archiving
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 46
Heat Map, Automatic Data Optimization and In-Database Archiving Summary
• Heat Map
• Automatically tracks access
• Database-aware: maintenance jobs, backups, etc don’t affect heat map
• Automatic Data Optimization
• Declarative easy-to-use syntax to define data compression & movement policies
• Extensible with business-specific logic
• In-Database Archiving
• Automatically hide “archive” data from normal users
• Keep archive data accessible, minimize impact on storage and performance
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 48
Oracle Partitioning in Oracle Database 12c
Core functionality Performance Manageability
Oracle 8.0 Range partitioning
Global Range indexes
Static partition pruning Basic maintenance:
ADD, DROP, EXCHANGE
Oracle 8i Hash partitioning
Composite partitioning (Range-Hash)
Partition-wise joins
Dynamic partition pruning
Expanded maintenance:
MERGE
Oracle 9i List partitioning Global index maintenance
Oracle 9i R2 Range-List partitioning Fast partition SPLIT
Oracle 10g Global Hash indexes Local Index maintenance
Oracle 10g R2 1M partitions per table Multi-dimensional pruning Fast DROP TABLE
Oracle 11g Virtual column based partitioning
More composite choices
REF partitioning
- Interval partitioning
- Partition Advisor
- Incremental statistics management
Oracle 11g R2 Hash-Hash partitioning
Expanded REF partitioning
“AND” pruning Multi-branch execution
Oracle 12c Interval-REF partitioning - Partition Maintenance on multiple
partitions
- Partial local and global indexes
- Asynchronous global index maintenance
for DROP/TRUNCATE
- Online partition MOVE
- Cascading TRUNCATE/EXCHANGE
Over a decade of development and better than ever before
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 49
Improved business modeling
– Interval-Reference Partitioning
– Advanced partition maintenance for Interval-Reference Partitioning
More efficient data maintenance
– Enhanced partition maintenance operations
– Asynchronous global index maintenance
– Partial indexing
Oracle Partitioning in Oracle Database 12c Make a robust and successful feature even better
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 50
Partitioning Improvements
Asynchronous Global Index Maintenance for
DROP and TRUNCATE partition
Cascade Functionality for TRUCATE and EXCHANGE partition
Multiple partition operations in a single DDL
Online move of a partition (without DBMS_REDEFINITION)
Interval + Reference partitioning
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 51
Improved Business Modeling Interval-Reference Partitioning
New partitions are automatically
created when new data arrives
All child tables will be
automatically maintained
Combination of two successful
partitioning strategies for better
business modeling
INSERT INTO orders
VALUES (’01-APRIL-2012’, ... );
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 52
Improved Business Modeling Interval-Reference Partitioning
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 53
Improved Business Modeling Cascading TRUNCATE and EXCHANGE PARTITION
ALTER TABLE orders TRUNCATE
PARTITION APRIL_2012 CASCADE;
Cascading TRUNCATE and
EXCHANGE for improved
business continuity
Single atomic transaction
preserves data integrity
Simplified and less error prone
code development
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 54
Improved Business Modeling Cascading TRUNCATE PARTITION
Parent
Child1
Grandchild 1 Grandchild 2
Child2
Grandchild3
Great Grandchild1
Proper bottom-up processing required
Seven individual truncate operations
Parent
Child1
Grandchild 1 Grandchild 2
Child2
Grandchild3
Great Grandchild1
One truncate operation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 55
Improved Business Modeling Cascading EXCHANGE PARTITION
Parent
Child1
Grandchild 1 Grandchild 2
Child2
Grandchild3
Great Grandchild1
Exchange (clear) out of target bottom-up
Exchange (populate) into target top-down
Parent
Child
Grandchild
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 56
Improved Business Modeling Cascading EXCHANGE PARTITION
Parent
Child1
Grandchild 1 Grandchild 2
Child2
Grandchild3
Great Grandchild1
Exchange (clear) out of target bottom-up
Exchange (populate) into target top-down
Exchange complete hierarchy tree
One exchange operation
Parent
Child
Grandchild
Parent
Child1
Grandchild 1 Grandchild 2
Child2
Grandchild3
Great Grandchild1
Parent
Child
Grandchild
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 57
Partitioning
Improved business modeling
– Interval-Reference Partitioning
– Advanced partition maintenance for Interval-Reference Partitioning
More efficient data maintenance
– Enhanced partition maintenance operations
– Asynchronous global index maintenance
– Partial indexing
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 58
Enhanced Partition Maintenance Operations Online Partition Move
Transparent MOVE
PARTITION ONLINE
operation
Concurrent DML and
Query
Index maintenance for
local and global
indexes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 59
Enhanced Partition Maintenance Operations Maintenance on Multiple Partitions
ALTER TABLE orders
MERGE PARTITIONS Jan2012, Feb2012, Mar2012
INTO PARTITION Quarter1_2012 COMPRESS FOR
ARCHIVE HIGH;
Partition Maintenance on multiple
partitions in a single operation
Full parallelism
Transparent maintenance of local
and global indexes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 60
Enhanced Partition Maintenance Operations
DROP and TRUNCATE complete immediately
We maintain a list of invalid data object ids and ignore those entries
in the index from then on
Automatic scheduler job PMO_DEFERRED_GIDX_MAINT_JOB
will run to clean up all global indexes
Can be run manually
Alter index [partition] CLEANUP is another approach
DROP and TRUNCATE become fast, metadata-only operations
Delayed Global index maintenance
Asynchronous Global Index Maintenance
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 61
Enhanced Partition Maintenance Operations Asynchronous Global Index Maintenance
11.2
12.1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 62
Enhanced Indexing with Oracle Partitioning
Local indexes
Non-partitioned or partitioned global indexes
Usable or unusable index segments
– Non-persistent status of index, no relation to table
Indexing prior to Oracle Database 12c
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 63
Enhanced Indexing with Oracle Partitioning
Local indexes
Non-partitioned or partitioned global indexes
Usable or unusable index segments
– Non-persistent status of index, no relation to table
Partial local and global indexes
– Partial indexing introduces table and [sub]partition level metadata
– Leverages usable/unusable state for local partitioned indexes
– Policy for partial indexing can be overwritten
Indexing with Oracle Database 12c
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 64
Partial indexes span only some partitions
Applicable to local and global indexes
Complementary to full indexing
Enhanced business modeling
Enhanced Indexing with Oracle Partitioning Partial Local and Global Indexes
Global Non-Partitioned Index
Table
Partition
Table
Partition
Table
Partition
Global Partitioned Index
Local Partitioned Index
Partial Global Index
Partial Local Partitioned Index
Partial Global Partitioned Index
Full Indexing
Indexing on
Partial Indexes
Indexing off
No Indexing
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 65
Enhanced Indexing with Oracle Partitioning Partial Local and Global Indexes