DB-03: A Tour of the OpenEdge™ RDBMS Storage Architecture
description
Transcript of DB-03: A Tour of the OpenEdge™ RDBMS Storage Architecture
DB-03:A Tour of the OpenEdge™ RDBMS Storage Architecture
Richard BanvilleTechnical Fellow
2 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
What’s New:Type II Storage Areas
Block clustering– Alleviates object fragmentation
– Improves I/O efficiency Concurrent Space Allocation
– Area space Allocation
– Object space allocation Improved object manipulation
– Table scan without index
– Fast object deletion
3 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Agenda
Physical Layout
Advantages
A Usage Example
Summary
4 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Type II Area Block Clusters
Block Cluster: 8, 64, or 512 adjacent blocks Configured in .st file Applied via prostrct Fixed size for area Unit of space allocation for objects Blocks within cluster are “non-social”
5 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Type II Area ObjectsObject are made up of one or more block clusters
(Objects: tables, indexes, lobs, area control)
6 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Type II Area ObjectsClusters chained together for fast access
(Objects: tables, indexes, LOBS, area control)
Allows table scan without an indexAllows fast table delete
7 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
What Else Is Different w/Type II
Area HWM increased a cluster at a time– More efficient block formatting– Concurrent Space Allocation
Database extend– MAX(64, cluster size)
Other– Reduced fragmentation and scatter– Allows more advanced tools to be designed
8 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Object Block (Type I Area):Space Allocation Chains
MixedObjectBlock
MixedRec
Block
MixedRec
Block
MixedRec
Block
MixedRec
Block
FreeBlock
FreeBlock
FreeBlock
FreeBlock
Record Free Chain
IdxdelBlock
IdxdelBlock
IdxdelBlock
Unique indexes
Only
Total blocks, HWM
9 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Object Block (Type II Area):Area Free Space Allocation Chains
AreaObjectBlock
FreeBlock
FreeBlock
FreeBlock
FreeBlock
Area Control Object
Total blocks, Cluster HWM
Free Cluster List (for Area)
10 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Object Block (Type II Area):Space Allocation Chains
IndexObjectBlock
RecFreeBlock
RecFreeBlock
RecFreeBlock
RecFreeBlock
FreeBlock
FreeBlock
FreeBlock
FreeBlock
IdxdelBlock
IdxdelBlock
IdxdelBlock
Unique indexes
Only
Table 2ObjectBlock
RecFreeBlock
RecFreeBlock
RecFreeBlock
RecFreeBlock
Table 1ObjectBlock
Total blocks,
Cluster HWM
Total blocks,
Cluster HWM
Total blocks,
Cluster HWM
11 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
This Is Important Stuff
Indexes– Index delete chain maintenance
– Index blocks are clustered together Mixed Areas
– Vast improvements multi table or multi index areas
– Won’t have index and record blocks intertwined Reduced fragmentation and scatter
– Fewer Dump and Loads! More efficient I/O
12 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Record Space Allocation
Record Packing Factor– Records stored variable length
– Space allocated from record block free chains
– RPF regulates record block free chain content Definitions
– Records per Block: (Blocksize / mean rec size)
– Create limit: rec block free space required for a created record’s expansion. Also the minimum rec fragment size. (75/150)
– Toss limit: free space required for a rec block to remain on the record block free chain (150/300)
13 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Changing RPF
Records Per Block– Changed via .st file
– Granularity Value per area
Create/Toss limits– _proutil <db> C setTableTossLimit <table> value
– Granularity Values per area in Type I storage area Values per object in Type II storage area
14 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
RPF Suggestions
Change Create Limit if:– Fragmentation occurs due to record updates
of newly created records
– You expect to see 1 fragment but get 2 Change Toss Limit if fragmentation occurs
due to record updates of existing records– You expect 1 or 2 fragments but get 3 or 4
Coordinate with RPB– (Blocksize / mean rec size)
Don’t change if you have no reason to
15 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Overall Type II Layout
Area
Control
Object
Object #1
Object #2
Cluster Free List
Object #3
Area Data (Free & in use)
16 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Agenda
Physical Layout
Advantages
A Usage Example
Summary
17 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Other Advantages of Type II Storage Areas
Block level Check sum– Identifies corrupt blocks prior to data change
– Allows for larger I/O in future Storage
– Rowids stored as 64 bits
– Variable length block header size Object information stored in block header
– Allows for improved maintenance
– Improves database repair operations
18 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Storage Management Advantages
Improvements through organization Efficient block formatting
– Cluster at a time
– Fewer bi/ai notes written Bottlenecks Resolved
– Concurrent space allocation
– Optimistic buffer and index locking protocols Migration Path
– Can use both Type I & Type II in same database
19 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
RootLevel 1
Level 2
Level 3(Leaf)
- Leaf entries contain pointer to record
Records
- Cursor maintains info or last key accessed
Table Scan via B-tree
20 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Select * from Customer;
- Cursor maintains info of last record accessed
- I/O Sequential through cluster
2nd Cluster
1st Cluster
3rd Cluster
4th Cluster
21 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Fast Object Delete
Area
Control
Object
Object #1
Object #2
Cluster Free List
Object #3
Area Data (Free & in use)
22 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Fast Object Delete
Area
Control
Object
Object #2
Cluster Free List
Object #3
Area Data (Free & in use)
23 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Fast Object Delete
Area
Control
Object
Object #2
Cluster Free List
Object #3
Area Data (Free & in use)
24 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
OpenEdge 10 Temp tables
Released in 10.0b– Fast delete– Fast delete/create on empty
Enhanced in 10.0b02– Avoid delete/create– Avoid I/O when formatting
Hybrid Type I & II Storage Area– Index Objects Type I– Other Objects Type II– 8 Block Clusters
Suggestions– “empty temp-table <name>”– -tmpbsize 1, -tmpbsize 8– Better performance with increased –Bt
25 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Agenda
Physical Layout
Advantages
A Usage Example
Summary
26 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Best Practices for Use
Physical– Include Striping (RAID or do it yourself)– File extent Location
Schema– Separate index and table data– Multi table area for small, medium & large records– Records per block properly set for each area
Growth– Always have a variable length extent– Enable large files
27 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Location, Location, Location
b /bi/exampleDB.b1 f 1024000
b /bi/exampleDB.b2 f 1024000
b /bi/exampleDB.b3
#
d "Schema Area":6,64 /db/exampleDB.d1
#
d “Customer Indexes":7,1;8 /db/exampleDB_7.d1 f 512000
d “Customer Indexes":7,1;8 /db/exampleDB_7.d2
#
d “Customer Data":8,128;64 /db/exampleDB_8.d1 f 1024000
d “Customer Data":8,128;64 /db/exampleDB_8.d2
28 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Cluster Size
b /bi/exampleDB.b1 f 1024000
b /bi/exampleDB.b2 f 1024000
b /bi/exampleDB.b3
#
d "Schema Area":6,64 /db/exampleDB.d1
#
d “Customer Indexes":7,1;88 /db/exampleDB_7.d1 f 512000
d “Customer Indexes":7,1;88 /db/exampleDB_7.d2
#
d “Customer Data":8,128;6464 /db/exampleDB_8.d1 f 1024000
d “Customer Data":8,128;6464 /db/exampleDB_8.d2
29 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Records Per Block
b /bi/exampleDB.b1 f 1024000
b /bi/exampleDB.b2 f 1024000
b /bi/exampleDB.b3
#
d "Schema Area":6,64 /db/exampleDB.d1
#
d “Customer Indexes":7,11;8 /db/exampleDB_7.d1 f 512000
d “Customer Indexes":7,11;8 /db/exampleDB_7.d2
#
d “Customer Data":8,128128;64 /db/exampleDB_8.d1 f 1024000
d “Customer Data":8,128128;64 /db/exampleDB_8.d2
30 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Multi Object Areas
d “Large Record Indexes":9,1;8 /db/exampleDB_9.d1 f 512000
d "Large Record Indexes":9,1;8 /db/exampleDB_9.d2
#
d “Large Record Tables":10,16;64 /db/exampleDB_10.d1 f 1024000
d “Large Record Tables":10,16;64 /db/exampleDB_10.d2
#
d “Small Record Indexes":11,1;8 /db/exampleDB_11.d1 f 512000
d “Small Record Indexes":11,1;8 /db/exampleDB_11.d2
#
d “Small Record Tables":12,256;64 /db/exampleDB_12.d1 f 1024000
d “Small Record Tables":12,256;64 /db/exampleDB_12.d2
31 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Cluster Size:Fast Growing Tables
d “Misc Indexes":13,1;6464 /db/exampleDB_13.d1 f 512000
d “Misc Indexes":13,1;6464 /db/exampleDB_13.d2
#
d “Fast Growing Tables":14,64;512512 /db/exampleDB_14.d1 f 1024000
d “Fast Growing Tables":14,64;512512 /db/exampleDB_14.d2 f 1024000
d “Fast Growing Tables":14,64;512512 /db/exampleDB_14.d3
#
a /ai/exampleDB.a1 f 51200
a /ai/exampleDB.a2 f 51200
a /ai/exampleDB.a3 f 51200
…
32 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Records Per Block:Fast Growing Tables
d “Misc Indexes":13,11;64 /db/exampleDB_13.d1 f 512000
d “Misc Indexes":13,11;64 /db/exampleDB_13.d2
#
d “Fast Growing Tables":14,6464;512 /db/exampleDB_14.d1 f 1024000
d “Fast Growing Tables":14,6464;512 /db/exampleDB_14.d2 f 1024000
d “Fast Growing Tables":14,6464;512 /db/exampleDB_14.d3
#
a /ai/exampleDB.a1 f 51200
a /ai/exampleDB.a2 f 51200
a /ai/exampleDB.a3 f 51200
…
33 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
AI File Location
d “Misc Indexes":13,1;64 /db/exampleDB_13.d1 f 512000
d “Misc Indexes":13,1;64 /db/exampleDB_13.d2
#
d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d1 f 1024000
d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d2 f 1024000
d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d3
#
a /ai/exampleDB.a1 f 51200
a /ai/exampleDB.a2 f 51200
a /ai/exampleDB.a3 f 51200
…
34 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
In Summary
More efficient layout Better Performance Migration path Foundation for the future
35 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Questions?
36 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
Thank you for your time!
37 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
38 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation
OpenEdge 10 RDBMS Advanced Storage Architecture
The following Progress courses cover related subject matter. Please visit: www.progress.com/education for course descriptions and relevant curriculum maps.
Database Administration