Oracle Advanced Compression...
Transcript of Oracle Advanced Compression...
![Page 1: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/1.jpg)
Oracle Advanced Compression Tests
Svetozar Kapusta15th of October 2009
![Page 2: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/2.jpg)
What is CERN?CERN is:
• CERN is the world’s largest particle h i l b t l t d i G
CERN is:≈2500 staff scientists (physicists, engineers, etc.)≈6500 visiting scientists(h lf f th ld'physics laboratory located in Geneva,
Switzerland• CERN hosts the Large Hadron Collider
(half of the world's particle physicists) Coming from ≈500 universities or institutesCERN hosts the Large Hadron Collider
(LHC) which is the biggest man-made accelerator
representing ≈80 nationalities.
• LHC will start its operation in November 2009 and will form, together with its experiments, theand will form, together with its experiments, the biggest sub-nuclear microscope in the world.
Courtesy of M. Girone
![Page 3: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/3.jpg)
LHC: a Very Large Scientific Instrument
LHC : 27 km long
Mont Blanc, 4810 m
LHC : 27 km long100m underground
ATLAS
Downtown Geneva
ALICECMS+TOTEM
Courtesy of M. Girone
![Page 4: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/4.jpg)
… Based on Advanced Technology
27 km of superconducting magnetscooled in superfluid helium at 1.9 Kp
Courtesy of M. Girone
![Page 5: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/5.jpg)
Experiments are ready for collisions
Courtesy of M. Girone
![Page 6: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/6.jpg)
The Data Acquisition
Ian.Bird@cern ch
6Courtesy of M. Girone
![Page 7: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/7.jpg)
Data Acquisition, First pass processing
Ian.Bird@cern ch
7
1.25 GB/sec (ions)
Courtesy of M. Girone
![Page 8: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/8.jpg)
CERN Openlab
Collaboration between CERN and industrial Openlab partners: HP Intel Oracle andOpenlab partners: HP, Intel, Oracle and Siemens
Framework for evaluating and integrating Framework for evaluating and integrating cutting-edge IT technologiesCERN i l t t h l CERN acquires early access to technology
CERN offers expertise and a demanding computing environment to push new technologies to their limits
CERN provides a neutral ground for carrying out advanced R&D
Excellent collaboration with Oracle
![Page 9: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/9.jpg)
Databases for physics at CERN
Relational databases play a key role in the experiments’ production dataflow chains
Listed among the critical services for the gLHC experiments
Bulk of physics data stored in files aBulk of physics data stored in files, a fraction of it in databases
Most applications are OLTP Most applications are OLTP Some data warehouse applications are also
iemerging
![Page 10: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/10.jpg)
Data Growth
Expected data growth is roughly ≈20-30 TB per year per experiment
Experiments need to have all data available pat any time During the experiments lifetimes (10-15 years)u g t e e pe e ts et es ( 0 5 yea s) Few extra years, as the data analysis will continue
We have to provide an efficient way of storing We have to provide an efficient way of storing and accessing the few Peta bytes of mostly read-only dataread-only data
Answer to our challenge is the compression available in 11G2 and Exadata2available in 11G2 and Exadata2
![Page 11: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/11.jpg)
Advanced Compression Tests Exadata2 located in Reading, UK
Half rack with 7 storage cells each of 12 disks eachHalf rack with 7 storage cells each of 12 disks each Accessed remotely from Geneva for 2 weeks
Data used The largest and representative production and test tables Exported compressed using Datapump Imported into Exadata2 using Datapump
ApplicationsPVSS ( l t l t d b th i t ) PVSS (slow control system used by the experiments)
GRID monitoring application GRID Test dataGRID Test data File transfer applications (PANDA) Logging application for ATLAS
First results the same day
![Page 12: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/12.jpg)
Compression factors for various compression types of various physics applications
60
70
20
30
40
50
ARCHIVE LOWARCHIVE HIGH
0
10
20
OLTP
BASICQUERY LOW
QUERY HIGHARCHIVE LOW
NO COMPRESSION
PVSS columns: 6 number, 4 TS(9) , 5 varchar2 , 3 binary_doubleLCG GRID Monitoring columns: 5 numberLCG TESTDATA columns: 6 number(38), 1 varchar2, 1 CLOB( )ATLAS PANDA FILESTABLE columns: 3 number, 12 varchar2, 2 date, 2 charATLAS LOG MESSAGES columns: 5 number, 7 varchar2, 1 TS
![Page 13: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/13.jpg)
Table creation times for various compression types of various physics applications. Normalized to no compression.
4045
1520253035
ARCHIVE LOWARCHIVE HIGH
05
1015
OLTP
BASICQUERY LOW
QUERY HIGHARCHIVE LOW
NO COMPRESSION
OLTP
PVSS columns: 6 number, 4 TS(9) , 5 varchar2 , 3 binary_doubleLCG GRID monitoring columns: 5 numberLCG TESTDATA columns: 6 number(38), 1 varchar2, 1 CLOB( )ATLAS PANDA FILESTABLE columns: 3 number, 12 varchar2, 2 date, 2 charATLAS LOG MESSAGES columns: 5 number, 7 varchar2, 1 TS
![Page 14: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/14.jpg)
Full table scans performance for various compression types of various physics applications. Normalized to no compression.
3
3.5
1.5
2
2.5
ARCHIVE HIGH
0
0.5
1
BASIC
QUERY LOWQUERY HIGH
ARCHIVE LOW
NO COMPRESSION
OLTP
PVSS columns: 6 number, 4 TS(9) , 5 varchar2 , 3 binary_doubleLCG GRID monitoring columns: 5 numberLCG TESTDATA columns: 6 number(38), 1 varchar2, 1 CLOB( )ATLAS PANDA FILESTABLE columns: 3 number, 12 varchar2, 2 date, 2 charATLAS LOG MESSAGES columns: 5 number, 7 varchar2, 1 TS
![Page 15: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/15.jpg)
Full table scans performance for various compression types of various physics applications. Normalized to no compression. Exadata offloading set to false.
25
30
10
15
20
25
ARCHIVE HIGH
0
5
10
BASIC
QUERY LOWQUERY HIGH
ARCHIVE LOW
NO COMPRESSION
OLTP
PVSS columns: 6 number, 4 TS(9) , 5 varchar2 , 3 binary_doubleLCG GRID monitoring columns: 5 numberLCG TESTDATA columns: 6 number(38), 1 varchar2, 1 CLOB( )ATLAS PANDA FILESTABLE columns: 3 number, 12 varchar2, 2 date, 2 charATLAS LOG MESSAGES columns: 5 number, 7 varchar2, 1 TS
![Page 16: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/16.jpg)
Exadata2 offloadingFull table scans performance for various compression types of ATLAS
logging application with and without Exadata offloading
1000
me
[s]
10
100
ble
scan
tim
1
10
Full
tab
Please note the logarithmic scale
![Page 17: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/17.jpg)
Export Datapump Compression
Compression factor for PVSS data Export Datapump ≈9X tar bzip2 utility
• ≈11X on non compressed exported PVSS data • ≈1.2X on the compressed exported PVSS data
Compression factor for LCG application p pp Export Datapump ≈13X tar bzip2 utilityp y
• ≈9X on non compressed exported LCG data• ≈1.2X on the compressed exported LCG data
![Page 18: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/18.jpg)
Conclusions
Tested basic, OLTP and hybrid columnar i d D t icompression and Datapump compression
The results for data from physics applications are rather impressing (2-6X OLTP, 10-70X EHCC archive high)
EHCC can achieve up to ≈3X better compression than tar bzip2 compression of p p pthe same data exported uncompressed
Oracle Compression offers a win-winOracle Compression offers a win win solution, especially for OLTP Shrinks used storage volume Shrinks used storage volume Improves performance
![Page 19: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/19.jpg)
Thank you for your attention
![Page 20: Oracle Advanced Compression Testsopenlab-mu-internal.web.cern.ch/openlab-mu-internal/03_documents/4... · Excellent collaboration with Oracle. Databases for physics at CERN Relational](https://reader030.fdocuments.in/reader030/viewer/2022040801/5e38acf2af50087f7b34cf3d/html5/thumbnails/20.jpg)
Backup
16
18
8
10
12
14
2
4
6
8
0
CPU Consumed vs No Cmp
Logical Reads vs No Cmp