Oracle RAC and Linux in the real enterprise
Oracle RAC and Linux in the real enterprise
October, 02Mark ClarkDirectorMerrill Lynch Europe PLCGlobal Database Technologies
October, 02Mark ClarkDirectorMerrill Lynch Europe PLCGlobal Database Technologies
AgendaAgenda The challenge
Current solutions
DBMS Utopia at Merrill Lynch
Selected architecture
Selected partners
Program history
Results and conclusions
The managed service
The challenge
Current solutions
DBMS Utopia at Merrill Lynch
Selected architecture
Selected partners
Program history
Results and conclusions
The managed service
The Challenge and MissionThe Challenge and Mission
“ Access to Oracle’s renowned availability, integrity, performance and scalability at a
commodity server price point”
“ Access to Oracle’s renowned availability, integrity, performance and scalability at a
commodity server price point”
Current solutionsCurrent solutions Risc based UNIX platforms
Strengths Well understood and integrated into ML
infrastructure Full and complete ISV support Robust and mature Low risk technically and financially Mass acceptance
Risc based UNIX platforms Strengths
Well understood and integrated into ML infrastructure
Full and complete ISV support Robust and mature Low risk technically and financially Mass acceptance
Current solutions (cont.)Current solutions (cont.) RISC based UNIX platforms
Weaknesses Platform costs moderately high Large compute platforms especially so Limited competition in server marketplace Emerging technologies slow to market Closed source
RISC based UNIX platforms Weaknesses
Platform costs moderately high Large compute platforms especially so Limited competition in server marketplace Emerging technologies slow to market Closed source
DBMS Utopia at Merrill LynchDBMS Utopia at Merrill Lynch Fully managed service
Actual usage-based charging model
Delivery at commodity prices
Flexible capacity model
High availability at Day 1
Fully managed service
Actual usage-based charging model
Delivery at commodity prices
Flexible capacity model
High availability at Day 1
Selected technologiesSelected technologies Oracle / RAC
A market leader of database management tools and products with new high availability/scalability options (RAC).
Intel Commodity server architecture, well understood at Merrill
Lynch in the SQL/server domain. Compelling technology roadmap
Linux Rapidly developing open source OS with a strong
technology partnership with Oracle and Intel. Leverages the Intel architecture. Just another UNIX like OS.
Oracle / RAC A market leader of database management tools and
products with new high availability/scalability options (RAC).
Intel Commodity server architecture, well understood at Merrill
Lynch in the SQL/server domain. Compelling technology roadmap
Linux Rapidly developing open source OS with a strong
technology partnership with Oracle and Intel. Leverages the Intel architecture. Just another UNIX like OS.
Selected partners Selected partners Oracle
Early access to certain product functionality and new features. Fast track to engineering and performance group
Hewlett Packard Current Intel server platform for Merrill Lynch Europe and
provider of server hardware for all testing (formerly Compaq)
Intel Provision of customer solutions centres and skills /
resources for the offsite proof of concept
Oracle Early access to certain product functionality and new
features. Fast track to engineering and performance group
Hewlett Packard Current Intel server platform for Merrill Lynch Europe and
provider of server hardware for all testing (formerly Compaq)
Intel Provision of customer solutions centres and skills /
resources for the offsite proof of concept
Program history Program history Key milestones
November 2001 (Phase 1) IA32, Oracle 9.0.1, Red Hat 7.2, Raw partitions
March 2002 (Phase 1b) Two node production RAC deployment
October 2002 (Phase 2) IA32, Oracle 9.2.0, Red Hat 2.1 AS, OCFS
Key milestones November 2001 (Phase 1)
IA32, Oracle 9.0.1, Red Hat 7.2, Raw partitions
March 2002 (Phase 1b) Two node production RAC deployment
October 2002 (Phase 2) IA32, Oracle 9.2.0, Red Hat 2.1 AS, OCFS
Program Results - Phase 1Program Results - Phase 1 November 2001 (Phase 1) POC Build
IA32, Oracle 9.0.1, Red Hat 7.2, Raw partitions
Functional Oracle worked
Performance More performant and lower cost comparable to
RISC/UNIX server of similar configuration
November 2001 (Phase 1) POC Build IA32, Oracle 9.0.1, Red Hat 7.2, Raw partitions
Functional Oracle worked
Performance More performant and lower cost comparable to
RISC/UNIX server of similar configuration
Program Results - Phase 1 (cont.)Program Results - Phase 1 (cont.) Gaps
Non-certified OS/Oracle combination Infrastructure and ISV alignment
Unmanageable RAW / No OMF support No cluster filesystem No SAN integration Backup / Recovery tools Performance management Monitoring / Alerting
Gaps Non-certified OS/Oracle combination Infrastructure and ISV alignment
Unmanageable RAW / No OMF support No cluster filesystem No SAN integration Backup / Recovery tools Performance management Monitoring / Alerting
Program Results - Phase 1bProgram Results - Phase 1b March 2002 (Phase 1b) - POC Build
IA32, Oracle 9.0.1, Red Hat 7.2, Raw partitions
Functional Oracle worked
Performance More performant and lower cost comparable to
RISC/UNIX server of similar configuration
March 2002 (Phase 1b) - POC Build IA32, Oracle 9.0.1, Red Hat 7.2, Raw partitions
Functional Oracle worked
Performance More performant and lower cost comparable to
RISC/UNIX server of similar configuration
Program Results - Phase 1b (cont.)Program Results - Phase 1b (cont.) Risks and mitigation Risks and mitigation
Gap / Risk Resolution / Mitigation
Non-certified OS/Oracle combination Took the risk(chose ML Standard Red Hat build)
Cluster interconnect single point offailure
Took the risk
Unmanageable RAW / No OMF support Took the risk
No cluster filesystem Took the operational hit
No SAN integration Local attached storage
Backup / Recovery tools Resolved to Infrastructure standard
Performance management Influenced vendors roadmap
Monitoring / Alerting Influenced vendors roadmap
Program results - Phase 2Program results - Phase 2October 2002 (Phase 2)
The GA BuildIA32, Oracle 9.2.0, Red Hat 2.1 AS, OCFS4 Node RAC configuration32 Processor16GB memoryEMC Clarion storage arrayBrocade switched SAN fabricGigabit cluster interconnect with redundant
switchesSingle instance per node (today)
October 2002 (Phase 2) The GA Build
IA32, Oracle 9.2.0, Red Hat 2.1 AS, OCFS4 Node RAC configuration32 Processor16GB memoryEMC Clarion storage arrayBrocade switched SAN fabricGigabit cluster interconnect with redundant
switchesSingle instance per node (today)
Program results - Phase 2 (cont.)Program results - Phase 2 (cont.)
d1d1 d2d2
d3d3 d4d4
EMC EMC ClarionClarion
d11d11
d13d13
d12d12
d14d14
h1h1
h3h3
h2h2
h4h4
d21d21
d23d23
d22d22
d24d24
d31d31
d33d33
d32d32
d34d34
d41d41
d43d43
d42d42
d44d44
SAN switchSAN switch
Interconnect Interconnect sw1sw1
Interconnect Interconnect sw2sw2
Program results - Phase 2 (cont.)Program results - Phase 2 (cont.) Functional
It still worked
Performance / Stress testing Server side Oracle CPU stress test I/O stress test (Direct path load) Client/Server realworld stress test
Critical failure condition testing Unexpected power failures Interconnect failure (single path) SAN connectivity failure
Functional It still worked
Performance / Stress testing Server side Oracle CPU stress test I/O stress test (Direct path load) Client/Server realworld stress test
Critical failure condition testing Unexpected power failures Interconnect failure (single path) SAN connectivity failure
Program results Phase 2 - (cont.)Program results Phase 2 - (cont.) Performance / Stress testing
Server side Oracle CPU stress test
Used to prove CPU scalability
Open source benchmarking tool based broadly on TPC-C Order-Status & Stock-Level transactions - “orabm” sourced from http://www.dbcool.com
In memory database (no I/O stress) - 300MB SGA total
orabm allows rapid deployment and results gathering - ie; low cost
Performance / Stress testing Server side Oracle CPU stress test
Used to prove CPU scalability
Open source benchmarking tool based broadly on TPC-C Order-Status & Stock-Level transactions - “orabm” sourced from http://www.dbcool.com
In memory database (no I/O stress) - 300MB SGA total
orabm allows rapid deployment and results gathering - ie; low cost
Program results Phase 2 - (cont.)Program results Phase 2 - (cont.)
TPC-C Order-Status/Stock-Level Transactions-per-secondBest Single Session Performance
0
50
100
150
200
250
300
350
400
Server Platform
TP
S t
ota
l
rh21as (compaq dl760)
rh21as (compaq dl580)
XXX
YYY
ZZZ
TPC-C Order-Status/Stock-Level Transactions-per-secondBest Single Session Performance
0
50
100
150
200
250
300
350
400
Server Platform
TP
S t
ota
l
rh21as (compaq dl760)
rh21as (compaq dl580)
XXX
YYY
ZZZ
• Results of best single session tps• (Oracle CPU stress test)
• Results of best single session tps• (Oracle CPU stress test)
Program results Phase 2 - (cont.)Program results Phase 2 - (cont.)
Performance / Stress testing I/O stress test (Direct path load)
Used to prove I/O scalability
Oracle direct path loader was used in a multi-stream parallel load activity
Fixed format input data loading the TPC-C CUSTOMERS table
Tests executed in a single node and multi node configuration
Performance / Stress testing I/O stress test (Direct path load)
Used to prove I/O scalability
Oracle direct path loader was used in a multi-stream parallel load activity
Fixed format input data loading the TPC-C CUSTOMERS table
Tests executed in a single node and multi node configuration
Program results Phase 2 - (cont.)Program results Phase 2 - (cont.)
• Results of I/O stress test• Results of I/O stress testOracle Data Load Performance (4 node cluster)
30000
40000
50000
60000
70000
80000
90000
0 2 4 6 8 10 12 14 16 18 20 22 24 26
Parallel Load Streams
Ro
ws
/se
c
0.0
10.0
20.0
30.0
40.0
50.0
60.0
MB
/se
c
Row s/sec
MB/sec
Note: 6 load streams per node (ie; 12 streams = 2 nodes)Note: 6 load streams per node (ie; 12 streams = 2 nodes)
Program results Phase 2 - (cont.)Program results Phase 2 - (cont.)
Performance / Stress testing Client/Server realworld stress test
Used to prove transaction throughput compared to existing Merrill Lynch systems
Using Quest Benchmark Factory
Loading generated 10x change volumes compared to heaviest loaded ML EMEA database
Performance / Stress testing Client/Server realworld stress test
Used to prove transaction throughput compared to existing Merrill Lynch systems
Using Quest Benchmark Factory
Loading generated 10x change volumes compared to heaviest loaded ML EMEA database
Program results Phase 2 - (cont.)Program results Phase 2 - (cont.)
Critical failure condition testing Unexpected power failures
Removed the power cable mid-processing No outage
Interconnect failure (single path) NIC failover by Software Completely transparent to Oracle No outage
SAN connectivity failure Removal of HBA cable mid processing No outage
Critical failure condition testing Unexpected power failures
Removed the power cable mid-processing No outage
Interconnect failure (single path) NIC failover by Software Completely transparent to Oracle No outage
SAN connectivity failure Removal of HBA cable mid processing No outage
Phase 2 - summaryPhase 2 - summary Gap resolution for DBMS Utopia Gap resolution for DBMS Utopia
Gap / Risk Resolution / Mitigation
Non-certified OS/Oracle combination Red Hat 2.1AS (ML Standard build)
Cluster interconnect single point offailure
NIC failover and redundant switches
Unmanageable RAW / No OMF support Oracle Cluster File System(Open source from Oracle Corp)
No cluster filesystem Oracle Cluster File System(Open source from Oracle Corp)
No SAN integration Oracle Cluster File System integrationwith EMC SAN fabric
Backup / Recovery tools Legato
Performance management ISV Delivery pending
Monitoring / Alerting Patrol delivered for Linux
The managed service - the futureThe managed service - the future
Based on the 2002 Phase 2 configuration and products
Flexible capacity model
Rapid time to solution provision
With an actual compute-based billing model
Due for launch in December 2002
Based on the 2002 Phase 2 configuration and products
Flexible capacity model
Rapid time to solution provision
With an actual compute-based billing model
Due for launch in December 2002
Top Related