Australian Computer Society – Sybase IQ & NSW Health Case StudyLee Marshall, Sybase Australia & Peter Brandt, NSW Health30 April 2008

Sybase Confidential

Gartner

" Of all the Vendors, Sybase made the most impressive move in the 2007 Magic Quadrant. They moved a very long way to the right although slightly lower. Sybase IQ is a columnar DBMS that is very, very fast for analytics. Sybase has been turning in very impressive revenue growth with Sybase IQ after they have positioned themselves correctly in their markets around analytics. Just last quarter and in all of 2007, Sybase has recorded their best revenue and profit growth in their history".

-- Donald Feinberg, 02/072008, presentation to a packed audience on "2007 Gartner MQ (for DW DBMS)" at the Gartner BI conference at Amsterdam

Sybase Confidential

Agenda

• Intro to Sybase• Challenges of BI • How does Sybase IQ address BI Challenges?• NSW Case Study

Sybase Confidential

Introduction to Sybase

Sybase Confidential

Why Sybase

Creating an Information Edge — Today Open, cross-platform software solutions that

securely deliver information anytime, anywhere

Strongest portfolio of information management and enterprise mobility solutions in the market

The world’s most critical data in commerce, communications, finance, government and healthcare runs on Sybase

~4,000 employees in 60 countries Profitable and growing, $1 billion revenue

mark crossed in 2007 2008 - Most profitable 1st quarter in our history

Sybase is the largest global enterprise software company exclusively focused on managing and mobilizing information, from the data center to the point of action.

Sybase Confidential

Challenges of BI

Sybase Confidential

Growing Challenges for BI & the DW

MORE DATAMORE DATA - GB to TB

More end users- 100’s to 1000’s

More complex queries- ad hoc reports

Higher systems costs

Slower performance

Missed reporting windows

Impact to financial performance

Delayed decisions or responses

Missed deadlines and penalties

Today: IT Pain: Business Pain:

The technology supporting BI today must provide SPEED, SCALABILITY, FLEXIBILITY and AVAILABILITY at a low operating cost.

Sybase Confidential

BI & DW Challenge Questions

How do you manage the data explosion?

How do you provide the query response times users now expect?

How do you provide analytics across large data sets without compromising performance

How do you achieve pervasive, operational BI?

Sybase Confidential

We are Pushing the Limits!

What’s the likelihood that… the amount of data will decrease? user demands will slow down? competitive pressures will lighten up? compliance is a passing fad?

Business Intelligence infrastructures need to adapt to keep pace

Sybase Confidential

How Does Sybase IQ Address BI Challenges?

Sybase Confidential

General purpose engines are not

optimized engines

Sybase IQ removes the restrictions and limits required by traditional DBMSs to maintain performance and prevent system failure as you add more data, more users, more complexity.

No limits to amount of data available for analytics No limits to amount of concurrent users & queries No limits on launching new applications No limits to the complexity of queries No limits to access windows

The right tool for the right job

Sybase Confidential

Introduction to Sybase IQ

Designed from the start for analytics — not transactions, unlike traditional RDBMS

Column-based structure and patented indexing put Sybase IQ in

Gartner Group’s Magic Quadrant as a “Visionary.”1

High speed data loads for analytics

Exceptional query performance with ad hoc query performance up to

100x faster than a row based RDBMS

Kills RDBMS on performance, appliances on TCO

A highly optimised analytics server, designed specifically to deliver ultra-high-speed business intelligence and reporting on standard hardware and operating systems.

1 Magic Quadrant for Data Warehouse Database Management Systems 2007. Gartner Group, October, 2007.

IQ Review

Sybase Confidential

Sybase IQ’s Secret Ingredient:Column-Based Architecture

Conventional Database

c5

c4

c3

c2

c1 …c

9c8

c7

c6

r1

r2

r3

r4

r5

SYBASE IQ

c5

c4

c3

c2

c1 …c

9c8

c7

c6

r1

r2

r3

r4

r5

Data is stored vertically – Each column is stored separately

The data is the index Retrieve only columns used in the query – Reduce

System I/O dramatically Allocate a thread for each column individually –

Process the query in parallel

Data is stored horizontally Querying without indexes and views is extremely

I/O intensive Building indexes and views is a huge time and

resource drain Database footprint must be dramatically expanded

to make the environment efficient for querying

Sybase Confidential

Column-Based ArchitectureDramatically Reduced I/O = Faster Query Results

Q: How many MALES are NOT INSURED in NSW?

GenderMMFMM-

800 Bytes/Row

StateVIC

NSWQLDNSWVIC

Row-based RDBMSInsured

YYNYN

800 Bytes x 10M 16K Page = 500,000 I/Os

Process large amounts of unused data Often requires full table scan

Gender Insured State

= 2+ +1101

1101

10MBits

10M Bits x 3 column/8 Bits16K Page = 234 I/OsColumn-based Sybase IQ

0101

10MROWS

MMFM

YNYN

NSWNSWVICNSW

Sybase Confidential

Can start with one server and add CPUs and memory as needed

Can add servers and CPUs with little or no loss in scalability

Can add terabytes of disk to the SAN and Sybase IQ will manage it efficiently

IQ

CPU CPU

Mem CPU

CPU Mem

IQ

CPU CPU

CPU Mem

CPU Mem

IQ

CPU CPU

Mem Mem

Mem Mem

IQ

CPU CPU

CPU CPU

Mem Mem

IQ

CPU Mem

CPU Mem

CPU Mem

Fiber Channel

With Sybase IQ, you can support multiple users per CPU

Scales like a Grid

Mutiplexing for Scalability

Sybase Confidential

Conventional DBMS

LOAD

Base table: 0.2 - 0.5 TB

Indexes: 0.05 - 0.3 TB

Aggr/Summ: 0 - 0.1 TB

SummariesAggregates

1 – 2 TB

Indexes0.5 – 3 TB

Base table“RAW data”no indexes0.9 – 1.1 TB

2.4-6TB

Same INPUT data:“Conventional DW”is 3x-6x larger than

Sybase IQ DW

0.25 - 0.9TB

INPUT DATA:1 TBSource: flat files,ETL, replication,

ODS

Lowest TCO from data compressionDramatically reduced storage and maintenance

LOAD

Sybase Confidential

VLDW CUSTOMERS Proven Data Compression Savings

SYBASE IQ DATA COMPRESSION EXAMPLES

Raw Data Loaded

Sybase IQ Compressed

Estimated Competitor’s Data Explosion

VLDW Ref. Architecture (InfoSizing – June 2007)

1PB 260 TB 2 – 3 PB

Telefonica 70TB 15TB 140 TB to 210TB

comScore Networks 40 TB 16 TB 80 TB to 120 TB

Health Insurance Review Agency 27 TB 12 TB 54 TB to 81 TB

Samsung Card 15 TB 7 TB 30 TB to 45 TB

Nielsen Media Research 12 TB 12 TB 24 TB to 36 TB

Large Credit Card Company 80 TB 39 TB 200 TB to 300 TB

Sybase IQ Data Compression: saves money – saves time

Sybase Confidential

Sybase IQ: BenchmarkWorld’s Largest Data Warehouse

Achievement: 1 Petabyte Why it mattersShowed an 85% data compression ratio by storing a Petabyte of raw transactional data in less than 260 Terabytes of actual disk space

The bigger your data warehouse gets, the more money you save on hardware with Sybase IQ- 250 TB of storage + 1 Server (Sybase IQ)- 1500 TB of storage + 3 Servers (OLTP)

Demonstrated a substantial reduction in the number of disk drives needed for storage, translating directly into at least 90% reduction in CO2 emissions over the lifetime of the Reference Architecture

• More than 10x the savings annually on electricity bill

• 10x reduction in annual CO2 emissions• Shrink storage foot print by a factor of 6x• Contribution to your company’s “green”

initiativesFor several benchmarks, half of the Transactional data was replaced with “EDM” (Emails, Documents and Multimedia) data, creating a data warehouse populated with 572 Terabytes of raw “EDMT” data

Sybase IQ is the only analytics server with a cost effective solution to manage large volumes of unstructured data at extremely low costs for near on-line archiving or for applications that can query both structured and unstructured data

Source: Sun Data Warehouse Reference Architecture for Structured and Unstructured Data, InfoSizing, August 20, 2007

Sybase Confidential

Scale, speed, compression

"The world's largest data warehouse was not built on Teradata or Oracle but on Sybase IQ, a columnar database with very impressive data compression. 1 PB was stored in 250 TB". -- Donald Feinberg, 07/02/2008, presentation to a packed audience on "Architectures for Modern Data Warehouses", Gartner BI conference, Amsterdam

Sybase Confidential

NSW Health Case Study

Sybase Confidential

Who are NSW Health?

The New South Wales Department of Health (NSW Health) provides a comprehensive range of health and health-related services to the 7 million residents of the state including:

Public hospitals Ambulance service Mental health services Drug and alcohol services Community health centers A range of ancillary services

These services are provided in a wide range of settings from primary care outposts in remote regional areas to metropolitan-based tertiary teaching hospitals.

Sybase Confidential

What is the HIE?

The HIE is a data warehouse solution built on Sybase ASE with a Sybase IQ server for reporting and analytics. It is used and maintained by the Demand and Performance Evaluation Branch (DPEB) and captures a wide range of critical information including:

Daily hospital events Admissions Discharges Supplies Ambulance Services Back-office Functions

Information on wait times at various locations, cancer incidences across the State over long timeframes, patient demographic analysis and emergency department performance are just a few examples of the information extracted from the data warehouse.

Given the nature of the information, and NSW Health’s mission statement, any and all information must be timely, accurate and representative.

Sybase Confidential

HIE Under Stress

Sybase Engagement Summary

Initial IQ health check – identified that new hardware was required and that the IQ server required some performance and tuning. This led to an IQ upgrade engagement.

IQ Upgrade – new hardware was purchased and new version of IQ was obtained.

Environment Discovery – only a small number of users accessing IQ due to data currency and performance problems. HIE ASE server under sever stress due to ad-hoc query usage, users frustrated with HIE performance.

Architecture Discovery – existing data movement architecture was examined in order to perform IQ migration, found to be lacking in several areas.

New Architecture Proposed – several alternatives were presented to NSW Health that would eliminate the complicated and time consuming approach currently in use.

Direct Load Architecture Implemented – simplified a very complicated load process and provided far greater flexibility, error management and reporting and progress monitoring.

Dataset Greatly Expanded – used Sybase PowerDesigner to reverse engineer expanded ASE schema and implemented in IQ.

Server Tuned and Loading Optimised – queries returned far quicker even when compared to previous IQ schema, load reduced to 8 – 10 hours. Meant users could now use IQ instead of ASE.

Sybase Confidential

Old Data Refresh Architecture

ASE12.5.0.2

Data Feeds

Pre-Process

008 Server (Tru64)

Ageing Files New Files

Updates Inserts

HIE

Backup Server

SAN

5

6

7

8

hdc005 Server (ASE Staging and IQ)

ASE 12.5.0.2

Backup Server

IQ 12.4.3

1 4

2

3

Sybase Confidential

New Data Refresh Architecture

ASE12.5.0.2

Data Feeds

Pre-Process

008 Server (Tru64)

Ageing Files New Files

Updates Inserts

HIE

New HP-UX Itanium Server

IQ 12.6

Sybase Confidential

Results

Highlights of What Sybase Delivered at the Department

Reduced severe stress on HIE – removing reporting load reduced stress on HIE and delivered reports to users in record times (in some cases 400+ times faster)

Delivered extremely fast reports to an expanded user community and on an expanded dataset – new IQ server able to support all reporting users with both historical and current datasets

Massive reduction in data load times – from 40 hours down to 8 – 12 hours

Far greater data currency - data loads can occur overnight allowing reporting against current data rather than week (or more) old data

70% Data Compression – required far less disk space than expected

Greatly reduced administration effort and load failures - simplified a very complicated load process and provided far greater flexibility, error management and reporting and progress monitoring

2007 Computerworld Honours Laureate – industry recognised innovative solution

Sybase Confidential

Success at NSW Health

• “Congratulations. Its a terrific result which will make reporting much more efficient” – Peter Brandt (Associate Director, NSW Health)

• “Just had a play for the first time. Bloody blistering fast” - James Prior (HIE Manager, Health Technologies)

Sybase Confidential

Summary

Sybase Confidential

IQ Summary

Sybase IQ is designed specifically to deliver ultra-high-speed business intelligence & reporting on standard hardware and operating systems

"We have been watching and waiting for database implementers to catch up with Sybase IQ and its unique table and index structure, which deliver impressive performance for complex queries against large data warehouses. Judging from sales over the past couple of years, it seems that the market finally 'gets it'.“

Carl Olofson, Research Vice President Information Management and Data Integration Software Research IDC, 2007

It delivers for today’s operational BI needs …high speed data loads for analyticsexceptional query performancead hoc query performance up to 100x faster than a row

based RDBMS dramatically lower TCO by reducing performance overhead,

storage costs & maintenance requirements

Sybase Confidential

Project Summary

“We have a great relationship with Sybase. Our consultant really listened to what we needed, brought fresh ideas to the discussion, and solicited our feedback on all improvements. Throughout this process, we always knew the Professional Services team had our best interests at the forefront.”

“This project is definitely a success.  We’ve been able to improve our user’s experience with queries as well as improve our operational efficiency.  This helps us meet our stated goals and preserve our IT resources.”

Peter Brandt, Associate Director of Data Collections and Reporting, New South Wales Health

Sybase Confidential

Analyst Summary

"Sybase IQ is a columnar DBMS that is very, very fast for analytics. Sybase has been turning in very impressive revenue growth with Sybase IQ after they have positioned themselves correctly in their markets around analytics. Just last quarter and in all of 2007, Sybase has recorded their best revenue and profit growth in their history".

-- Donald Feinberg, 02/072008, presentation to a packed audience on "2007 Gartner MQ (for DW DBMS)" at the Gartner BI conference at Amsterdam

Sybase Confidential

Customer Summary

Strong, established, fast-growing customer base More than 2000 customer projects at more than 1200 sites

worldwide

Sybase Confidential