SQLServer 2008 Fast Track Data Warehouse

SQL Server 2008 Fast Track SQL Server 2008 Fast Track

Data Warehouse 2.0Data Warehouse 2.0

Speaker: Phil Hummel of WinWire Technologies

Presentation developed by: Bruce Campbell

Western Region Data Warehouse Specialist, Microsoft

Silicon Valley SQL Server User Group

February 16, 2009

Mark Ginnebaugh, User Group Leader,

[email protected]

Agenda

• DW vs. OLTP

• Balanced Architecture Approach for DWFast Track Defined

• Fast Track Reference Architectures•

• Next Steps

END USER TOOLS

Microsoft DW & BI Stack

DELIVERY

PerformancePoint

Services

BI & DW PLATFORM (RDBMS, ETL, OLAP, Reporting)

DW versus OLTP

DatabaseDatabase•• Designed Designed for for analytical operations: operations:

Strategic Strategic focusfocus

•• Optimized Optimized for bulk load and large, complex, for bulk load and large, complex, unpredictable queriesunpredictable queries

•• Fewer concurrent users relative to Fewer concurrent users relative to OLTPOLTP

StorageStorage

DatabaseDatabase•• Designed Designed for for operational requirements: requirements:

Tactical Tactical focusfocus

•• Optimized for transactions: “single row” Optimized for transactions: “single row” entry and retrievalentry and retrieval

•• Thousands of concurrent usersThousands of concurrent users

StorageStorage

•• Primary focus on Read operationsPrimary focus on Read operations

•• Optimized Optimized for disk scan over seek for disk scan over seek operationsoperations

•• Storage Storage optimization focused on disk optimization focused on disk scan rate (MB/s)scan rate (MB/s)

StorageStorage

•• Emphasizes Emphasizes transactional transactional performanceperformance

•• Optimized Optimized for disk seek over scan for disk seek over scan operationsoperations

•• Storage Storage optimization focused on optimization focused on I/O operations/s (IOPs)I/O operations/s (IOPs)

Sequential I/O

Sequential I/O

• Scans on large data stores are usually read with sequential read patterns and not random read patterns

• Scalable, predictable performance

Random I/O

• OLTP usually random-read centric. Discrete lookups benefit from index optimization and random read capability.

• Not as predictable & scalable for • Scalable, predictable performance

• Requires 1/3 or fewer drives to match server I/O consumption capability.

• Not as predictable & scalable for data warehousing

• Requires large number of drives to match server I/O consumption capability.

All databases contain both scans and seeks among with other types of reads and writes, DW workload indicate

that the vast majority of reads are sequential – not all

Some SQL Data Warehouses today

Big SAN

Big 64-core Server

Connected together

What’s wrong with this picture?What’s wrong with this picture?

Answer: system out of balance

• This server can consume 16 GB/Sec of IO, but the SAN can only deliver 2 GB/Sec– Even when the SAN is dedicated to the SQL Data

Warehouse, which it often isn’t

– Lots of disks for Random IOPS BUT

– Limited controllers � Limited IO bandwidth– Limited controllers � Limited IO bandwidth

• System is typically IO bound

• Queries are slow

Result: significant investment, not delivering performanceResult: significant investment, not delivering performance

The Alternative: A Balanced System

• Design a server + storage configuration that can deliver all the IO bandwidth that CPUs can consume when executing a SQL Relational DW workload

• Avoid sharing storage devices among servers• Avoid overinvesting in disk drives

– Focus on scan performance, not IOPS

• Layout and manage data to maximize range scan performance and minimize fragmentation

• Layout and manage data to maximize range scan performance and minimize fragmentation

Potential Performance Bottlenecks

FC

HBA

A

B

FC

HBA

A

B FC SWITCH

STORAGE

CONTROLLERA

B

A

BCACHE

SERVER

CACHE

SQL SERVER

WINDOWS

CPU CORES

A

B

DISK DISK

LUN

DISK DISK

LUN

HBA B FC SWITCH

SQL SERVER

CPU CORES

CPU Feed Rate HBA Port Rate Switch Port Rate SP Port Rate

LUN

SQL Server

Read Ahead Rate

LUN Read Rate Disk Feed Rate

SQL Server Fast Track Data Warehouse

• A method for designing a cost-effective, balanced system for Data Warehouse workloads

• Reference hardware configurations developed in conjunction with hardware partners using this

Solution to help customers and partners accelerate their data warehouse

deployments

conjunction with hardware partners using this method

• Best practices for data layout, loading and management

Relational Database Only – Not SSAS, IS, RS

Fast Track Data Warehouse Components

Software:

• SQL Server 2008 Enterprise

• Windows Server 2008

Configuration guidelines:

• Physical table structures• Physical table structures

• Indexes

• Compression

• SQL Server settings

• Windows Server settings

• Loading

Hardware:

• Tight specifications for servers,

storage and networking

• ‘Per core’ building block

Fast Track Scope

Data Path

Analysis Services

Cubes

Reporting Services

Web Analytic Tools

Integration

Services ETL

Supporting Systems BI Data Storage Systems Presentation Layer Systems

Presentation Data

Presentation Data

Data Path

Data Warehouse PerformancePoint

Dedicated SAN,

Storage Array

SharePoint Services

Microsoft Office SharePoint

Data Staging,

Bulk Loading

Reference Architecture Scope (dashed)

Excel Services

Presentation Data

Presentation Data

Two SQL DW Infrastructure Options:

SQL Classic DW or Fast Track SQL DW

Shared Network

Bandwidth

Enterprise Shared

SAN Storage

Dedicated

Network

Bandwidth

SQL Classic DW

Architecture

Leverages Shared SAN

Fast Track SQL DW Architecture

Architecture modeled after DW Appliances

Teradata, DATAllegro..etc “ Appliance Like”

Uses Dedicated SAN arrays and Network

Dedicated SAN

SQL 2008 Data Warehouse

SMP Server

SAN Arrays 1:4 cpu cores

8 Data Disk / Array – 4 Raid 1 Pairs

Simultaneous SQL Server Reads

2 Log and 1 Hot Spare

EMC AX4 – HP MSA2312

IBM 3400

OLTP Applications SQL Fast Track DW supports “Scan Centric” DW

workloads that are index light

Optimizing storage layout for scan

intensive workloads• LUN configuration is based on

RAID1 pairs

– Optimal for scan type access patterns

• Striping across storage is

accomplished via SQL Server data

S

P

A

03 04

RAID GP02RAID GP02

LUN3LUN3

LUN4LUN4

03 04

RAID GP02

LUN3

LUN4

01 02

RAID GP01

LUN1

LUN2

RAID GP03RAID GP03RAID GP03 RAID GP04RAID GP04RAID GP04

09 10

RAID GP05RAID GP05

LUN0

(Logs)

LUN0

(Logs)

09 10

RAID GP05

LUN0

(Logs)

HO

T S

PA

RE

accomplished via SQL Server data

files

• Observed throughput for a single

RAID pair >= 130 MB/s

S

P

B

05 06

RAID GP03RAID GP03

LUN5LUN5

LUN6LUN6

05 06

RAID GP03

LUN5

LUN6

07 08

RAID GP04RAID GP04

LUN7LUN7

LUN8LUN8

07 08

RAID GP04

LUN7

LUN8

HO

T S

PA

RE

Storage Layout Implications for SQL Server

LUN16 LUN 2 LUN 3LUN 1

Permanent FG

Permanent_1.ndf

Perm

anant_DB

Permanent_2.ndf Permanent_3.ndf Permanent_16.ndf

Local Drive 1

Log LUN 1

Permanent DB

Log

TempDB

TempDB.mdf (25GB) TempDB_02.ndf (25GB) TempDB_03ndf (25GB) TempDB_16.ndf (25GB)

Perm

anant_DB

Stage

Database Stage FG

Stage_1.ndf Stage_2.ndf Stage_3.ndf Stage_16.ndf

Stage DB Log

Creating Sequential Data Layout

• Goal: Align logical and physical ordering of data within a Filegroup

• Two primary ways Fast Track optimizes allocation for Sequential Scan– Minimize Fragmentation– Minimize Fragmentation– Manage Load processing

Maximum Consumption RateTheoretical throughput for IO stack

• Using a 2x quad-core server as a building block / starting point

• Ensure that the per-core data consumption rate can

Maximum theoretical throughput for IO stack

components sized for an 8 CPU core Fast Track system

(assumes 200 MB/s per core)

consumption rate can be delivered by all elements of the IO stack

• Sticker on the new car: “Miles Per Gallon”

Server

Windows Server OS

MCR 1.6 GB/s

Storage Enclosure

Storage EnclosureFiber Switch

500 MB/s

500 MB/s

500 MB/s

500 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

HBA

HBA

Min2

GB/s

Min2

GB/s

CPU

Socket

(4 Core)

CPU

Socket

(4 Core)

Scaling the IO stackFiber

Switch

Storage Enclosure

Storage Processor

Storage Processor

RAID-1RAID-1RAID-1RAID-1


RAID-1RAID-1

CPU

Socket

(4 Core)

CPU

Socket

(4 Core)

CPU

Socket

(4 Core)

CPU

Socket

(4 Core)

Storage Enclosure

Storage Processor

Storage Processor



RAID-1RAID-1

Storage Enclosure

Storage Processor

Storage Processor



RAID-1RAID-1

Storage Processor RAID-1RAID-1RAID-1RAID-1

RAID-1RAID-1

CPU

Socket

(4 Core)

CPU

Socket

(4 Core)

CPU

Socket

(4 Core)

CPU

Socket

(4 Core)

ServerHBA

HBA

Storage Enclosure

Storage Processor

Storage Processor



RAID-1RAID-1

Storage Enclosure

Storage Processor

Storage Processor



RAID-1RAID-1

Storage Enclosure

Storage Processor

Storage Processor



RAID-1RAID-1

Storage EnclosureStorage Processor

RAID-1RAID-1RAID-1

RAID-1RAID-1

Storage Enclosure

Storage Processor

Storage Processor



RAID-1RAID-1

(4 Core) (4 Core)

HBA

HBA

HBA

HBA

HBA

HBA

Fast Track Data Warehouse Reference

ConfigurationsServer CPU

CPU

CoresSAN Data Drive Count

Initial

Capacity*

Max

Capacity**

HP Proliant

DL 385 G6

(2) AMD Opteron Istanbul

six core 2.6 GHz

12 (3) HP MSA2312fc (24) 300GB 15k SAS 6TB 12TB

HP Proliant

DL 380 G6

(2) Intel Xeon® 5500 Series

Quad core

8 (2) HP MSA2312 (16) 300GB 15k SAS 4TB 8TB

HP Proliant

DL 585 G6

(4) AMD Opteron Instanbul

six core 2.6 GHz

24 (6) HP MSA2312fc (48) 300GB 15k SAS 12TB 24TB

HP Proliant

DL 580 G5

(4) Intel Xeon® 7400 Series six

core


HP Proliant

DL 785 G6

(8) AMD Opteron Istanbul

six core 2.8 GHz


* Core-balanced compressed capacity based on 300GB 15k SAS not including hot spares and log drives. Assumes 25% (of raw disk space) allocated for Temp DB.** Represents storage array fully populated with 300GB15k SAS and use of 2.5:1 compression ratio. This includes the addition of one storage expansion tray per enclosure.

30% of this storage should be reserved for DBA operations

DL 785 G6 six core 2.8 GHz

Dell PowerEdge

R710

(2) Intel Xeon Nehalem quad

core 2.66 GHz

8 (2) EMC AX4 (16) 300GB 15k FC 4TB 8TB

Dell Power Edge

R900

(4) Intel Xeon Dunnington

six core 2.67GHz

24 (6) EMC AX4 (48) 300GB 15k FC 12TB 24TB

IBM X3650 M2 (2) Intel Xeon Nehalem quad

core 2.67 GHx

8 (2) IBM DS3400 (16) 200GB 15K FC 4TB 8TB

IBM X3850 M2 (4) Intel Xeon Dunnington six

core 2.67 GHz

24 (6) IBM DS3400 (24) 300GB 15k FC 12TB 24TB

IBM X3950 M2 (8) Intel Xeon Nehalem four

core 2.13 GHz

32 (8) IBM DS3400 (32) 300GB 15k SAS 16TB 32TB

Bull Novascale

R460 E2

(2) Intel Xeon Nehalem quad

core 2.66 GHz

8 (2) EMC AX4 (16) 300GB 15k FC 4TB 8TB

Bull Novascale

R480 E1

(4) Intel Xeon Dunnington

six core 2.67GHz

24 (6) EMC AX4 (48) 300GB 15k FC 12TB 24TB

SQL Server Fast Track Data Warehouse 2.0 for

HP – now on G6 PlatformFive AMD and Intel based Reference configurations available for HP:

AMD Based Reference Architectures

2 Processor Configuration

– Server: HP ProLiant DL385 G6 with 2 6-core AMD Opteron CPUs

– Storage server: MSA Storage

– Scalability: 4 – 12 TB


– Server: HP ProLiant DL 585 G6 with 4 6-core AMD Opteron CPUs



8 processor Configuration

– Server: HP ProLiant DL 785 G6 with 8 6-core AMD

Opteron CPUs


– Scalability: 24 – 48TB


HP – now on G6 Platform

Intel Based Reference Architectures

2 Processor Configuration– Server: HP ProLiant DL380 G6 with 2 4-core Intel Xeon® 5500

Series CPUs



4 Processor Configuration– Server: HP ProLiant DL 580 G5 with 4 6-core Intel Xeon®

7400 Series CPUs



New Fast Track Data Warehouse 2.0 for IBM

Three Reference configurations available for IBM:2 Processor Configuration

– Server: IBM System x3650 M2 with 2 Quad-core Intel Xeon CPUs– Storage server: IBM System Storage DS3400– Scalability: 4 – 8 TB

4 Processor Configuration– Server: IBM System x3850 M2 with 4 6-core Intel Xeon CPUs– Storage server: IBM System Storage DS3400– Scalability: 12 – 24 TB

8 processor Configuration– Server: IBM System x3950 M2 with 8 Quad-core Intel Xeon CPUs– Server: IBM System x3950 M2 with 8 Quad-core Intel Xeon CPUs– Storage server: IBM System Storage DS3400– Scalability: 16 – 32TB


DELLTwo Reference configurations available for DELL:


– Server: Dell Power Edge R710 with 2 Quad-core Intel Xeon processors

– 8 CPU Cores

– 32GB Memory– 32GB Memory

– Storage server: EMC CLARiiON AX4



– Server: Dell Power Edge R900 with 4 6-core Intel Xeon processors

– 24 CPU Cores

– 96 GB Memory



SQL Server Fast Track Data Warehouse for BULL

Two Reference configurations available for BULL:

2 Processor Configuration– Server: Bull Novascale R460 E2 with 2 Quad-core Intel Xeon

processors



4 Processor Configuration– Server: Bull Novascale R480 E1 with 4 6-core Intel Xeon

processors



• Also included in the Rack:– SQL Server Analysis Services

– SQL Server Reporting Services

– SQL Server Integration Services

– HA Server

– Administration Server (with Management Studio, Backup Server

Fast Track Data Warehouse Benefits

• Lower TCO

– Minimizes risk of overspending on un-balanced hardware configurations

– Commodity Hardware

• Choice• Choice– HW platform

– Implementation vendor

• Reduced Risk

– Validated by Microsoft

– Encapsulates best practices

– Known performance & scalability

SummaryFaster time to solution

High scale: up to 48TB

Low TCO with better price performance; industry standard hardware

Better performance out of the box and predictable performance

Reduced risk through balanced hardware & Best practices

Integration with Madison Hub & Spoke Architecture

Fast Track Dataoffers customers

Twelve reference architectures from HP, Dell, Bull, EMC and IBMSQL Server Fast Track Data

Warehouse has 2 componentsIBM

System Integrators with industry solution templates –Avanade, HP, Hitachi, Cognizant and EMC

Warehouse has 2 components

Next Steps

• Proof Steps

– Quick Start DW Roadmap Service

– Architectural Design Session

– Madison Technology Preview (MTP)

– Review Madison, SQL Server Classic or Fast Track – Review Madison, SQL Server Classic or Fast Track DW HW/SW configurations and pricing

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market

conditions,

it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.

MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SQLServer 2008 Fast Track Data Warehouse

Documents

Transcript of SQLServer 2008 Fast Track Data Warehouse