SQL201 - Microsoft SQL Server 2008 R2 Mark Souza Director Microsoft SQL Server.

SQL201 - Microsoft SQL Server 2008 R2

Mark SouzaDirectorMicrosoft SQL Server

SQL Server 2008 – Strong ReleaseTrusted Platform Productivity Business Intelligence

SQL Server IS mission critical

20+ points increase in all DB Tracker perception metrics

Better value – less expensive up front, easier to manage

Highest DBA productivity

Industry-leading developer tools

Extend apps to the cloud with SQL Azure

Strong, winning strategy

Build BI into the stack, through familiar tools for users

Gartner MQ leader for BI platforms

The SQL Server 2008 R2 Journey

The origins of KilimanjaroSelf-service Business IntelligenceApplication & Multi-server ManagementScaling for the next generation enterprise

High End Scale out Data WarehousesCEP – Complex Event Processing

Reaching the summit

http://en.wikipedia.org/wiki/File:Kibo_summit_of_Mt_Kilimanjaro_001.JPG

& Project codenamed “Madison”

Project “Gemini” Excel Add-in Report Builder 3.0 StreamInsight, Complex Event

Processing Master Data Services SharePoint Publishing

Scalable & Trusted Platform

IT & Developer Efficiency

Self Service Business

Intelligence

Application & Multi-Server Management

Project “Gemini” SharePoint Management Console

StreamInsight .Net Extensions

Enterprise-level security, scalability

Supports up to 256 Logical Processors

SQL Server System Preparation Enhanced Data Compression

Solid Foundation for Enterprise Workloads

Hyper-V™

Live Migration

Support for largest Windows Server hardware

MPP support for 100+ terabyte data warehouses

Appliance-like data warehouse on industry standard hardware

Project “Madison”Better Together with Windows

Server




Reaching the summit


What's in a name…

Gemini - Gemini (pronounced /ˈgɛmɪnaɪ/, Latin: twins, symbol ) is one of the ♊ constellations of the zodiac known as "the twins"

http://en.wikipedia.org/wiki/Wikipedia:IPA_for_English



http://en.wikipedia.org/wiki/Latin_language

http://en.wikipedia.org/wiki/Twin

http://en.wikipedia.org/wiki/Constellation

http://en.wikipedia.org/wiki/Zodiac

The corporate Twins: IT Pro/End UserA widening gap between end user and IT needs

• End Users:– Access to corporate data– Mix in their own data– Aggregate, augment data– Organize, present solutions– Share insights with others

• IT Professionals:– Know data is secure– Know data is consistent– Keep systems running– Keep the cost down– Track data access & usage

There need not be an end-user versus IT conflict or gap in meeting user needsThe gap is caused by lack of enabling technology, heavy “app lifecycle” costs

I’m not exactly sure what I need but I know I need it

now…If I help this time I’m stuck

maintaining it forever…

The Challenges

New formal BI solutions need time and resources

Diverse users have diverse data needs

Ad-hoc requests stress I.T. capacity

Data warehouses do not cover all data or all users

Power users bypass I.T. with unsanctioned sources

Bottleneck

Chaos

Gemini: Uniting the Twins

Re-draws the line betweenI.T. and end-user roles

Empowered to create without IT dependence

Managing compliance and

resources without user obstruction

I.T.

• Provision• Administer• Secure• Track data

Users

• Directly model

• Analyze• Personalize• Share data

Excel is key for IW/Users“It has to be Excel”

“We don’t get OLAP & dimensional models”“What is data modeling anyway?”“Just make my Excel better"

Use Excel as a catch all tool toCollect dataClean, prepare and integrate itEnrich and AnalyzeCreate reports and visualizationsShare them with others

Easy sharing of insights is criticalEach power user publishes data to 10’s-100’s consumersIT needs to know !

BI!

SNEAK PEAK NICHOLAS DRITSAS PROGRAM MANAGERSQL SERVER PRODUCT TEAM

IT manage the "Spreadmarts"•Excel is the IW tool of choice, but for IT:• Excel is a problem - “unmanageable”• Excel is an addiction – users “can’t quit it”

•Why not make Excel part of the solution?• Include Excel as part of a complete BI solution

• Structured and manageable• Give IT insight into its usage

•Provide IT with the technology to• Have insight and management• Become a strategic differentiator• Without being a bottleneck

•Enable managed Self-Service



High End Scale out Data WarehouesCEP – Complex Event Processing

Reaching the summit


Trends

Challenges: People vs. Hardware

Number of database apps

Number of DBA’s

1990 2000 2010

Hardware computing ca

pacity Underutilized hardware

Overburdened Administrators

Database apps increasing at a higher rate than DBAs

Overburdened DBAs

Hardware computing capacity exploding

Underutilized hardware

Control server sprawl with 1 to many management – setup is fast and easy

Introducing a better wayToday Tomorrow

Manage capacity through policies – save time, optimize resources

Single unit of deployment – increase deployment and upgrade efficiency

Key ConceptsData-Tier Application Component (DAC)

Think of this as the new unit of deployment for T-SQL apps and providing similar benefits of a MSI in a very general sense. There is a definition of all the parts that make up the app along with services such as Install, Uninstall, Upgrade, and eventually Repair.

SQL Server Manageability

Confidential – Internal Use Only

17

Data-Tier Application Unit (DAU) • Think of this as the overall unit of management. Or

the deployed instance of a DAC• Maps to a plain database in KJ. In SQL 11, a CDB - a

more self-contained database (with additional dependent objects).

• Provides namespace and resource isolation.

DAC

DAC Deployment ProfileDeployment Requirements, Management Policies, Failover Policies

LogicalTables, Views,

Constraints, SProcs, UDFs

Users, Logins

PhysicalIndexes, Partitions

FileGroups…

Unit of Deployment

DAU – (C)DB

SchemaTables, Views, Constraints, SProcs, UDFs, Users, Logins

Indexes, Partitions, FileGroups

DAC

Properties & MetadataDeployment Requirements, Management Policies, Failover Policies

Unit of Management

Key Concepts (continued..)

SQL Server Manageability

Confidential – Internal Use Only

18

Utility Control Point (UCP)• Think of this as the central reasoning

point of the utility. • From here operations such as policy

evaluation, discovery, deployment, impact, and what if analysis can be performed.

Connection Virtualization (Medusa)• Think of this as DNS for connection

strings• Decouples application from the

physical location of DAU (CDB)• Uses Active Directory (KJ).

Management Studio

DBA

SQL0

2SQ

L03

SQL0

4

SQL0

1

Managed Instances

SQL0

5

UCP

New wizards in SSMS – fast and easy setupCreate a Control PointEnroll instancesInsights refreshed every 15 minutes

Key Benefits Control • Optimization • Efficiencies

Management Studio

Database Administrator

Microsoft Confidential—Preliminary Information Subject to Change

SQL Server Control Point

Managed Server Group

Gain Visibility and Control

At-a-glance views for insights



Improve Resource Optimization

Simple UI for policy adjustments

ID consolidation opportunities

Quickly drill-down to detailed views

Application & Multi-Server Management

• Creating the UCP• Insights – Health Check

Demo•

Single unit of deploymentIntegration with Visual StudioStreamlined deployments & upgrades Client

“Finance”

Management Studio

Database Administrator Central management


Data-Tier Developer


Managed Server Group

Improve Efficiencies

Application & Multi-Server Management• Creating the DAC• Migrating the DAC

demo

Application & Multi-server Management

Productive database application development and management viaIntroduction of new Database Application Components (DAC)Application of Policy Based Administration to DACsIntellisense integration with Visual StudioAbility to version, deploy and reverse engineer a DAC

Multi-server Management made easier throughDAC experiences integrated with Management Studio and Visual StudioImport and Export of database application artifacts Support for reverse engineering a DAC from down-level systemsDeployment to one or more target systemsMonitoring of multiple instances of a database application on several servers via Management Studio




Reaching the summit


The Data Warehousescale journey Project Madison

Massive scale-out to 100’s TBMassive Scale with Low TCOIntegration with Microsoft BIMTP – August 2009 RTM 1H CY10

Scale

-u

p

Massive Scale-out

FastTrack Reference Architecture – 10s TBEasier, predictable and cost effecient

10s of TB

Accelerate scalable Data Warehouse deployments at lower TCO

Pre-configured, pre-tested HW reference architectures (4-32 TB)

SI Solution Templates

Fast Track DW

Appliance-like time to valueFlexibility through choice of HW platformsLow TCO through commodity hardware and value pricing. Reduced risk through pre-tested and pre-tuned configurationsProvides a clear upgrade path to “Madison” via Hub/Spoke

Microsoft Confidential—Preliminary Information Subject to Change 27

Scale out Data WarehousingProject “Madison” integrates the MPP innovations in DATAllegro with SQL ServerMassive DW scale at lower TCO targeting many 10s to many 100s TB data warehouses‘Appliance-like’ user experience via hardware partners1st Half CY2010 with delivery as a new “edition” of SQL Server

INDUSTRY STANDARDNETWORKING

INDUSTRY STANDARDSTORAGE

INDUSTRY STANDARDSERVERS

Reference hardware platforms

“Madison”

Massively Parallel Processing

True MPP, Shared Nothing ArchitectureServer/CPU’s have their own dedicated resourcesSecret Sauce is MPP Query Optimizer supporting Parallel operationsLightning-fast Queries, Data Loads And UpdatesLinear ScalabilityLower TCO- Reduced DBA time

MPP

Control Rack

High-Level Madison Architecture

30

Landing Zone

ETL Load Interface

Control NodeActive/Passive

Client Drivers

Corp. Backup Solution

Backup Node

Management NodeActive/Passive

Data RackDatabase Server

NodesStorage Nodes

Infi

nib

an

d

Spare Node

Fib

reC

ha

nn

el

Compute Node

Date Dim

D_DATE_SK

D_DATE_ID

D_DATE

D_MONTH

…

Item

I_ITEM_SK

I_ITEM_ID

I_REC_START_DATE

I_ITEM_DESC

…

Store Sales

Ss_sold_date_sk

Ss_item_sk

Ss_customer_sk

Ss_cdemo_sk

Ss_store_sk

Ss_promo_sk

Ss_quantity

…

Promotion

P_PROMO_SK

P_PROMO_ID

P_START_DATE_SK

P_END_DATE_SK

…

Store

S_STORE_SK

S_STORE_ID

S_REC_START_DATE

S_REC_END_DATE

S_STORE_NAME

…

Customer

C-CUSTOMER_SK

C_CUSTOMER_ID

C_CURRENT_ADDR

…

Customer

Demographics

CD_DEMO_SK

CD_GENDER

CD_MARITAL_STATUS

CD_EDUCATION

…

Database TablesMadison Appliance Nodes

Data Distribution with Replication

C I

D

CD

S

P

C I

D

CD

S

P

C I

D

CD

S

P

C I

D

CD

S

P

C I

D

CD

S

P

C I

D

CD

S

P

SS

SS

SS

SS

SS

SS

Large TablesAre Hash Distributed

Smaller TablesAre Replicated

DBA Work Made EasyCreate Database <dbname>With(AUTOGROW = ON | OFFDISTRIBUTION_SIZE = value_in_GBREPLICATION_SIZE = value_in_GBLOG_SIZE = value_in_GB

CREATE DATABASE sampledb_288 ON PRIMARY (NAME = N'sampledb_288', FILENAME = N'[DRIVE_LETTER]:\primary\sampledb_288.mdf', SIZE = 3MB, MAXSIZE = UNLIMITED, FILEGROWTH = 10%), FILEGROUP DIST_A (NAME = N'DIST_A_1', FILENAME = N'[DRIVE_LETTER]:\data_01\sampledb_288_DIST_A_1.ndf', SIZE = 625MB, MAXSIZE = UNLIMITED, FILEGROWTH = 4MB), FILEGROUP REPLICATED (NAME = N'REPLICATED_9_1', FILENAME = N'[DRIVE_LETTER]:\data_01\sampledb_288_REPLICATED_9_1.ndf', SIZE = 125MB, MAXSIZE = UNLIMITED, FILEGROWTH = 4MB), LOG ON (NAME = N'sampledb_288_LOG_1', FILENAME = N'[DRIVE_LETTER]:\log_01\sampledb_288_LOG_1.ldf', SIZE = 1000MB, MAXSIZE = UNLIMITED, FILEGROWTH = 10%);

ALTER DATABASE sampledb_288 SET AUTO_CREATE_STATISTICS ON; ALTER DATABASE sampledb_288 SET AUTO_UPDATE_STATISTICS ON; ALTER DATABASE sampledb_288 SET RECOVERY SIMPLE;

Mad

ison

G

enerates




Reaching the summit


What Is CEP?Complex Event Processing (CEP) is the continuous and incremental processing of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency.

Database Applications Event-driven Applications

Query Paradigm

Ad-hoc queries or requests

Continuous standing queries

Latency Seconds, hours, days Milliseconds or less

Data Rate Hundreds of events/sec Tens of thousands of events/sec or more

request

response

Eventoutput streaminput

stream

Microsoft’s CEP SolutionData Sources, Operations, Assets, Feeds, Sensors, Devices

Monitor &

Record

Operational Data Store & Archive

CEP Engine

f(x) g(y)

CEP Enginef(x) f'(x)

g(y) h(x,y)

History

Deploy

Resultsf'(x) h(x,y)

Manage&

Benefit

Mine&

Design

Input Data Streams

Input Data Streams

OutputData Streams

CEP Deployment Alternatives

Data Sources

Aggregation & Correlation

CEP

CEP

CEP

CEP for lightweight processing and filteringCEP for aggregation and correlation of in-flight events

CEP for complex analytics including historical data

Event processing engines are deployed at multiple places on different scales•At the edge – close to the data source• In the mid-tier – consolidate related

data sources • In the data center – historical archive,

mining, large scale correlation

Devices

Sensors

Web servers

Feeds

CEP

CEP CEP

CEP

CEP CEP

CEP CEP CEP

Complex Analytics &Mining

LINQ Query Examples

LINQ Example – GROUP&APPLY, WINDOW:

from e3 in MyStream3group e3 by e3.i into SubStreamsfrom s4 in SubStreamsfrom e4 in s4.SlidingWindow(FiveMinutes,ThreeSeconds)select new {

pl = new MyNewPayload(e4.i, e4.f)};

LINQ Example – JOIN, PROJECT, FILTER:

from e1 in MyStream1join e2 in MyStream2 on e1.ID equals e2.IDwhere e1.f2 = “foo”select new { e1.f1, e2.f4 };

Join

FilterProject

Grouping

Window

Recap: CEP Platform from Microsoft

CEP Engine

Output Adapters

Input Adapters

Event

Standing Queries

Event sources Event targets

`

Event

Event

Event

Event

Event

Event

C_ID C_NAME C_ZIP

Event

Event

Static reference data

CEP Application Development

Development experience with .NET, C#, LINQ and Visual Studio 2008

CEP platform from Microsoft to build event-driven applications

Event-driven applications are fundamentally different from

traditional database applications: queries are

continuous, consume and produce streams, and compute

results incrementally

Flexible adapter SDK with high performance to connect

to different event sources and sinks

The CEP platform does the heavy lifting for you to deal with

temporal characteristics of event stream data

http://msdn.microsoft.com/vstudio/products/default.aspx

question & answer

SQL201 - Microsoft SQL Server 2008 R2 Mark Souza Director Microsoft SQL Server.

Documents

Transcript of SQL201 - Microsoft SQL Server 2008 R2 Mark Souza Director Microsoft SQL Server.