Sq l 2014 New Features

SQL Server 2014 New and Enhanced

Features

Presentation by Kevin G. [email protected]

Kevin G. Boles

SQL Server Consultant Extraordinaire“The Wolf” for the Relational EngineIndicium Resources, Inc.AIM/Twitter: TheSQLGuruGTalk/GMail: [email protected] 2007-2012, MCT, MCITP, yada-yada-yada

Who Am I??

mailto:[email protected]

World-class relational engine expert Almost 45,000 hours invested in SQL Server Absolutely LOVE doing targeted performance

analysis and tuning work, and designing and building high-scale, highly-performing database applications

VERY good at making others better at interacting with SQL Server

Couldn’t SSAS my way out of a paper bag ◦ … with both ends open!

What Do I Do?

AGENDA

64Worst Thing NOT Changed

128But it WAS!!

4/16Next Worst Thing NOT Changed

What’s being delivered◦ New cardinality estimator◦ Incremental statistics for partition◦ Parallel SELECT INTO

Main benefits◦ Better query performance:

Better choice of query plans Faster/more frequent statistics refresh at partition level

◦ Consistent query performance ◦ Better supportability using two steps (decision making

and execution) to enable better query plan troubleshooting

◦ Loading speed into table improved significantly using parallel operation

Query Processing Enhancements

SQL Express+ Feature SQL Server transaction commits can now be

either fully durable or delayed durable (aka Lazy Commit)

Delayed transaction durability reduces the latency due to log I/O by keeping the transaction log records in memory and writing to the transaction log in batches, thus requiring fewer I/O operations

When to use delayed transaction durability ◦ You can tolerate some data loss◦ You are experiencing a bottleneck on transaction

log writes.◦ Your workloads have a high contention rate.

Delayed Durability

The in-memory transaction log is flushed to disk when:◦ A fully durable transaction in the same database makes a

change in the database and successfully commits (KEY GOTCHA HERE!)

◦ The user executes the system stored procedure sp_flush_log successfully

◦ The in-memory transaction log buffer fills up and automatically flushes to disk

Delayed Durability

ALTER DATABASE … SET DELAYED_DURABILITY = { DISABLED | ALLOWED | FORCED }

DELAYED_DURABILITY = { OFF | ON } (with NATIVE_COMPILED sprocs ONLY

COMMIT [ { TRAN | TRANSACTION } ] [ transaction_name | @tran_name_variable ] ] [ WITH ( DELAYED_DURABILITY = { OFF | ON } ) ]

In-Memory Built-In

New In-Memory OLTP

Enhanced In-Memory ColumnStore for DW

In-Memory BI with PowerPivot (tabular model is sits in memory too!)

Buffer Pool Extension to SSDs & Enhanced Query Processing

Before

15,000 requests/s

ec

16x FasterWith In-Memory

250,000 requests/s

ec

SQL Server 2014 with In-Memory

Bwin Game Application

Bwin Game Application

Select only highly utilized tables to be in-memory

Optimize in-memory to fit existing hardware

In-memory performance across OLTP, DW and BI

All in a single SKU

On average 10x faster, without having rewrite entire app

Leverage full SQL Server capabilities

2012 Column Store IndexesIndex Storage Design• Column-Orientation

– Store data vertically instead of per-row– String Dictionaries for variable-length data

• Segment data into groups (1 million rows/group)• Benefits

– DW queries usually pick only a subset of columns– You can do the IO only for those columns– We can also compress that data effectively since it often has lots of duplicates– Space savings of 1.5x-2x vs. a row-based page-compressed equivalent

IO Patterns for:CI Scan

Column-based scan 3 colsColumn-based

w/Compression

Column-Oriented Sized to fit within L2 cache

(1000 rows at a time) Multiple Operators work on a

batch sequentially Goal: Reduce avg. per-tuple

cost ◦ Reducing L2 data and instruction

cache misses

2012 Batch-Mode Query Execution

Off The CPU Die SUCKS … a LOT!!Latency Comparison Numbers

L1 cache reference 0.5 ns

Branch mispredict 5 ns

L2 cache reference 7 ns 14x L1 cache

Mutex lock/unlock 25 ns

Main memory reference 100 ns 20x L2 cache, 200x L1

Compress 1K bytes with Zippy 3,000 ns

Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms

Read 4K randomly from SSD* 150,000 ns 0.15 ms

Read 1 MB sequentially from memory 250,000 ns 0.25 ms

Round trip within same datacenter 500,000 ns 0.5 ms

Read 1 MB sequentially from SSD* 1,000,000 ns 1 ms 4X memory

Disk seek 10,000,000 ns 10 ms 20x datacenter roundtrip

Read 1 MB sequentially from disk 20,000,000 ns 20 ms 80x memory, 20X SSD

Send packet CA->Netherlands->CA 150,000,000 ns 150 ms

Credit

Peter Norvig: http://norvig.com/21-days.html#answers

Off The CPU Die SUCKS … a LOT!!

Another Way to Look At It

L1 cache reference : 0:00:01 (one second)Branch mispredict : 0:00:10L2 cache reference : 0:00:14Mutex lock/unlock : 0:00:50Main memory reference : 0:03:20 !!!Compress 1K bytes with Zippy : 1:40:00Send 1K bytes over 1 Gbps network : 5:33:20Read 4K randomly from SSD : 3 days, 11:20:00Read 1 MB sequentially from memory : 5 days, 18:53:20 !!!Round trip within same datacenter : 11 days, 13:46:40Read 1 MB sequentially from SSD : 23 days, 3:33:20 Disk seek : 231 days, 11:33:20 !!!Read 1 MB sequentially from disk : 462 days, 23:06:40 !!!Send packet CA->Netherlands->CA : 3472 days, 5:20:00

Updatable Columnstore Index

Only One of these

Hopefully few of these

Hopefully lots of these, vast majority with ~1M row segments!

The Column Store Index IS the data now!

Table consists of column store and row store

DML (update, delete, insert) operations leverage delta store

INSERT Values◦ Always lands into delta store

DELETE◦ Logical operation◦ Data physically remove after REBUILD

operation is performed. UPDATE

◦ DELETE followed by INSERT. BULK INSERT

◦ if batch < 100k, inserts go into delta store, otherwise columnstore

SELECT ◦ Unifies data from Column and Row stores -

internal UNION operation. “Tuple mover” converts data into

columnar format once segment is full (1M of rows)

Updatable Columnstore Index

What’s being delivered◦ Clustered and updateable columnstore index◦ Columnstore archive option for data compression◦ Global batchaggregation◦ MUCH better Batch Mode operations, fewer limitations

Main benefits◦ Real-time super fast data warehouse engine◦ Ability to continue queries while updating without the

need to drop and recreate index or partition switching◦ Huge disk space saving due to compression◦ Better performance and more efficient (less memory)

batch query processing using batch mode rather than row mode

Columnstore Enhancements Summary

In-Memory OLTPSQL Server engine

Low latency ,Up to 10 times the improvement in performance

2 to 5 times the improvement in scalability

Takes advantage of investments in SQL Server

New high-performance, memory-optimized online transaction processing (OLTP) engine integrated into SQL Server and architected for modern hardware trends

• Integrated into SQL Server relational database• Full ACID support• Memory-optimized (no B-Tree indexes or buffer pools)• Non blocking multi-version optimistic concurrency

control (no locks or latches)• T-SQL compiled to native code

Memory-optimized table

file group

In-Memory OLTP engine: Memory-optimized tables & indexes

Native compiled SPs & schema

In-Memory OLTP compiler

TDS handler and session management

Transaction log Data file group

Buffer pool for tables & indexes

Poc/plan cache for adhoc, T-SQL; interpreter, plans, etc.

Parser, catalog,

optimizer

* All performance critical data already fits in memory* Transaction locking or physical latching causing stalls and blocking* Relatively short-lived transactions* High degree of concurrent transactions from many connections Performance problems isolated to subset of tables and SPs

OLTP Rocket Science

bit All integer types: tinyint, smallint, int, bigint All money types: money, smallmoney All floating types: float, real date/time types: datetime, smalldatetime,

datetime2, date, time numeric and decimal types All non-LOB string types: char(n), varchar(n),

nchar(n), nvarchar(n), sysname Non-LOB binary types: binary(n), varbinary(n) Uniqueidentifier

In-Memory OLTP Limitations

No DML triggers No FOREIGN KEY or CHECK constraints No IDENTITY columns (added in CTP 2) 8060 byte hard limit for row length No UNIQUE indexes other than for the PRIMARY

KEY A maximum of 8 indexes, including the index

supporting the PRIMARY KEY No schema changes are allowed once a table is

created (drop/recreate) Indexes are always and ONLY created at as part

of the table creation

In-Memory OLTP Limitations

What’s being delivered◦ Use of non-volatile drives (SSD) to extend buffer pool◦ NUMA-Aware large page and BUF array allocation

Main benefits◦ BP extension for SSDs

Improve OLTP query performance with no application changes No risk of data loss (using clean pages only) Easy configuration optimized for OLTP workloads on commodity

servers (32 GB RAM) Scalability improvements for systems with more than

eight socketsExample:

ALTER SERVER CONFIGURATION

SET BUFFER POOL EXTENSION ON(FILENAME = 'F:\SSDCACHE\EXAMPLE.BPE‘, SIZE = 50 GB)

SSD Buffer Pool Extension

SSD Buffer Pool Extension

Four new permissions◦ CONNECT ANY DATABASE(server scope)◦ IMPERSONATE ANY LOGIN (server scope)◦ SELECT ALL USER SECURABLES (server scope)◦ ALTER ANY DATABASE EVEN SESSION (database scope)

Main benefit◦ Greater role separation to restrict multiple DBA roles◦ Ability to create new roles for database administrators who are not sysadmin◦ Ability to create new roles for users or apps with specific purposes

Examples:

1.Database administrator but cannot see user data2.Database administrator but cannot modify logins3.Database administrator but cannot impersonate any logins4.New roles (e.g. auditors) to read all data but not admin rights5.New roles to read all metadata for monitoring purposes (e.g. SCOM) but cannot see

user data6.New roles for middle tier app to impersonate any login except sa, sysadmin, or other

high privilege logins

Separation of Duties Enhancement

SQL 2014 Scalability

Up 640 logical processors

64 vCPUs/VM

1 TB memory/VM

64 nodes/cluster

Network virtualization provides flexibility and isolation

Assign minimum and maximum bandwidth

Storage virtualization

Enterprise-class network storage on standard hardware

Storage tiering for higher performance

SQL Server 2014• Resource Governor adds

IO governance

• SysPrep at cluster level (prepare even CLUSTER!!)

Windows Server 2012 / R2• Hyper-V

• Storage Spaces

• NIC Teaming

• Online VHDX Resize

• Live migration

• Network & Storage QoS

Pool 1

Pool 2

Pool 3

Ability to differentiate workloads Ability to monitor resource usage per group Limit controls to enable throttled execution

or prevent/minimize probability of “run-aways”

Prioritize workloads Provide predictable execution of workloads Specify resource boundaries between

workloads

Resource Governor Goals

Represents physical resources of server Can have one or more workloads

assigned to pool Pools divided into shared and non-shared Pools control min/max for CPU/memory

and now IOPS

Resource Pools

CREATE RESOURCE POOL pool_nameWITH( [ MIN_CPU_PERCENT = value ][ [ , ] MAX_CPU_PERCENT = value ] [ [ , ] CAP_CPU_PERCENT = value ] [ [ , ] AFFINITY {SCHEDULER = AUTO | (Scheduler_range_spec) | NUMANODE = (NUMA_node_range_spec)} ] [ [ , ] MIN_MEMORY_PERCENT = value ][ [ , ] MAX_MEMORY_PERCENT = value ] [ [ , ] MIN_IOPS_PER_VOLUME = value ][ [ , ] MAX_IOPS_PER_VOLUME = value ])

What’s being delivered◦ Add max/min IOPS per volume to Resource Governor

pools◦ Add DMVs and perfcounters for IO statistics per pool

per volume◦ Update SSMS Intellisense for new T-SQL◦ Update SMO and DOM for new T-SQL and objects

Main benefits◦ Better isolation (CPU, memory, and IO) for

multitenant workloads◦ Guarantee performance in private cloud and hosters

scenario

IO Resource Governance

SysPrep: Why Invest?Support SQL Server images in Azure Gallery

-Provide quick and flexible SQL Server provisioning for IaaS scenarios-Support SQL Server configuration as part of the provisioning process-Need to be faster than full installationRemove limitations that currently exist

This has been long requested by customers

VM OS Selection

CREATE VIRTUAL MACHINE

VM OS Selection

SQL Server 2012 Web Edition Service Pack 1 (64-bit) on Windows Server 2008 R2 Service Pack 2. This image contains the full version of SQL Server, including all components except Distributed Replay, Always On, and Clustering capabilities. Some SQL Server components require additional setup and configuration before use. Medium is the minimum recommended size for this image. To evaluate advanced SQL Server 2012 capabilities, Large or Extra-Large sizes are recommended.

Microsoft SQL Server 2012 Standard ...

Microsoft SQL Server 2012 Evaluatio ...

Microsoft SQL Server 2008 R2 Web E...

Microsoft SQL Server 2012 Web Editi...

Microsoft SQL Server 2008 R2 Standa...

1 2 3

SysPrep Recent Enhancements Sysprep support for

◦ Database engine◦ Reporting Services◦ Analysis Services◦ Integration Services◦ Management Tools

(SSMS)◦ Other Shared Features◦ Performance

Improvements

Delivered in SQL Server 2012 SP1 CU2

Sysprep for SQL Server Cluster What’s being delivered:

◦ Extensions to SQL Server SysPrep functionality in order to support image based deployment of clustered SQL Server instances

Main benefit:◦ Supports full automation of SQL Server Failover Cluster

deployment scenarios◦ Reduces deployment times for SQL Server Failover Clusters◦ Combined together, these features allow customers to

automate the provisioning of SQL Server Failover Clusters both on-premises and through Infrastructure as a Service (IaaS)

◦ Built on top of SQL Server 2012 SP1 CU2 Sysprep enhancements

High Availability

Multi-DB failover, manual or automatic

Up to 8 secondary replicas

100% availability during all online operations

HA without expensive SANs

Offload reads & backups to secondaries

Deployment & management wizards

Monitoring dashboard within SSMS & System Center

SQL Server 2014• Enhanced AlwaysOn

• Replica Wizard (create

replika on Windows Azure!)

• Online Ops

Windows Server 2012/R2• Clustered Shared

Volume support

• VHDX support

• RefS support

System Center 2012 R2• Manage on-premises

and cloud apps

Primary Secondary 2Running BI Workloads

Secondary 1Running Back-

ups

Synchronous data

movement

Asynch

ronous

data

movementAG_Listener

What’s being delivered:◦ Increase number of secondaries to 8 (from 4), 2

SyncReplicas◦ Increased availability of Readable Secondaries◦ Support for Windows Server 2012 Cluster Shared Volumes

(CSV)◦ Enhanced Diagnostics

Main benefits:◦ Further scale-out read workloads across, possibly geo-

distributed, replicas ◦ Use readable secondaries despite network failures

(important in geo-distributed environments)◦ Improve SAN storage utilization

Avoid drive letter limitation (max 24 drives) via CSV paths Increase resiliency of storage failover

◦ Ease troubleshooting

AlwaysOn in SQL Server 2014

What’s being delivered:◦ Single partition online index rebuild(!!)◦ Managing Lock Priority for table SWITCH and Online

Index Rebuild◦ Built-in diagnostics

Main benefits:◦ Increased concurrency and application availability◦ New partition-level granularity for online index rebuild

allows lower resource (CPU/memory) usage for customers who need 24/7 access

◦ Provide customers greater control over impacting running transactions if using switch partition or online index rebuild commands

Online Ops Enhancements

On-Prem and/or Cloud (Hybrid)

Platform for Hybrid Cloud

Hybrid Cloud SolutionsEnable new hybrid scenarios like cloud backup and cloud DR to reduce costs and improve on-premises DR

Easy On-Ramp to CloudEasily migrate and run SQL Server in the cloud to gain benefits of cloud computing

Complete & ConsistentDeploy a complete and consistent data platform from on-premises to cloud

Hybrid Cloud Solutions

Manual or Automatic

At an instance level with point in time restore

Measures DB usage patterns to set backup frequency

Fast disaster recovery (low RTO)

Easy to deploy & manage

Cloud bursting

Greater global reach

Better isolation of internal assets

On-Premise Network

Domain Controller

Periodic Snapshot (automatic/manual)

SQL Server

Restore to VM

Geo Replication

*CAPEX –(Capital Expendures -investments for IT Solutions)**OPEX -(Operational Expendures - cost for operate the IT solutions




Cloud bursting



On-Premise Network

Domain Controller

VPN Tunnel (Windows Azure Virtual Network)Asynchronous Commit

Primary Replica Secondary Replica

Synchronous Commit

Run backups

Run BI reports

Manual or Automatic






Cloud bursting



Manual or Automatic



On-Premise Network

Domain Controller(authenticating all users on-prem & cloud)SQL Server

Lin

ked

Serv

er

Web Server

Complete & Consistent Data Platform

Microsoft Cloud

SQL Server in a

Windows Azure VM

SQL Database

service

HDInsightTablesBlobs

SQL Server on Windows

Server

PDW Appliance

SQL Server in a

Windows Server VM

BI features in O365

SQL Server in a

Windows Server VM

3rd Party

HD Insight Server

Complete & Consistent Data Platform

OTHER (not a complete list!) Backup Encryption

◦ AES 128/192/256, Triple DES◦ DO NOT LOSE YOUR CERTIFICATE!◦ No appending to backup◦ This IS Standard Edition Feature

TempDB write ‘deferal’ (Bob Dorr blog post)

SQL Server Data & Log Files in Windows Azure Storage (silly?)

OTHER - BI Cra … er, Stuff :-)Self-Serve BI

◦PowerQuery, PowerPivot, PowerView, PowerMap, PowerPotty

Tabular/BISM in SSASHDInsightPDW V2, PolyBase

Kevin G. Boles◦ @TheSQLGuru◦ [email protected]

Don’t forget about the #sqlhelp hash tag Don’t forget about free downloads on

SQLSaturday.com Schedule Pages

QUESTIONS??

mailto:[email protected]

Sq l 2014 New Features

Documents

Transcript of Sq l 2014 New Features