SQL Server Notifications: Invalidating Cached Results Michael Pizzo Software Architect Microsoft...

SQL Server Notifications: Invalidating Cached ResultsMichael PizzoSoftware ArchitectMicrosoft [email protected]

Session Code: Dat405

AgendaBuilding a Scalable EnterpriseWhidbey SqlNotifications and CachingImplementingSummary

Tools

Client Application Model

Avalon Windows Forms

Web & Service Application Model

ASP.NET / Indigo Win FSCompact

FrameworkYukon Mobile PC Optimized

System.Help

System.Drawing

System.NaturalLanguageServices

Data Systems Application Model

Presentation Data

Mobile PC & Devices Application Model

Communication

Command Line

NT Service

DataSet

Mapping

ObjectSpaces

ObjectSpace

Query

Schema

Item

Relationship

Media

Audio

Video

Images

System.Messaging System.Discovery

System.DirectoryServices

System.Remoting

System.Runtime.Remoting

ActiveDirectory

Uddi

System.Web.Services

Web.Service

Description

Discovery

Protocols

System.MessageBus

TransportPort

Channel

Service

QueuePubSub

Router

System.Timers

System.Globalization

System.Serialization

System.Threading

System.Text

System.Design

Base & Application Services

Fundamentals

System.ComponentModel

System.CodeDom

System.Reflection

System.EnterpriseServices

System.Transactions

SecuritySystem.Windows.TrustManagement

System.Web.Security

System.MessageBus.Security

Authorization

AccessControl

Credentials

Cryptography

System.Web.Configuration

System.MessageBus.Configuration

System.Configuration

System.ResourcesSystem.Management

System.Deployment

System.Diagnostics

Configuration Deployment/Management

System.Windows System.WindowsSystem.Windows.Forms

System.Console

System.ServiceProcess

System.Windows.Forms System.Web System.Storage System.Data.SqlServ

er

Animation

Controls

Control

Design

Panel

Controls

Dialogs

SideBar

Notification

System.Windows

Documents

Text Element

Shapes

Shape

Ink

UI Element Explorer Media

System.Windows.Forms

Forms

Control

Print Dialog

Design

System.Web.UI

Page

Control

HtmlControls

MobileControls

WebControls

Adaptors

Design

Ports

InteropServices

System.Runtime

System.IO

System.Collections

Generic

System.Search

Annotations

Monitoring

Logging

Relevance

System.Data

SqlClient

SqlTypes

SqlXML

OdbcClient

OleDbClient

OracleClient

Core

Contact

Location

Message

Document

Event

System.Storage

System.Web

Personalization

Caching

SessionState

System.Xml

Schema

Serialization

Xpath

Query

Permissions

Policy

Principal

Token

System.Security

System.Collaboration

RealTimeEndpoint

TransientDataSession

SignalingSession

Media

Activities

HttpWebRequest

FtpWebListener

SslClientStream

WebClient

System.Net

NetworkInformation

Sockets

Cache

System.WebAdministration

Management

Navigation

Peer Group

Policy

Serialization

CompilerServices

Recognition

System.Speech

Synthesis

AgendaBuilding a Scalable Enterprise

Scalable Data-Driven Enterprise ApplicationsScaling SolutionsWeb Caching BenefitsCaching in ASP.NET 1.xCaching Dynamic Content

Whidbey SqlNotifications and Caching Implementing Summary

Scaleable Data-Driven Enterprise Applications

Data-Driven Enterprise ApplicationsWebServicesDynamic Web Applications

Product-driven storefrontSearch sitesInformational sites (weather, traffic, news, …)PortalsInternal lookup tables (zip codes, tax tables,…)Etc…

Increased Scalability==Reduced Contention for shared resources

In Data-Driven applications, data becomes the shared state that limits scalability

Scaling SolutionsPartitioning

Scale out by partitioning the data across multiple serversRoute request to the appropriate serverAppropriate when:

Results can be served from a single partitionAppropriate server can be easily identified

Replication CachingNormalized view is most appropriateFew results are common, so base data is candidateBase data too costly to invalidate, and must be maintainedRich query support required

Web Application CacheTransformations/processing between normalized data in Database and objects in cacheMany results (or candidate results) are commonConstruction of results are not too costly to be invalidatedCached objects can be adequately queried with ADO.NET or application code

These types of caching are highly Complementary

Web Caching BenefitsImprove Response Time

Reduce Code ExecutionReduce process hopsReduce managed/unmanaged transitionsReduce Kernel/User mode transitionsReduce database round-tripsEliminate server hops

Improve ScalabilityDatabase is generally bottleneck

Shared stateReduce number of requests to Database

Caching In ASP.NET 1.xObject Caching

ASP.NET Applications can add/remove objects from cache through dictionary functions or Cache APIs

DataSet ds = (DataSet)Cache[ProductID]; if (null == ds)

{ ds = new DataSet(); SqlDataAdapter adapt = GetDataAdapter(); adapt.Fill(ds);

Cache.Insert(ProductID, ds); }

Response CachingPage Directives can instruct ASP.NET to cache responses

<%@ OutputCache Duration="3600" VaryByParam="CategoryId;selection" %>

Fragment CachingPage Directives can instruct ASP.NET to cache values from individual controls

Entries invalidated based on Time (TTL, absolute time, time unused), File-event, or removal of other entries

Caching Dynamic ContentDelay in publishing new Content

i.e., new Products, CategoriesDelay in returning new information

i.e., from a WebServiceInconsistent results across WebFarm

Refreshed at different intervalsNo ability to invalidate

i.e., when prices change/invalid informationUsers not seeing their changes

i.e., configuration settings in a PortalWorkarounds

Coordinated absolute time expirationData that can accept latency

Explicit invalidationsWhere application controls changes to data

Agenda Building a Scalable Enterprise

Whidbey SqlNotifications and CachingSQL Server Query Processor NotificationsReceiving NotificationsObject Caching in ASP.NETResult Caching in ASP.NET

Implementing Summary

SQL Server QP Notifications

User Requests Notifications on QuerySets as command properties

Specifies SQL Service Broker queueSpecifies message to post on change

QP generates a “notification template”Supports common Selects, Stored procs, Batches containing notifiable queriesSupports same queries as indexed views

Must be deterministicMust not be overly complex

Well suited for small number of queries with many parameter valuesNOT Designed for large number of ad-hoc queries

Posts to SQL Service Broker when results changeDurable; doesn't require continuous connectionProvides activation model for processing message

Supported in SQL Server “Yukon”

Using QP Query Notifications

Create a CommandCreate a SqlNotificationRequest

Specify message, message queue, timeoutAssociate the SqlNotificationRequest with one or more commandsExecute the command(s)

Execution registers for notificationsUse ExecuteNonQuery to register without retrieving results

Query SSB message queue for your message

Poll or blocking “waitfor” querySynchronous or Asynchronous

Application

Query Processor Notifications

SSB

Client Machine Database Server

SqlNotificationRequest

SqlCommandTemplat

e

Execute()Execute()

SqlCommand Execute()Execute()

Products

SqlDependencySqlDependency uses QP Notifications to raise a client side event when results change

Registers with a runtime listener upon creationListens on HTTP or TCP port

Can be associated with one or more commandsCreates a SqlNotificationRequest under the coversSpecifies routing message and default message queue

Exposes an OnChanged EventNotification Dispatcher fires notification from database

Activated upon delivery of message to queueUses information in message to determine destination

Using SqlDependencyCreate a SqlDependencyRegister for the OnChanged eventAssociate SqlDependency with one or more commandsExecute the command(s)Event raised when results changepublic DataSet getData() public DataSet getData() {{ SqlDependency depend = new SqlDependency();SqlDependency depend = new SqlDependency(); depend.OnChanged += new EventHandler(ChangeEventHandler);depend.OnChanged += new EventHandler(ChangeEventHandler); ds = new DataSet();ds = new DataSet(); SqlDataAdapter adapt = getDataAdapter();SqlDataAdapter adapt = getDataAdapter(); depend.AddCommandDependency(adapt.SelectCommand);depend.AddCommandDependency(adapt.SelectCommand); adapt.Fill(ds);adapt.Fill(ds); return ds;return ds;}}static void ChangeEventHandler(Object Sender, EventArgs e) {static void ChangeEventHandler(Object Sender, EventArgs e) {

MessageBox.Show("Results Changed");MessageBox.Show("Results Changed");}}

Query Processor Notifications

NotificationDeliveryService

SSBApplication

DependencyListener

Client Machine Database Server

SqlCommand

SqlDependency

Template

OnChangeAddCommandDependency()AddCommandDependency()

Execute()Execute()

Invalidate(ID)Invalidate(ID)

SqlNotificationRequestProducts

SqlDependency

Object Caching With Data Invalidations

Objects placed in ASP.NET Object Cache can be dependent on database resultsUse new SqlCacheDependency

Supports QP NotificationsPass command into constructor

Creates a SqlDependency under the coversAlso supports polling for pre-Yukon databases

Specify database and tablename in constructorDatabase and tables must be configured to support

listener tables, triggers and stored proceduresCaching information added to web server configuration fileObject invalidated when any change is made to the dependent tableSee WS330 for more information

Using Object Caching With Data Invalidations

Generate the object to be cachedAssociate a SqlCacheDependency with each command (or table) used to generate the objectExecute the command(s) to generate the response

Add the object to the cache, specifying the SqlCacheDependency

DataSet ds = (DataSet)Cache[ProductID]; if (null==ds) { ds = new DataSet(); SqlDataAdapter adapt = getDataAdapter(); SqlCacheDependency cacheDep =

new SqlCacheDependency(adapt.SelectCommand); adapt.Fill(ds); Cache.Insert(ProductID, ds, cacheDep);}

Response Caching With Data InvalidationsASP.NET Responses can be dependent on database

resultsExplicitly associate SqlCacheDependency(s) with ResponseOr Specify as part of caching page directive

For query notifications, specify:SqlDependency=“CommandNotification”SqlDependency=“CommandNotification”

ASP.NET generates a SqlDependency, associates it with the ASP.NET generates a SqlDependency, associates it with the response, and adds the dependency to the application response, and adds the dependency to the application contextcontextCommands automatically enlist in contextCommands automatically enlist in context

For table dependencies, specify:SqlDependency=“database:tablename”SqlDependency=“database:tablename”

When used with II6.0, responses are pushed to Kernel Mode Response cache!

Using Response Caching With Data invalidations

Set up the dependency manuallySpecify that the page is to be cachedGenerate the page

Create a SqlCacheDependency for each command used to generate the response

Add the SqlCacheDependency(s) to the Response

<%@ OutputCache Duration=3600 VaryByParam=“none”<%@ OutputCache Duration=3600 VaryByParam=“none”SqlDependency=“CommandNotification" %>SqlDependency=“CommandNotification" %>

Response.Cache.SetExpires(DateTime.MaxValue);Response.Cache.SetExpires(DateTime.MaxValue);SqlDataAdapter adapt = getDataAdapter();SqlDataAdapter adapt = getDataAdapter();SqlCacheDependency cacheDep = SqlCacheDependency cacheDep =

new SqlCacheDependency(adapt.SelectCommand);new SqlCacheDependency(adapt.SelectCommand);adapt.Fill(ds);adapt.Fill(ds);Response.AddCacheDependency(cacheDep);Response.AddCacheDependency(cacheDep);

Or use the outputcache directiveSpecify: SqlDependency="CommandNotification" Set Command.NotificationAutoEnlist to false for any

commands you don’t want to notify on

ASP.NET Application/ASP.NET Application/WebServiceWebService

<%<%@ OutputCache @ OutputCache Duration=3600 Duration=3600 VaryByParam=“none”VaryByParam=“none”SqlDependency= SqlDependency= “CommandNotification” “CommandNotification” %>%>

Response Caching In ASP.NET

NotificationDeliveryService SSB

Http.Sys

Middle Tier Database Server

SqlCommand

ChangeDetectionExecute()Execute()

Invalidate(ID)Invalidate(ID)

ResponseCache

SqlCommand Execute()Execute()

IIS

Products

Web Application/ WebService Data Caching

Agenda Building a Scalable Enterprise Whidbey SqlNotifications and Caching

ImplementingDeciding What to CacheSqlClient Notification Support

Summary

Deciding What To CacheData Caching

Cache queryable data (i.e., DataSet) in Object CacheAppropriate when some subset of data is commonly useful across requests, objects

Object CachingCache common objects based on resultsAppropriate when same object may be used by multiple requestsAdded benefit if object creation/transformations is expensive

HTTP Response CachingSpecify that entire Response should be cachedAppropriate when entire response is common to multiple users

Fragment (Partial Page) CachingSpecify that some subset of response is to be cachedAppropriate when a subset of response is common, but also contains dynamic content

SqlClient Notification Support

New SqlNotificationRequest ObjectUsed to specify Message, Message Queue, Timeout

New SqlDependency ObjectAddCommandDependency() Method

Sets Notification information on a CommandOnChanged Event

Raised when any results changeDelivery properties

Protocol, Port, Timeout,…New SqlCommand Properties

New Notification PropertyUsed for setting SqlNotificationRequest object

NotificationAutoEnlist PropertyAllows SqlCommand not to be enlisted in Notification Context

Summary Of Microsoft Web Caching Architecture

Developer has a choice!Cache transformed data or objects in application cacheEasy to use

SqlDependency, ASP.NET Page DirectivesExplicit control

Use Page DirectivesEnlist commands by default

Specify commands that shouldn’t be enlistedSpecify dependent tables

Support for pre-Yukon ServersCreate explicit SqlCacheDependenciesCreate SqlDependencies

Specify Transport, Port, alternate message queueAssociate with individual command(s)

Use SqlNotificationRequestSpecify custom message queueQuery queue directly using blocking or unblocking, synchronous or asynchronous queries

Questions?

SQL Server ResourcesWeek long SQL Server “Ask the Experts” lounge in:

Foyer outside Room 309Support for SQL Server “Yukon” PDC Preview at SQLJunkies

http://www.sqljunkies.com/forumsSQL Server “Yukon” FAQ Blog at SQLTeam

http://yukonblog.sqlteam.com/

Other Key Resourceshttp://www.microsoft.com/sql/communityhttp://msdn.microsoft.com/sqlserver/34 world wide user groups,

http://msdn.microsoft.com/usergroups/find.aspOur most active SQL Server newsgroups,

Microsoft.public.sqlserver.programmingMicrosoft.public.sqlserver.serverMicrosoft.public.sqlserver.dtsMicrosoft.public.sqlserver.olapMicrosoft.public.sqlserver.setupMicrosoft.public.sqlserver.replicationMicrosoft.public.sqlserver.msde

http://www.sqljunkies.com/forums



http://yukonblog.sqlteam.com/

http://www.microsoft.com/sql/community

http://msdn.microsoft.com/sqlserver/

http://msdn.microsoft.com/usergroups/find.asp

SQL Server Notifications: Invalidating Cached Results Michael Pizzo Software Architect Microsoft...

Documents

Transcript of SQL Server Notifications: Invalidating Cached Results Michael Pizzo Software Architect Microsoft...