Developer’s guide understand caching

Developer’s guide

How to enhance your SP performance:

Understand caching

Shai Petel

VP Research and DevelopmentMCPD, MCT, MVP: SharePoint Server

[email protected] | @shaibs | http://kwizcom.blogspot.com

Topics

TOPICSWhy do we need cache?

Understand Caching

Other performance coding tips

SharePoint performance challenge

Pages• Dynamic list forms• Dynamic pages and page layouts

Content• Dynamic content structure• Extensible (field types)• Stored in Content DB

Customizations• Custom UI elements (Web parts, user controls, field controls)• Other API plugins (event handlers, workflows)

Customizable by design.

SharePoint Page Processing

What can be done better?

Minimizing round trips to DB on every call

Reduce ACL rules

Pre-compiling user controls and pages

Pushing supporting files from DB to each WFE

Saving the output HTML of pages that do not change frequently

Reusing DB connections and requests when possible

What does SharePoint already do?

Ghost files on WFE

Caches our .NET code in native code (If put in GAC)

Pre-compiles user controls

Shares context objects (SPContext.Current Site, Web, List) with all controls for a single request

Apply some sort of object caching by default (good example is BCS)

When is caching good for me?

Information updates

Security Views & Risk

Sources

Size

Topics


Understand Caching


Understand Caching

BLOB Cache Ghosting files Object Cache

Page output cache

Custom cache profiles

BLOB cacheWhat is?

WFE local disk-based cache One per web application Reduce database load and network traffic Limit by file types Built on demand Configure cache size PublishingHTTPModule Only document library Security trimmed

BLOB cacheWhen to use?

Ideal for publishing with many anonymous visitors.

Sites that contain lots of media assets that are read-only or rarely changes.

Support site that has PDF and Doc user manuals for download

Collaboration sites for creative team– don’t cache images

For super heavy use, store BLOB on a separate hard drive

DO NOT delete files directly from BLOB cache storage!

Edit web.config, “<BlobCache … />” existing tag

Flush BLOB cachePoSH:

STSADM:

C#:

<BlobCache location=“D:\BLOB\WebApp1\" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif| tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled=“true" />

$webApp = Get-SPWebApplication “http://mywebapp:port“[Microsoft.SharePoint.Publishing.PublishingCache]::FlushBlobCache($webApp)Write-Host "Flushed the BLOB cache for:" $webApp

stsadm –o setproperty –propertyname blobcacheflushcount –propertyvalue 11 –url http://mywebapp::port

Microsoft.SharePoint.Publishing.PublishingCache.FlushBlobCache( Microsoft.SharePoint.Administration.SPWebApplication.Lookup( new Uri("http://mywebapp:port") ));

BLOB cacheHow to configure?

Ghosting filesWhat Is?

No content in content DB table

Reduces load and (ab)use of DB and network

Ghost files != BLOB cache, yet similar

Updates only need to be done to local template file

When modifying a ghost file, only the differences will be stored (unghost)

Reset back to definition (reghost) is possible

Saves on expensive DB storage

Ghosting filesWhen to use?

Template files that do not change much, such as master pages, page layouts, forms, images, CSS files, JS files, ASPX files.

Recommended to use whenever you deploy files using a feature.

If you expect a lot of site-level customizations of the file, do not mark it as ghost.

Ghosting filesHow to configure?

Include the files you need in a module element

Set Type to Ghostable / GhostableInLibrary

<File Path="SiteAssets\ProjectFolder\ActionPage.aspx" Url="SiteAssets/ProjectFolder/ActionPage.aspx" Type="Ghostable" />

Object cacheWhat is?

Reduces traffic between WFE and DB Stores lists, libraries, site settings and page layouts, as

well as cross list queries Cost memory on WFE “Security tolerance” (“Cross list query multiplier”) Query DB 2 times! Portal super user and super reader Some API objects and OOB controls are cached

automatically by using the standard API, but there is more!

Object CacheWhen to use?

When performance is important, memory is cheap and minor data inconsistency is acceptable*

When your data does not update often

* You can enhance performance even more by setting object cache not to check for changes on every query

Object CacheHow to configure?

Requires “SharePoint Server Publishing” feature

Site collection level setting, web application memory cap

Set memory limit, duration, and “security tolerance”

Flush object cache is done via site collection administration web UI.


Configure Super User (must have full control) and Super Reader (must have full read access)

Default users are wrong!


Map users to service: Create users in central admin Next, use PoSH to specify super user and super reader

account names:

Perform IIS Reset to complete the operation.

$wa = Get-SPWebApplication -Identity "<WebApplication>“$wa.Properties["portalsuperuseraccount"] = "<SuperUser>“$wa.Properties["portalsuperreaderaccount"] = "<SuperReader>“$wa.Update()

Object cacheHow to use?

Add using to Microsoft.SharePoint.Publishing Run the following code

using (SPSite site = new SPSite("http://mywebapp:port")){ CrossListQueryInfo crossListQueryInfo = new CrossListQueryInfo(); crossListQueryInfo.Lists = "<Lists BaseType=\"5\" />"; //Load all items that have "important" in their title crossListQueryInfo.Query = "<Where><Contains><FieldRef Name=\"Title\" /><Value Type=\"Text\">important</Value></Contains></Where>"; crossListQueryInfo.RowLimit = 100; crossListQueryInfo.UseCache = true; crossListQueryInfo.ViewFields = "<FieldRef Name=\"Title\" />"; crossListQueryInfo.Webs = "<Webs Scope=\"SiteCollection\" />"; crossListQueryInfo.WebUrl = site.ServerRelativeUrl; CrossListQueryCache crossListQueryCache = new CrossListQueryCache(crossListQueryInfo); SiteDataResults results = crossListQueryCache.GetSiteDataResults(site, false);}

Page output cacheWhat is?

Native ASP.NET technology

Caches the HTML output of ASPX page

Caches multiple versions

When a page request is answered with a cached version: No calls to any data source No load/merge master page, page layout, field and web controls Does not run any server side code

Cache profiles A more precise and customized output cache mechanism on top of ASP.NET A list-style set of rules that can be applied to page layouts, site, site collection

Page output cacheWhen to use?

To be used on content pages (publishing / wiki)

When you have few contributors and a lot of readers / visitors

When pages do not change frequently

If you plan to have many requests for the same pages (landing page)

Page output cacheHow to configure?

Requires “SharePoint Server Publishing” feature

“Check for changes” option in cache profile NLB - set persistent sessions (or sticky sessions) Monitoring cache behaviour: check the “enable debug

cache information on pages”


Configure on web application (in web.config)

Configure at site collection, allow publishing sub-sites & page layoutsto override

<OutputCacheProfiles useCacheProfileOverrides="false" varyByHeader="" varyByParam="*" varyByCustom="" varyByRights="true" cacheForEditRights="false" />


Sitecollectionsettings


Sub site settings:


Page layout settings:


Create new cache profiles Use “Vary by X” to enable this same profile to store

different versions of the cache based on specific criteria:Header: by language for exampleQuery string parameter: http://site/page.aspx?ID=1 User rights: based on the ACL evaluation of each user. Users

with same permissions will get the same cached content.Custom: “browser”, or custom settings (code)

http://site/page.aspx?ID=1


New cacheprofile:


ASP.NET cacheability:Member name DescriptionNoCache Sets the Cache-Control: no-cache header. Without a field name, the directive

applies to the entire request and a shared (proxy server) cache must force a successful revalidation with the origin Web server before satisfying the request. With a field name, the directive applies only to the named field; the rest of the response may be supplied from a shared cache.

Private Default value. Sets Cache-Control: private to specify that the response is cacheable only on the client and not by shared (proxy server) caches.

Public Sets Cache-Control: public to specify that the response is cacheable by clients and shared (proxy) caches.

Server Specifies that the response is cached only at the origin server. Similar to the NoCache option. Clients receive a Cache-Control: no-cache directive but the document is cached on the origin server. Equivalent to ServerAndNoCache.

ServerAndNoCache Applies the settings of both Server and NoCache to indicate that the content is cached at the server but all others are explicitly denied the ability to cache the response.

ServerAndPrivate Indicates that the response is cached at the server and at the client but nowhere else. Proxy servers are not allowed to cache the response.

Custom cache profilesWhat is?

A custom class that inherits from IVaryByCustomHandler

Deployed to GAC and registered in global.asax

Builds a “cache key” string

Custom cache profilesWhen to use?

When you want to use output cache, but your application renders different HTML based on custom rules, like:

User profile association (Like: HR info, VaryByLocation, VaryByLastName)

Device/Environment rules (like: allow scripts? Allow CSS?)

Custom cache profilesHow to configure?

Register the event handler in global.asax file:

Pass string parameters to “vary by custom parameter” to trigger your handler (Optional)

//Register the VaryByCustom string in the Global.asax file.<%@ Assembly Name="Microsoft.SharePoint"%><%@ Assembly Name=“CacheExample.VaryByCustomExample"%><%@ Application Language="C#" Inherits="CacheExample.VaryByCustomExample" %>

Custom cache profilesHow to use?

Example, Vary by CSS availability, support XMLHttp, and is user site admin. Parameter(s) to be set in profile “Vary by custom parameter”: “SupportsCSS;SupportsXmlHttp;SiteAdmin”

using System;using System.Collections.Generic;using System.Linq;using System.Text;using Microsoft.SharePoint.ApplicationRuntime;using System.Web;using Microsoft.SharePoint;

namespace CacheExamples{ public class VaryByCustomExample : SPHttpApplication, IVaryByCustomHandler { … }}

Custom cache profilesHow to use?

//supported strings in "Vary by custom string" to look for in order for our logic to kick inpublic enum SupportedParams { SupportsCSS, SupportsXmlHttp, SiteAdmin }

public override void Init(){ base.Init(); this.RegisterGetVaryByCustomStringHandler( (Microsoft.SharePoint.ApplicationRuntime. IVaryByCustomHandler)this );}

public string GetVaryByCustomString(HttpApplication app, HttpContext ctx, string custom) { //look for parameters specified in the cache profile StringBuilder sb = new StringBuilder(); string[] strings = custom.Split(';'); foreach (string str in strings) { SupportedParams param; if (!Enum.TryParse<SupportedParams>(str, out param)) continue;//Not in our enum... Skip

switch (param) { case SupportedParams.SupportsCSS: sb.Append(ctx.Request.Browser.SupportsCss + ";"); break; case SupportedParams.SupportsXmlHttp: sb.Append(ctx.Request.Browser.SupportsXmlHttp + ";"); break; case SupportedParams.SiteAdmin: sb.Append(SPContext.Current .Web.UserIsSiteAdmin.ToString() + ";"); break; default: continue; } } return sb.ToString().TrimEnd(';');}

Topics


Understand Caching



SPList.Items performance trap

Poor Performing Methods and Properties

Better Performing Alternatives

SPList.Items.Count SPList.ItemCount

SPList.Items.XmlDataSchemaCreate an SPQuery object to retrieve only the items you want.

SPList.Items[System.Guid] SPList.GetItemByUniqueId(System.Guid)

SPList.Items[System.Int32] SPList.GetItemById(System.Int32)

SPList.Items.GetItemById(System.Int32)

SPList.GetItemById(System.Int32)

SPFolder.Files.Count SPFolder.ItemCount


Paginate large list queries

SPWeb oWebsite = SPContext.Current.Web;SPList oList = oWebsite.Lists["Announcements"];SPQuery oQuery = new SPQuery();oQuery.RowLimit = 10; //Limit # of items to under 2000int intIndex = 1;do { Response.Write("<BR>Page: " + intIndex + "<BR>"); SPListItemCollection collListItems = oList.GetItems(oQuery);

foreach (SPListItem oListItem in collListItems) Response.Write(oListItem["Title"].ToString() +"<BR>"); //Move to next page, same query – this is all that changes. oQuery.ListItemCollectionPosition = collListItems.ListItemCollectionPosition; intIndex++;} while (oQuery.ListItemCollectionPosition != null);


Query Throttling List items view threshold, for users and for site

administrators Developers override (for privileged users)

http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.spquerythrottleoption(v=office.14).aspx

“Happy hour” Limit # of lookup, person, or workflow status fields that

can be included in a single database query




Column indexing can help! Query throttling limits the number of DB rows that are

affected Query that filters or sorts on a non-indexed column

affects all list items, even if only 10 items return By indexing that column, only 10 rows in the DB will be

affected Indexing cost resources


Cache objects in memoryUse Page.Cache, per WFEOnly cache thread safe object

SPWeb, SPSite, SPListItemCollection – No!SPListItemCollection.GetDataTable() – Yes!Consider security when caching

Poor man’s cache dependency – use cache version number to invalidate your data once updated.


Multi-threaded, use lockprivate static object _lock = new object();public void CacheData() { SPListItemCollection oListItems; oListItems = (SPListItemCollection)Cache["ListItemCacheName"]; if(oListItems == null) { lock (_lock) { //Ensure the data not loaded by a concurrent thread while waiting for lock. oListItems = (SPListItemCollection)Cache[“ListItemCacheName”]; if (oListItems == null) { oListItems = DoQueryToReturnItems(); Cache.Add("ListItemCacheName", oListItems, null, DateTime.Now.AddSeconds(60), Cache.NoSlidingExpiration, CacheItemPriority.High, onRemove); } } }}


Cache on client Use cookie to store settings instead of reading them

every time Use view state to store data between postbacks Override SaveViewState/LoadViewState to “zip” large

data or “encrypt” sensitive information(I reduced 8MB viewstate to 65KB)


Use lazy loading Use scripting on demand<SharePoint:ScriptLink ID="scriptLink1" runat="server" Name="My_Script.js" LoadAfterUI="true" OnDemand="false" />Not on demand:<script type="text/javascript">// <![CDATA[document.write('<script type="text/javascript" src="/_layouts/my_script.js"></' + 'script>');// ]]></script>On demand:<script type="text/javascript">RegisterSod("my_script.js", "\u002f_layouts\u002fmy_script.js");</script>


Work with script on demand SP.SOD.registerSod(key,url); To call function from this script, use:

SP.SOD.executeFunc(key, functionName, fn); Script dependency:

SP.SOD.registerSodDep(key, dependOnKey);


Make sure your code runs after all dependency scripts finished loading:SP.SOD.executeOrDelayUntilScriptLoaded(func,

scriptName)

In your script files, notify when your classes finished registring:SP.SOD.notifyScriptLoadedAndExecuteWaitingJobs("m

y_script.js");


Enable Bit Rate Throttling If you are steaming a lot of large media files Limit rate those files are delivered to client to conserve

network bandwidth A part of IIS Media Services WMV, MP4, SWF files bit rates are detected

automatically Good for “you tube” type of solution

Topics

Why do we need cache?Understand CachingOther performance

coding tips

See also

Documentationhttp://technet.microsoft.com/en-us/library/cc261797.aspxhttp://msdn.microsoft.com/en-us/library/aa589700.aspx

* except for “BLOB” sub pagehttp://technet.microsoft.com/en-us/library/ee424404.aspxhttp://msdn.microsoft.com/en-us/library/bb687949(office.1

2).aspx

http://msdn.microsoft.com/en-us/library/ff798465.aspx Sean McDonough on BLOB Cache

http://sharepointinterface.com/2009/06/18/we-drift-deeper-into-the-sound-as-the-flush-comes/

http://technet.microsoft.com/en-us/library/cc261797.aspx

http://msdn.microsoft.com/en-us/library/aa589700.aspx

http://technet.microsoft.com/en-us/library/ee424404.aspx

http://msdn.microsoft.com/en-us/library/bb687949(office.12).aspx



http://msdn.microsoft.com/en-us/library/ff798465.aspx

http://msdn.microsoft.com/en-us/library/ff798465.aspx





Questions?

=2B |! 2B ? Cache : Live;

?

Developer’s guide understand caching

Technology

Transcript of Developer’s guide understand caching