Optimizing and Accelerating your SharePoint Farm

©2012 Quest Software, Inc. All rights reserved..

Chris McNulty

SharePoint Strategic Product Manager

April 2012

SharePoint Saturday Twin Cities

Optimizing and Accelerating your SharePoint Farm

©2012 Quest Software, Inc. All rights reserved. Confidential.

2


2

AmericasEMEAAPJa

Sales/MrkgR&DSupport

178 CountriesAll VerticalsGlobal 200SMB

Database, Monitoring, Data Protection, User Workspace/Virtualization, Windows (SharePoint, AD, Messaging), Identity Mgmt

60 Offices, 3 HQs

3600+ Employees

100,000+ Customers

Quest Market Presence

Multiple Business Lines

©2012 Quest Software, Inc. All rights reserved. Confidential.

3


3

4


@cmcnulty2000

Chris McNulty• SharePoint Strategic Product

Manager at Quest Software

• 10+ years with SharePoint

• 20 years consulting (led KMA SharePoint practice) and financial services technology (Santander, John Hancock/Manulife, GMO, State Street)

• MBA in Inv Mgmt from Boston College

• Write and speak often on Microsoft IW technologies (blogs & books)

• MCSE MCTS MSA MVTSP MCC

• Hiking, cooking, playing guitar, colonial history, photography

• My family: Hayley, three kids (17, 8, 5) and my dog Stan

5


@cmcnulty2000

Pictures

• [taken out to save space!]

In the spirit of…

7


@cmcnulty2000

Hometown News• National: Newark NJ Mayor in Rescue (MSNBC)

• Newark, N.J., Mayor Cory Booker suffered second-degree burns and smoke inhalation as he helped rescue one of his next-door neighbors from a burning building Thursday. “There was a time when I didn’t think we’d make it out of there,” Booker told media at a news conference Friday morning, describing the rescue as his "proverbial 'come to Jesus' moment."

• Local: (Suffolk Times, NY) : Cops: 89-year-old woman crashes car into pole in Cutchogue • An 89-year-old woman crashed her car into a utility pole on Main

Road in Cutchogue Thursday afternoon, Southold Town Police officials said. At about 3:30 p.m., Margaret Spicijaric of Southold was traveling alone in a silver 2002 Honda heading eastbound before crashing into a utility pole, officials said.

• Weather (Minneapolis area, from weather.com)• Sun and clouds mixed with a slight chance of thunderstorms

during the afternoon. Warm. High 73F. Winds S at 5 to 10 mph. Chance of rain 30%.

• Overnight lows 56, chance of rain or thundershowers 90%

http://media.timesreview.com.s3.amazonaws.com/suffolktimes/files/IMG_4226.jpg

8


@cmcnulty2000

Agenda

• Architecture, Design & Planning

• Support• Monitoring and Operations

• Optimization

• Patching

• SQL Maintenance

• Metrics

• Best Practices

9


@cmcnulty2000

Presentation Governance

• Out Of Scope• Deep Dives (e.g. PowerShell, BI, Upgrade, SQL DBA)

• Development

• Customization

• Power User (e.g. Library Customization, Designer Workflows, etc.)

• Rules• Move fast, PowerPoint is shared –

http://slidesha.re/GztaOY

• Questions – time permitting during session

• Any time after session – email etc. - @cmcnulty2000

http://slidesha.re/GztaOY

Architecture and Design

11


Server Farm – Web Front End

• Typical Roles:• http services

• Search query

• Scaling• Add servers to load

balanced cluster

• Performance Optimization

• RAM

• Easily virtualized

12


Server Farm - Application Server

• Typical Roles:

• Search index/crawl

• Excel calculation

• User profiles

• Managed Metadata

• Scaling

• Add search servers and partitions

• Move shared services to dedicated servers


• CPU

13


Server Farm - Database

• Typical Roles:• Data storage

• SQL Reporting

• Scaling• Add storage capacity


• Disk I/O

14


Sizing - Single Server

• Typical Roles:• Small teams

• Small document pools

• Considerations• Performance & fault

tolerance less of a concern

• SQL & Web on same system

• Search not a core function

15


Sizing - Medium Farm

• Typical Roles:• 100-10,000 users

• 10,000 – 1MM documents

• Scenarios• Enterprise portal

• Large scale collaboration

• Broader applications platform

• Larger external search pool

• Mix and match internal external front end servers on common content databases

16


Sizing - Large Farm

• Typical Roles:

• Large distributed enterprise users (10000+)

• Large pools of documents (>1MM)

17


Sizing – No Servers – Office 365Office 365 Enterprise Plans

E1 E2 E3 E4

SharePoint Online

√ √ √ √

Office Web Apps

√ √ √

Local Copy of Office Professional 2010 Plus

√ √

Forms Services, Vision Services, Access Services

√ √

Monthly cost per user (USD)

$10 $16 $24 $27

• Constraints/Unavailable• Custom, non sandbox solutions

• Power Pivot

• SQL Server Reporting Service Integration

• Business Connectivity Services (OK for web services- based remote data in O365 BCS.)

• FAST Search Server Integration

• Web Analytics

• Site collections greater than 100GB

18


Logical Components

• High capacity!

• Maximums

• 250,000 sites per site collection

• 5,000 site collections per content DB

• 200GB max content DB (single site collection)

• >200GB post SP1

• 300 Content DBs per web application

• 30MM documents/library

• 2GB document size

• 2011 News• 14TB Demo

19


@cmcnulty2000

Disk SizingContent Search

Initial Content Size XXX GB External Crawl Size YYY GB

Initial User Pool U

User Collab Size .25GB

n YR Growth Rate – Archive Rate

G%

End Content Size XXX (1+G)n = ECS End Search Size YYY (1+G)n = ESS

End User Collab Size .25 * U * (1+G)n = EUCS

Content DBs ECS + EUCS

Search DBs .05 * (ECS + EUCS + ESS)

Search Index Files .05 * (ECS + EUCS + ESS)

• Inputs: Size of SharePoint content and non-SharePoint content included in search

• For DBs, don’t forget transaction logs, disk dumps (if used for backup) which can add 1-3X.

• In SAN or virtual environments, not all disk need be provisioned early

Content will grow; searching remote stores saves disk but isn’t free

20


@cmcnulty2000

Memory Sizing

• Web Front End (WFE)• 8GB minimum

• 12GB larger

• 16GB max

• Application• 8-12GB

• SQL• 8-24GB

• HP Sizing Tool• http://

h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html

• Don’t forget about the swap files (1-2X size of RAM)

http://h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html

http://h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html

21


@cmcnulty2000

Psst…about SQL…

• Sidebar on SQL Disk optimization• RAID 1: Mirroring (Wastes disk)

• RAID 5: Parity Bit (write performance hit)

• RAID 10 Stripes across mirrors (costly)

• Physical location of data, log, temp and/or backup files. If virtualization or SAN technology doesn’t isolate the disks, not much performance gain,

• Performance optimization/fault tolerance by:• RAID1 on boot disks

• RAID5 on data disks

• RAID10 on log disks

• No RAID, or RAID 5 on backup disks

• RBS reduces size (and count) of content databases but doesn’t reduce size of total storage

• IOPS, IOPS, IOPS!

22


@cmcnulty2000

Database sizing

• Automatic database growth:• 50-100MB clumps – not by percentage.

• Using a small size leads to more frequent, but smoother, steady state growth.

• Presize tempdb to about 20% the size of the single largest content database.

@cmcnulty2000

MonitoringMonitoring and Operations

24


Proactive Issue Resolution

Developer Dashboard• Empower developers and

users

Integrated Health Analyzer• Runs when necessary

• Alerts anomalies

• Fixes when it can

Web Analytics• User usage

• Resource usage

25


Logging, Monitoring, and Alerts

• Unified Logging

• Out-of-the-box reports

• Richer Web Analytics

• Open Schema

• SCOM Integration

• PLUS• Developer Dashboard

• Health Analyzer

ULS Logs

Windows Events

Page requests

Feature Logging

Health data

Logging DB

http://images.google.com/imgres?imgurl=http://api.ning.com/files/MSkY5K50pB2mD02AgzOnIUnjFf7Du4hPd30u0JfqdoziCaWWJlO9PKOgVxtZtCdLuGNvfOOTqZzFrLh-mvFYsOKg91gxn0m7/Excel.jpg&imgrefurl=http://datashaping.ning.com/profile/VincentGranville&usg=__sTBSrEUG-RCSVa5bBuRBphZGCTU=&h=321&w=388&sz=13&hl=en&start=2&sig2=AvI5TaaWsHWb7tIz6Kqteg&um=1&tbnid=KNBphhuWmyfA5M:&tbnh=102&tbnw=123&prev=/images?q=excel&hl=en&sa=N&um=1&ei=JBEWSvbOJoTUswOOnNDhCA

27


Developer Dashboard Improve customized solutions with the Developer Dashboard

SQL Queries Performance

Memory Usage

Data-Request Trace

28


Monitoring – WSS_Logging

• Query Database Views Directly

• Requires Timer Jobs Enabled• Diagnostic Data Provider: Trace Log

• Diagnostic Data Provider: Event Log

• ULS Configuration Matters

• Database will GROW!

• Aggregates from ALL Servers

• Sample: • SELECT * FROM [WSS_Logging].

[dbo].[ULSTraceLog] WHERE CorrelationID = '04377DAE-C2FD-4DBE-A57E-101B3005059E'

30


Development Support – Three RegionsDevelopment

• often internal to developers

• problem reproduction that require advanced inspection tools (e.g. Visual Studio) are done here

• permissions can be looser, may have multiple environments for multiple developers

• sensitive data from production cannot be copied here without masking or customer signoff

• changes here can be deployed ad hoc

Staging/Test

• no Visual Studio, no MS Office

• match/mirror production as closely as possible; match hardware/system performance as closely as practical

• security permissions match production

• any sensitive data copied here stays under production-grade controls

• test accounts should be created in a separate OU if possible

• changes here can only be delivered and deployed from source control and according to production release methods

Production

• optimized hardware configurations

• highly secure• no use of user rotating

password accounts as service accounts

• changes here can only be delivered and deployed from source control and according to production release methods

31


demoMonitoringDeveloper DashboardHealth AnalyzerLogging and Reporting

@cmcnulty2000

OptimizationOptimization

33


@cmcnulty2000

Optimization

• IIS Bit Rate Throttling (Video)• Part of IIS Media Services 2.0 or greater

• Download http://www.iis.net/download/BitRateThrottling

• Configured through IIS Manager

• Large File Upload Support:• Central Admin: 600MB Upload Size

• IIS7 Connection Timeout 65536 seconds

• Web app web.config = change line to • <httpRuntime maxRequestLength="2097151" executionTimeout="999999" />

• in 14hive\template\layouts\web.config, change above line to same

• Finally, as per http://support.microsoft.com/kb/944981/en-us add section to web.config to allow 600MB content length (it’s a new requirements in IIS7).• <system.webServer>

• <security>

• <requestFiltering>

• <requestLimits maxAllowedContentLength="629145600"/>

• </requestFiltering>

• </security>

http://www.iis.net/download/BitRateThrottling



http://support.microsoft.com/kb/944981/en-us%20add%20section%20to%20web.config

34


@cmcnulty2000

Optimization

• Disk-based BLOB Caching• Local store for audio/video, PDF other frequent read only files

• Edit in Web.config (C:\Inetpub\wwwroot\wss\virtualdirectories\...)

• <BlobCache location="" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />

• Location = Local Disk Location

• maxSize = GB

• Enabled = true

• Different from RBS/EBS!

• For publishing sites

35


@cmcnulty2000

Patching – Notes

• Test before installation!!!

• Cumulative Updates every two months

• Service Pack every 6-18 months

• Service Pack 1 REQUIRES at least the June 2011 CU

• December 2011 CU • iOS 5 Mobile Safari support for Performance Point

• Fixed Administrator updates to user profile pictures

• February 2012 CU most current

• Check my blog for latest: http://www.chrismcnulty.net/blog/Lists/Categories/Category.aspx?CategoryId=5&Name=Version-Build Numbers

36


@cmcnulty2000

SQL Maintenance

• Backups• Local Disk – easy but storage intensive

• Agents – remote, requires extra software

• RBS Maintenance• BLOB Orphans

• Log Sizing• Full logged (default) generates huge t-logs

• Simple doesn’t but prevents point in time restore

• Maintenance Plans

37


@cmcnulty2000

Metrics and Results

• IOPS <400• Improve disk I/O or reduce content database sizes

• Processor/CPU utilization >60%• You may need to add servers or redistribute services to reduce the load.

• Available memory <50%• Again, add RAM.

• Disk average reads/writes >15ms. • ~66.67 IOPS Common on virtualized SQL servers. That’s slow, and predicts

reduced performance even for smaller content databases on a SQL server. Use physical disk or move the volume to dedicated storage.

• Sustained network bandwidth higher than 25%. Network latency output queue >0• Network can’t keep up with users or the server. Add physical switched Ethernet

ports, adding servers to divide the load.

• Or move a virtualized guest server to a new host.

38


demoIOMeterPerformance Monitor

Best Practices

40


@cmcnulty2000

Troubleshooting – Top Support Questions

• Users Receive “Cannot Connect to Configuration Database” Web Page• SharePoint farm account is locked out

• No one can upload anything but site is up• Database disk volume is full – check transaction logs, backups

• In virtualized environment, host file systems may be full

• I can’t find a document I think I should see; Someone can’t see a file I just uploaded• Security and permission variations

• Document “movement” (a/k/a ECM) try search by name or Document ID. Check ECM logs/audits

• Confirm permissions, and make sure document is checked in (Required properties may be missing)

41


@cmcnulty2000

Troubleshooting – More Support Questions

• Repeated requests to re-enter Windows credentials• Add to Local intranet zone, add site, custom level, automatic login with

current user name and password (it’s the last thing in the item list)

• OR Trusted sites

• My workflow didn’t start• Recycle timer service

• “FixSharePoint.exe” = IISReset & Timer Service Recycle

• I’m not seeing the right search results• Confirm that crawls are running and complete by checking crawl logs;

restart a full crawl if crawls finish OK

• I need a file back that I deleted• Recycle Bin Recovery

• Use Backup & Restore

42


@cmcnulty2000

Miscellaneous Tips and Tricks I• Bad web parts on page

• ?Contents=1 on end of URL

• HyperV & DiskPart• From the run menu type "diskpart.exe" to enter the command

line utility to resize disk partitions in Windows Server 2003.

• The command list volume will show you all the available volumes.

• Select your volume as shown below. select volume 1 corresponds to the "D" volume.

• Extend the volume with the extend command; the partition will be immediately extended under the Disk Management snap in.

• User Profile Sync• DirSync permissions in AD are essential (Replicating Directory

Changes )

• PSCONFIG for a stuck wizard• psconfig.exe -cmd upgrade -inplace b2b -wait

43


@cmcnulty2000

Miscellaneous Tips and Tricks II• How do I make a bigger template?

• stsadm -o setproperty -propertyname max-template-document-size -propertyvalue size_in_bytes

• Auditing• In Central Admin | Security, configure Information

Management Policy (/_admin/Policyfeatures.aspx). Make sure Auditing is Available

• At the Site Collection Audit Settings screen (in Site Collection Administration) select the events you want to audit

• Site Setting | Site Collection Administration | Audit Log Reports

• Service Account Management in Central Admin• Security | Managed Service Accounts

• When all else fails – web.config!!!!

http://kmamssd02:12345/_admin/Policyfeatures.aspx

http://kmamssd02:12345/_admin/Policyfeatures.aspx

48


•Questions

•Contact Me

•SharePint

•Prizes

49


@cmcnulty2000

More information

• SharePoint architecture design patterns in Chris’ e-book entitled SharePoint 2010 Consultant’s Handbook – A Practical Field Guide • Get your free copy here http://www.quest.com/get-chris-book

http://www.quest.com/get-chris-book

http://www.quest.com/get-chris-book

50


@cmcnulty2000

Thank you…

• Contact• Email [email protected]

• Blog http://www.chrismcnulty.net/blog

• Also http://www.sharepointforall.com

• Twitter: @cmcnulty2000

• LinkedIn:http://www.linkedin.com/in/cmcnulty

• Upcoming:• April 2012 – The Experts Conference San Diego

• May 2012 – New England SPUG

• June 2012 – SPS Silicon Valley, New Hampshire SPUG

• July 2012 – SPS Toronto, SharePoint Saturday NYC

http://www.chrismcnulty.net/blog

http://www.chrismcnulty.net/blog

http://www.sharepointforall.com/

http://www.linkedin.com/in/cmcnulty

http://www.linkedin.com/in/cmcnulty

@cmcnulty2000

Optimizing and Accelerating your SharePoint Farm

Technology

Transcript of Optimizing and Accelerating your SharePoint Farm