Too Slow?: Use VS2010 Profiling & Load Testing to Manage Performance Issues

Benjamin Day

Benjamin Day

• Consultant, Coach, Trainer• Scrum.org Classes

Professional Scrum Developer (PSD) Professional Scrum Foundations (PSF)

• TechEd, VSLive, DevTeach, O’Reilly OSCON• Visual Studio Magazine, Redmond Developer News• Microsoft MVP for Visual Studio ALM• Team Foundation Server, TDD, Testing Best Practices,

Silverlight, Windows Azure• http://blog.benday.com• benday@benday.com

Thanks.

• To my content reviewers Jerri Chiu, Microsoft Dennis Bass, Microsoft

The Agenda

• Basics of Web Performance Tests Load Tests Load Test Rigs Performance Explorer

• Existing application Identify performance problems Go from bad performance to good ASP.NET & WCF

Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.

Here we go.

The Purpose of Load Testing

• Identify the capabilities of your application• Possible goals

Explore Verify Find the limits Crush

Best Practice:The Obligatory “Eat Your Vegetables”

• Load test throughout the development cycle.

• Catch problems early• Establish performance baseline• Watch for performance trends

• Cheaper to fix early

• Plan for performance

Cheaper To Fix Early

Early Less Early Nearly Done Production0

10

20

30

40

50

60

Time

Co

st t

o f

ix

Dose Of Reality

• You know you should work that way.• You probably don’t.

• This talk is for you.

• You have problems.• How do you figure out what’s going wrong?

Inspiration for this talk

• Real customer engagement• Spent years developing a web application• Released it • Seriously grumpy customers• Huge hardware• ~20 simultaneous users• Fixed it

4000+ simultaneous users Reduced hardware

The Tools

Visual Studio

Ultimate

• Web Performance Tests• Load Tests• Load Test Rigs

Visual Studio

Premium

• Profiling Tools• Performance Wizard

TOUR OF THE APPLICATIONdemo

Disaster Strikes.

• The application is deployed to production.• Users are losing their minds.

Intermittent errors on the site. Site is slow.

• Your boss is seriously angry. “Fix it!”

The Problem.

• “No Repro” “Works on my box.” You don’t know what the errors are. The site seems to work fine.

• Wild guess: it dies under load.

• You need to create some load.

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests

3. Simulate lots of users with a Load Test Rig

4. Reproduce the errors...hopefully

5. Find code problems with the Performance Wizard

6. Fix the code

Web Performance Tests.

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests

3. Simulate lots of users with a Load Test Rig

4. Reproduce the errors...hopefully

5. Find code problems with the Performance Wizard

6. Fix the code

What is a Web Performance Test?

• Test type in Visual Studio 2010 Ultimate• Simulate a user using an ASP.NET application

HTTP traffic

• Building block of Load Tests

Tip: Web Tests & User Stories

• Web Tests should simulate a User Story• Example:

As an administrator, I need to search for a person by username so that I can deactivate that person’s account.

As a customer, I want to order a pizza and pay with a credit card.

• Helps to organize your suite

CREATE A BASIC WEB TESTdemo

Data-driven Web Tests

• Run web test once per row in data source• Simulate multiple users doing *similar* actions• Data source types

OLE DB, CSV, XML

• Parameterization

• Better tests, better coverage

WEB TESTS IN THE UNDERPERFORMING APPLICATION

demo

Load Tests.

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests

3. Simulate lots of users with a Load Test Rig

4. Reproduce the errors...hopefully

5. Find code problems with the Performance Wizard

6. Fix the code

What are Load Tests?

• Mix of Web Performance and Unit Tests

• Simulates a mix of users doing a mix of things

• Simulate Network speeds Different browsers Varying user activity loads

Why would you put a Unit Test in a Load Test?• Helpful for testing WCF• Cumbersome to simulate WCF via Web Test

(You’d have to re-invent the wheel.)

• Unit tests can re-use the WCF libraries (Done and done.)

• Important for Silverlight and SOA• http://wcfloadtest.codeplex.com/• http://sqlloadtest.codeplex.com/

HEY KIDS, LET’S CREATE SOME LOAD TESTS!

demo

(…and then you all say, “Hooray!”)

Running Load Tests.

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests

3. Simulate lots of users with a Load Test Rig

4. Reproduce the errors...hopefully

5. Find code problems with the Performance Wizard

6. Fix the code

Two ways to run load tests.

• Generate load from Visual Studio 2010 Uses only 1 core/processor Limited to 250 simulated users

• Generate load from a Test Rig Far better test Simulate lots more users

Load Test Agent Licensing• Microsoft Visual Studio Team System 2010

Test Load Virtual User Pack• Simulate up to 1000 virtual users• Used to cost $$$

Visual Studio 2010 Load Test Feature Pack• Used to cost ~$3500.00 per 1000 simulated users

• Free starting March 8, 2011• Visual Studio 2010 Ultimate with MSDN• Simulate unlimited amount of load

(Limited only by your hardware.)

• http://www.microsoft.com/visualstudio/en-us/products/2010-editions/load-test-virtual-user-pack

Load Test Rig & The Application

• Controller Orchestrates Licensing Perf Mon

Counters

• Agent (1..n) Simulates

users

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests

3. Simulate lots of users with a Load Test Rig

4. Reproduce the errors...hopefully

5. Find code problems with the Performance Wizard

6. Fix the code

RUN LOAD TESTS ON THE RIGdemo

The Core Problem.

‘sp_who2’ says lots of open connections.

Performance Profiling

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests

3. Simulate lots of users with a Load Test Rig

4. Reproduce the errors...hopefully

5. Find code problems with the Performance Wizard

6. Fix the code

Profiling Tools

• Load Tests help you find issues in a system “Surface” issues

• Profiling helps you find issues in the code

• Unit Tests, Web Tests, Load Tests Repeatable Good for targeted tuning

• System under “natural” load Not repeatable Looking for rough patterns

Profiling Methods

• Gathers information at intervals (clock cycles)• No code modifications• Use this for first pass explorations

Sampling

• Modifies your code• Gathers detailed timing and count informationInstrumentation

• Multi-threaded code profilingConcurrency

• Object creation• Garbage collection.NET Memory

• Interaction between your application and SQL Server via ADO.NETTier Interaction

Load Test Performance Sessions Via Wizard

• Must be only Web Performance Tests• Can’t profile via Load Test Rig• ASP.NET must be running in x86 mode

LET’S GO PROFILE SOME STUFF.

demo

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests

3. Simulate lots of users with a Load Test Rig

4. Reproduce the errors...hopefully

5. Find code problems with the Performance Wizard

6. Fix the code

Any last questions?

For More Information: Articles

http://tinyurl.com/3xjqgvz

http://tinyurl.com/2ulvvvr

For More Information: Microsoft Links

• Visual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/

• Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52

For More Information: Microsoft Bloggers

• VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/

• Ed Glas - http://blogs.msdn.com/edglas/• Bill Barnett - http://blogs.msdn.com/billbar/• Sean Lumley - http://blogs.msdn.com/slumley/• Dennis Stone - http://blogs.msdn.com/densto/• Mike Taute - http://blogs.msdn.com/mtaute/• Neelesh Kamkolkar -

http://blogs.msdn.com/b/nkamkolkar/

Thank you.

blog.benday.com | www.benday.com | benday@benday.com

Additional information that I practically never have time to cover.

Miscellaneous Tips.

Include Load Tests In Your Nightly Build

• Create a Test List (*.vsmdi)• Include the Test List in the build• Captures trend data at 24 hours intervals• Compare the load test data with Excel

Static Code Analysis / FxCop

• Help you to find issues in your code• Show you IDispose errors

• Rules are configurable• Use your judgment about which are relevant for your team

Avoid Static Methods & CA1822

• Static Code Analysis Rule CA1822 is evil.

• Sacrifices maintainability for perfomance.

• Blog post: “Static Methods Are A Code Smell”

• http://bit.ly/cnEsuV

Only optimize known performance problems.• You’d be surprised by what *ISN’T* a performance problem• Don’t spend a ton of time coding fancy solutions to

performance problems you *THINK* you might have• Find and fix *REAL* problems

Tip: WCF Message Size

• Data structure formatting can make a huge difference• Example: Banks and the States they operate in• Option 1: Dictionary<string, string>

Key = Bank Id Value = State Abbreviation

• Option 2: Dictionary<string, string[]> Key = State Abbreviation Value = Array of Bank Id’s

• Option 2 was approximately 90% smaller

Tip: Add Performance Counters

• Create Performance Counters (perfmon) in your app• Do this early in the development cycle

• More directed profiling of your app during Load Tests

• Good for managing your app when it’s in production (Operations people love this.)

Using Performance Sessions on x64

• You can’t profile x64 IIS processes from the IDE

• Option #1: Profile from the command line Start ASP.NET profiling using VSPerfAspNetCmd.exe Run the web test

• Option #2: Run the ASP.NET process in x86 mode

How To Run ASP.NET in x86 Mode

• Set “Enable 32-bit Applications” to true on your application’s AppPool