ECOMMERCE V2

STACK

LOAD TESTTesting Performance through response time

and throughput

Stephen Rylander

Session

• Basis for Load Test

• What is Performance?

• Page Views

• Physical Tiers

• Response Times

• IIS Counters

• Preliminary Findings

BASIS FOR LOAD TEST

Purpose

•Generate baseline of site by….

• increasing load on the infrastructure and

application in a…

• Controlled and predictable manner

Specifics

• The ramp up…

• 4 steps

• 250 ::: 6:14 min

• 500 ::: 8:07 min

• 1000 ::: 8:53 min

• 2000 ::: 11:45 min

Location

• AlertSite’s cloud based (Amazon Web Services) load test

infrastructure

Steps

START Home

TLP (team landing)

Dlp (department)

Dlp page 2

Post filter / get filtered dlp

T-shirt (product)

Dlp (t-shirt)

Dlp (bedding)

Pdp bedding set

Add to cart / get View cart

Continue to checkout/get sign in page

Post checkout start/ get address info (guest)

Post address / view payment info

Post payment / view confirmation END

WHAT IS PERFORMANCE?

2 ways to measure…

1 - Throughput

• Throughput

• Requests per second

• Transactions per minute

• Executions per click

2 - Response

• Response Time

• Duration of a task

• Seconds per click

• Page load

• DNS lookup

• Length of time between <click> and seeing page

Together

• The system should handle 3M page views an hour with

each page returning in less than 2.5 seconds.

Thinking about Measurements

Averages

• Casual statistic

• Not bad for short periods

• Heavy variance masked

• Can hurt customer

Percentiles

• Different way of viewing

• Like a standardized test

• “Home page should load

in under 2 seconds for

95% of executions.”

Our customers

feel the

variance, not

the mean.

GE – Six Sigma

Shoppers vs. Page Views

• 11/14

• 6PM

• 166,397 page views = 30K visitors/hour (sitetracker)

• 7 PM

• 172,716 page views = 31K visitors/hour (sitetracker)

Shoppers/Visitors is only a business concept

applied to historical averaged user behavior.

It is a predictive tool not a capacity planning

tool.

PAGE VIEWSThe numbers via throughput

2000 1000 500 250

per/min 2,560.96 2,267.53 2966.6 1930

per/sec 42.68 37.79 49.443333 32.1667

per/hour 153,657.39 136,051.76 177,996.00 115,800.00

Reported

2000 1000 500 250

per/min 4,217.48 3,779.29 4948.6 3216.6667

per/sec 70.29 62.99 82.47666667 53.611111

per/hour 253,048.70 226,757.65 296,916.00 193,000.00

Real Feel

This year high: NK /hour

RESPONSE TIME

0

5

10

15

20

25

30

35

40

45

50

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

2000 users 7.5 12.2 17.8 20.9 46.2 24.13 23.29 22.3 22.9 47.8 0 11.1 19.4 30.4 14.7

1000 users 4.2 7 9.2 10.1 21.5 9.7 8.5 7.9 7.5 12.8 0 4.6 10.5 10.1 13.8

500 users 3.1 3.8 4.8 5.3 9 4.1 4 3.4 3.2 5.6 0 3.99 7.1 7.5 10.3

250 Users 0.9 1 1.4 1.8 5.7 1.3 1.49 1.2 1.1 1.66 0 0.922 1.02 2.3 8.09

Axis

Tit

le

Average Base Page Response Times

Wednesday Nov. 11 ::: 123K page views/hour

Queuing?

Queuing?

Trending over position in test 1

Trending over position in test 2

Trending over position in test 3

WEB COUNTERS

Req/sec

Reqs

Queued

Reqs/sec: avg. 49; max 117

Queued: avg. 273; max 1138

Reqs current

Reqs queued

Reqs executing

SERVERS

WebTrans02

WEB02

TAKE AWAYS

What we saw

• ~20% load increase over our busiest hour this year

• At peak ~10K orders/hour

• Linear scaling; linear slow down

• Physical server capacity ok

Concerns

• IIS queuing because of…

• Down stream services

• Payment, inventory, etc

• Connection limits

• SQL connection limits per/pool

• Web server worker thread starvation

What we’ve been doing

• Adding more web tier capacity

• Splitting more app pools

• Bumping up worker processes in external web services

Next steps

• Verifying new web tier and services capacity

• Analyzing daily performance monitors

• Investigate downstream connections

ENDThanks.