Performance Testing [not] somebody I used to know

Somebody that I used to know

Performance Testing

“- Gotye

Tuesday, 19 February 13

10,000 hour rule

“The key to success in any field is, to a large extent, a matter of practicing a specific task for a total of around 10,000 hours.


about 34 projects multiplied by 3 months each by 20 days per month by 8 hours per day at 20% efficiency

equals ...

My ‘time on the mainframe’


Knowledge vs. Time

3,264 hours

K

How much you think you know

How much you actually know

t


20% Efficiency?


Mostly Effective

Most performance test efforts are effective ...

“Testing was completed”“NFRs were met”“We survived go-live”


Rarely Efficient

Performance test efforts are rarely efficient ...


Lossiness

(lossy) characterized by or caused by dissipation of energy.


Tools

1,000 VU license = enough to feed a village

“We waste a LOT of energy on tools and people to execute the performance test effort.


Why so Much?

http://1.bp.blogspot.com/_T3Gqqkz9Mm0/TFEoMfpM6-I/AAAAAAAAB0o/8RslwtCum-4/s1600/tail.jpg

Tools are driving the process!We are suffering from market distortion ...




Perfect Competition


“Describes markets such that no participants are large enough to have the market power to set the price of a homogeneous product.




http://en.wikipedia.org/wiki/Market_power




Imperfectly Competitive

At first there were few sellers and now there are many s e l l e r s w i t h h i g h l y differentiated products ...


At first there were few


And now there are many


Selenium vs. QTP


JMeter vs. LoadRunner


Cloud Impact?

Force MultiplierTuesday, 19 February 13

Biggest Drivers for Tools

A r e c o s t s , o p e n a n d extendable languages, integration with popular t e s t f r a m e w o r k s a n d continuous delivery and or deployment, access to high level APIs, cross platform and ease of use.


How to select the right tool?

Effective & Efficient Almost as important as the job itself.


Think about the price

and cost effectiveness


Think about the support


Think about the integration

Everything from source c o n t r o l , m o n i t o r i n g , communications, alerting, deployment and beyond.


Think about the protocol

HTTP 1999XHR 2008WebSocket 2011

Think about the protocol


Environments

environments {start|stop|restart}

We spend a LOT of time waiting for test environments to be provisioned, patched, restarted or deployed.


Increased Burn Rate

The rate at which hours (allocated to a project) are being used ...


Increased Complexity

Solving for simultaneous equations


Analysis Paralysis

“Are we writing scripts to launch a space shuttle?

http://www.flickr.com/photos/gsfc/6385412737/Tuesday, 19 February 13

http://www.flickr.com/photos/gsfc/6385412737/

http://www.flickr.com/photos/gsfc/6385412737/

Extinct by Instinct


Trending Now

“Let’s be *more* agile and promote an iterative cycle that emphasizes working products over product specifications ...


In the Future


Future is Now


Fire Fighting in Production


Replicating in Stage


A DSL for Test Plans


Scaled out in 5 minutes


Results Shared in Real Time


Short & Sharp Tests


In 2013

“ W e h a v e s i g n i f i c a n t l y reduced performance test costs and increased efficiency of performance test efforts.


Questions?

Tim KoopmansMelbourne, [email protected]@90kts

contact:locate:email:

blog:follow:

GridinitTuesday, 19 February 13

mailto:[email protected]

mailto:[email protected]

Performance Testing [not] somebody I used to know

Documents

Transcript of Performance Testing [not] somebody I used to know