Software Testing with Visual Studio 2013 &

Team Foundation Server 2013Benjamin Day

Benjamin Day• Brookline, MA• Consultant, Coach, & Trainer• Microsoft MVP for Visual Studio ALM• Team Foundation Server, Software Testing,

Scrum, Software Architecture• Scrum.org Classes

– Professional Scrum Developer (PSD)– Professional Scrum Foundations (PSF)

• www.benday.com, benday@benday.com, @benday

Online courses at Pluralsight.com

On with the show.

Overview ofVisual Studio ALM.

A comprehensive ALM offeringTeam

Foundation Server

Planning SCM Work Item Tracking Continuous Deployment Build Automation Feedback

Management

Three types of testing in the VS ALM World.• Developer

– “Does the code work?”

• Manual Testing / Quality Assurance (QA)– “Does the app work?”

• Load Testing & Performance Testing– “Does the app work under load?”

Test types & features.• Developer

– Unit Tests

• Manual / QA– Coded UI tests

• Load & Performance– Web Performance Tests (WPTs)

Recorded or Coded– Load Tests

There are only so many minutes in the day.

The real question is ‘what to test?’

Testing ROI.

Testing ROI for Developers.• Write unit tests.

– Test First. Test-Driven. Test-Eventually. (Whatever.)– Write something and try to be honest.– Watch your code coverage.

• Why?– Helps you refactor.– Helps you modify feature functionality.– Helps you to know if it’s working a lot faster.– Tends to push you into better/cleaner architecture.

$1m for IT to support an application.

Q1 2010

Q2 2010

Q3 2010

Q4 2010

Q1 2011

Q2 2011

Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Q1 2013

Q2 2013

Q3 2013

Q4 2013

0%10%20%30%40%50%60%70%80%90%

100%

New Feature Budget vs. Maintenance Budget

Maintenance New Features

That’s looking grim.

What’s that feel like?• Costs more to run the app than you actually have.

• App breaks if you breathe funny.

• Can’t add new features.

• Changes take *FOR-EVER*.

• Competitors are catching up.

• You’re hosed. It’s hopeless.

Why’s it getting like that?• Technical Debt.

• Excess “inventory”.

• Too much architecture.

• Brittle code.

• Too hard to refactor.

Time to re-write the app.

Use tests to keep it clean.

Or start adding tests to change the ROI trajectory.

$1m for IT to support an application.

Q1 2010

Q2 2010

Q3 2010

Q4 2010

Q1 2011

Q2 2011

Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Q1 2013

Q2 2013

Q3 2013

Q4 2013

0%10%20%30%40%50%60%70%80%90%

100%

New Feature Budget vs. Maintenance Budget

Maintenance New Features

Solid developer testing pays huge dividends.

Better long-term ROI.

Faster time to market on new features.

More responsive to market withchanges to existing features.

Catch problems early in dev process.

Vastly higher quality builds for QA testing.

Testing ROI for QA / Manual Testing.• Track your requirements with TFS.

• Track test plans, progress, and do defect trackingwith TFS and/or Microsoft Test Manager (MTM).

• Use MTM Action Recordings to minimize tedium.

• Look for opportunities for test automationwith MTM and Coded UI Tests.

Rule of thumb:At least one Test Case per Scrum PBI.

Coded UI Tests.

Coded UI helps you test a running application’s user interface.

Unit Tests vs. Coded UI TestsUnit Tests• Classes and methods

at the API level• If it tests a UI, it’s

testing an abstraction.– (not quite testing the UI)– UI testing has been hard

• Test stuff as you build it

Coded UI Tests• UI testing• Test a running

application• Simulates a user’s

keyboard and mouse activities

• Test stuff that’s pretty much done

• Integration testing

Us

e

r

In

t

erf

a

ce

(

AS

P

.N

E

T,

X

A

ML

,

Wi

n

Fo

r

m

s,

e

tc

.)

P

re

s

e

nt

a

tio

n

(

M

V

C /

M

VV

M

)

Do

m

ai

n

M

o

d

el /

S

er

v

ic

e

La

y

er

Repository / Data Access

The Relational Database

(Tables, FKs, Views, Stored Proc's)

The layers in your app.

U

se

r

I

nt

e

rfa

c

e

(

A

SP

.

NE

T

, X

A

M

L,

W

in

F

or

m

s

, e

t

c.)

P

r

es

e

n

ta

ti

on

(M

V

C

/ M

V

VM

)

D

om

a

in

Mo

d

e

l / S

e

rv

i

ce

L

ay

e

r

Repository / Data Access

The Relational Database

(Tables, FKs, Views, Stored Proc's)

The layers in your app.

Coded UI tests

Unit tests

Unit tests test your APIs.

Coded UI Tests test your running user interfaces.

I tend to think of QA identifying a case that needs automation and then requesting a

Coded UI Test from developers.

Action Recordings vs. Coded UI Tests• Action Recordings

– Exist in Microsoft Test Manager– It’s there to help QA automate away tedious clicks and typing.– Make QA testing go faster.– The idea: created by non-technical users.

• Coded UI Tests– Action Recordings on steroids.– It really helps to be a programmer.– You can do “asserts”.

(aka. you can do actual checks)– You can still associate them to MTM Test Cases as

“Associated Automations.”

Structure of a Coded UI Test

• The Test Fixture Class– [CodedUI] attribute– Editable like any other

class– (You can even make it

data-driven!)

• UIMap.uitest– Auto-generated XML-

based “map” of your UI– Not editable

• UIMap.designer.cs– Supporting information for

the test– Auto-generated by the

recorder– Partial class

• UIMap.cs– Partial class– Customizations and

extensions to the stuff in UIMap.designer.cs

Avoiding hard-coded paths in Coded UIs.• BrowserWindow class

• ApplicationUnderTest window

Load & Performance Testing.

You have an application.

The Goal: Performance Test The Application

The purpose of performance tests?• Identify the capabilities of your application

• Possible goals– Explore– Verify– Find the limits– Crush

Performance testing in a nutshell.1. Throw traffic at the app.

2. Does it perform as expected?

3. Recreate any performance problems.

4. Fix the performance problems.

5. Repeat.

What can you load test with Visual Studio 2012 Ultimate?• ASP.NET with an HTML UI

– Web Forms or MVC– (This is the sweet spot.)

• SharePoint Applications

• HTTP-based applications– ASP.NET-based SOA apps– Service-based apps with WebAPI or REST

• Coded UI Tests

• Pretty much anything you can call from a Visual Studio Unit Test

Demos.

Demos• Unit Tests• Code Coverage• QA Testing

– Project Management– Sprint Planning– Test Case Management– Test using MTM– Test using web– Test using web + test

runner– Test environment manager

• Web & Load Tests– WPT Basics– Load Test Basics– Custom PerfMon Counters– Load in the Cloud– Load using Test Rigs

Any last questions?

Thanks.benday@benday.com | www.benday.com