Software Testing with Visual Studio 2013 & Team Foundation Server 2013
description
Transcript of Software Testing with Visual Studio 2013 & Team Foundation Server 2013
![Page 1: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/1.jpg)
Software Testing with Visual Studio 2013 &
Team Foundation Server 2013Benjamin Day
![Page 2: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/2.jpg)
Benjamin Day• Brookline, MA• Consultant, Coach, & Trainer• Microsoft MVP for Visual Studio ALM• Team Foundation Server, Software Testing,
Scrum, Software Architecture• Scrum.org Classes
– Professional Scrum Developer (PSD)– Professional Scrum Foundations (PSF)
• www.benday.com, [email protected], @benday
![Page 3: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/3.jpg)
Online courses at Pluralsight.com
![Page 4: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/4.jpg)
![Page 5: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/5.jpg)
On with the show.
![Page 6: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/6.jpg)
Overview ofVisual Studio ALM.
![Page 7: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/7.jpg)
A comprehensive ALM offeringTeam
Foundation Server
Planning SCM Work Item Tracking Continuous Deployment Build Automation Feedback
Management
![Page 8: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/8.jpg)
Three types of testing in the VS ALM World.• Developer
– “Does the code work?”
• Manual Testing / Quality Assurance (QA)– “Does the app work?”
• Load Testing & Performance Testing– “Does the app work under load?”
![Page 9: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/9.jpg)
Test types & features.• Developer
– Unit Tests
• Manual / QA– Coded UI tests
• Load & Performance– Web Performance Tests (WPTs)
Recorded or Coded– Load Tests
![Page 10: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/10.jpg)
There are only so many minutes in the day.
![Page 11: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/11.jpg)
The real question is ‘what to test?’
![Page 12: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/12.jpg)
Testing ROI.
![Page 13: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/13.jpg)
Testing ROI for Developers.• Write unit tests.
– Test First. Test-Driven. Test-Eventually. (Whatever.)– Write something and try to be honest.– Watch your code coverage.
• Why?– Helps you refactor.– Helps you modify feature functionality.– Helps you to know if it’s working a lot faster.– Tends to push you into better/cleaner architecture.
![Page 14: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/14.jpg)
$1m for IT to support an application.
Q1 2010
Q2 2010
Q3 2010
Q4 2010
Q1 2011
Q2 2011
Q3 2011
Q4 2011
Q1 2012
Q2 2012
Q3 2012
Q4 2012
Q1 2013
Q2 2013
Q3 2013
Q4 2013
0%10%20%30%40%50%60%70%80%90%
100%
New Feature Budget vs. Maintenance Budget
Maintenance New Features
That’s looking grim.
![Page 15: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/15.jpg)
What’s that feel like?• Costs more to run the app than you actually have.
• App breaks if you breathe funny.
• Can’t add new features.
• Changes take *FOR-EVER*.
• Competitors are catching up.
• You’re hosed. It’s hopeless.
![Page 16: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/16.jpg)
Why’s it getting like that?• Technical Debt.
• Excess “inventory”.
• Too much architecture.
• Brittle code.
• Too hard to refactor.
![Page 17: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/17.jpg)
Time to re-write the app.
![Page 18: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/18.jpg)
Use tests to keep it clean.
![Page 19: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/19.jpg)
Or start adding tests to change the ROI trajectory.
![Page 20: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/20.jpg)
$1m for IT to support an application.
Q1 2010
Q2 2010
Q3 2010
Q4 2010
Q1 2011
Q2 2011
Q3 2011
Q4 2011
Q1 2012
Q2 2012
Q3 2012
Q4 2012
Q1 2013
Q2 2013
Q3 2013
Q4 2013
0%10%20%30%40%50%60%70%80%90%
100%
New Feature Budget vs. Maintenance Budget
Maintenance New Features
![Page 21: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/21.jpg)
Solid developer testing pays huge dividends.
![Page 22: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/22.jpg)
Better long-term ROI.
![Page 23: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/23.jpg)
Faster time to market on new features.
![Page 24: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/24.jpg)
More responsive to market withchanges to existing features.
![Page 25: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/25.jpg)
Catch problems early in dev process.
![Page 26: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/26.jpg)
Vastly higher quality builds for QA testing.
![Page 27: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/27.jpg)
Testing ROI for QA / Manual Testing.• Track your requirements with TFS.
• Track test plans, progress, and do defect trackingwith TFS and/or Microsoft Test Manager (MTM).
• Use MTM Action Recordings to minimize tedium.
• Look for opportunities for test automationwith MTM and Coded UI Tests.
![Page 28: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/28.jpg)
Rule of thumb:At least one Test Case per Scrum PBI.
![Page 29: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/29.jpg)
Coded UI Tests.
![Page 30: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/30.jpg)
Coded UI helps you test a running application’s user interface.
![Page 31: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/31.jpg)
Unit Tests vs. Coded UI TestsUnit Tests• Classes and methods
at the API level• If it tests a UI, it’s
testing an abstraction.– (not quite testing the UI)– UI testing has been hard
• Test stuff as you build it
Coded UI Tests• UI testing• Test a running
application• Simulates a user’s
keyboard and mouse activities
• Test stuff that’s pretty much done
• Integration testing
![Page 32: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/32.jpg)
Us
e
r
In
t
erf
a
ce
(
AS
P
.N
E
T,
X
A
ML
,
Wi
n
Fo
r
m
s,
e
tc
.)
P
re
s
e
nt
a
tio
n
(
M
V
C /
M
VV
M
)
Do
m
ai
n
M
o
d
el /
S
er
v
ic
e
La
y
er
Repository / Data Access
The Relational Database
(Tables, FKs, Views, Stored Proc's)
The layers in your app.
![Page 33: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/33.jpg)
U
se
r
I
nt
e
rfa
c
e
(
A
SP
.
NE
T
, X
A
M
L,
W
in
F
or
m
s
, e
t
c.)
P
r
es
e
n
ta
ti
on
(M
V
C
/ M
V
VM
)
D
om
a
in
Mo
d
e
l / S
e
rv
i
ce
L
ay
e
r
Repository / Data Access
The Relational Database
(Tables, FKs, Views, Stored Proc's)
The layers in your app.
Coded UI tests
Unit tests
![Page 34: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/34.jpg)
Unit tests test your APIs.
Coded UI Tests test your running user interfaces.
![Page 35: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/35.jpg)
I tend to think of QA identifying a case that needs automation and then requesting a
Coded UI Test from developers.
![Page 36: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/36.jpg)
Action Recordings vs. Coded UI Tests• Action Recordings
– Exist in Microsoft Test Manager– It’s there to help QA automate away tedious clicks and typing.– Make QA testing go faster.– The idea: created by non-technical users.
• Coded UI Tests– Action Recordings on steroids.– It really helps to be a programmer.– You can do “asserts”.
(aka. you can do actual checks)– You can still associate them to MTM Test Cases as
“Associated Automations.”
![Page 37: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/37.jpg)
Structure of a Coded UI Test
• The Test Fixture Class– [CodedUI] attribute– Editable like any other
class– (You can even make it
data-driven!)
• UIMap.uitest– Auto-generated XML-
based “map” of your UI– Not editable
• UIMap.designer.cs– Supporting information for
the test– Auto-generated by the
recorder– Partial class
• UIMap.cs– Partial class– Customizations and
extensions to the stuff in UIMap.designer.cs
![Page 38: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/38.jpg)
Avoiding hard-coded paths in Coded UIs.• BrowserWindow class
• ApplicationUnderTest window
![Page 39: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/39.jpg)
Load & Performance Testing.
![Page 40: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/40.jpg)
You have an application.
![Page 41: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/41.jpg)
The Goal: Performance Test The Application
![Page 42: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/42.jpg)
The purpose of performance tests?• Identify the capabilities of your application
• Possible goals– Explore– Verify– Find the limits– Crush
![Page 43: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/43.jpg)
Performance testing in a nutshell.1. Throw traffic at the app.
2. Does it perform as expected?
3. Recreate any performance problems.
4. Fix the performance problems.
5. Repeat.
![Page 44: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/44.jpg)
What can you load test with Visual Studio 2012 Ultimate?• ASP.NET with an HTML UI
– Web Forms or MVC– (This is the sweet spot.)
• SharePoint Applications
• HTTP-based applications– ASP.NET-based SOA apps– Service-based apps with WebAPI or REST
• Coded UI Tests
• Pretty much anything you can call from a Visual Studio Unit Test
![Page 45: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/45.jpg)
Demos.
![Page 46: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/46.jpg)
Demos• Unit Tests• Code Coverage• QA Testing
– Project Management– Sprint Planning– Test Case Management– Test using MTM– Test using web– Test using web + test
runner– Test environment manager
• Web & Load Tests– WPT Basics– Load Test Basics– Custom PerfMon Counters– Load in the Cloud– Load using Test Rigs
![Page 47: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/47.jpg)
Any last questions?
![Page 48: Software Testing with Visual Studio 2013 & Team Foundation Server 2013](https://reader035.fdocuments.in/reader035/viewer/2022062501/568163a7550346895dd4b56b/html5/thumbnails/48.jpg)
[email protected] | www.benday.com