VSTS 2010 for Testers

Agenda

• Introduction - Visual Studio 2010 for test• Details - Test Types• Details - Load Test• Details - Web Test• Details - Test Manager

Microsoft Strategy

• Create one-stop solution for product development and test life cycles.

• Design test tools for Test plan, test management, defects tracking and analyzing with just one tool - Visual Studio.

• Capture the lucrative market of test tools which is dominated by IBM and HP.

Visual Studio Test Editions

• Following editions of Visual studio contains “Test” related features.– VS Test Professional 2010– VS Ultimate 2010

• Both the editions contain a separate test management tool called as “Microsoft Test Manager” used for managing test plan, test cases, test labs etc.

Supported Test Types

• Unit Test• Record-Playback Test (using Coded-UI)• Load Test• Web Performance Test• Database Unit Test

Test Flow

• Create Test Plan linking to requirements.• Create Test Suites as per requirement

modules.• Create and Manage Tests.• Run Test Cases and Submit Results.• Log Bugs/Defects.• Track test deliverables – Bugs, Test case

execution report, Automation report tc.

Web Performance Testing

• To find out performance of the web page under different load conditions.

• Define baseline for accepted server loads under given conditions.

• Determine compliance with requirements.• Improve overall quality of web application

under test.

(contd.)

• Performance of web application is found by creating1. Web Test

- Series of test sequence actions to be performed on the web site

2. Load Test - Bunching of Web tests adhering to business

scenarios to carry out performance analysis at different loads.

Performance test flow

• Identify Test Environment and acceptance criteria.

• Create a web performance test for the web application under test. This test will then be used during load testing.

• Create Load test for the specified load conditions – virtual users, load type, test distribution, step increments etc.

(contd.)

• Configure Test Environments for the tests to be run.

• Implement Test Design and execute the tests.• Analyze and Report load test results.

Reprioritize and retest for verifying performance adherence of each part of web application under test.

Web Test

• Consists of series of HTTP requests which are captured while recording the test.

• Simulate Javascript at runtime by using Web test plug-ins, Web test request plug-ins, extraction rules, or coded Web tests.

• Can be used for functional, data-driven, performance testing purposes.

• Records request in the order it is been captured during test formation.

(contd.)

• Cannot capture client-server data interchange, this means Javascript, images and CSS are not recorded during test.

• However can withstand the above request while playback using the promotion of dynamic parameters.

Web Test Tasks

• Record a Web Test.• Add Extraction, Validation rules to it.• Create Request and Web Test Plug-Ins.• Create Coded Web Test• Execute Web Test.

Web test hiccups

• HTTP request errors• Extraction and validation rule errors• Test level, Request level exceptions• Dependent request expired• Page content modified• One time user data• Javascript redirects (windows.location)

Load Test

• To evaluate the performance of application under test under the specified load conditions within limits.

• Test Types included for load test– Web Performance Test– Unit Test

Load Test Tasks

• Configure Test Controller and Test agents.• Configure load test database.• Identify the load test scenario.• Specify the counter sets to capture specific

test parameters.• Configure load test run settings.• Run load tests and Analyze test results.

Load Test Considerations

• Determine acceptance criteria– Response Time– Throughput– Resource utilization– Maximum user load

• Identify Business scenarios• Create workload model

(contd.)

• Find Load distribution pattern– Constant– Step– Goal-based

• Identify Metrics– Connection type– Sample rate– Think time

Load Test Wizard

• Run load test wizard• Specify new load test scenario.• Specify initial load patterns.• Specify Test Mix model• Specify Test Mix• Specify Network Mix• Specify Browser Mix

(contd.)

• Configure test computers.• Configure Counter sets on each computer.• Specify Initial run settings.• Optional – Add additional run settings.• Modify Run settings.

Analyze Load Test Results

• Using load test editor - Add threshold rule to the load test.

• Using Load Test Analyzer - Enable threshold violation to graphs.

• Calculate response time.• Calculate deviation from SLA time.

Microsoft Test Manager (MTM)

• New product shipped with VSTS 2010.• Developed on WPF.• Used for the test management activities like

creating test plans, test suites, test configurations, test cases, test steps etc.

• Also useful in diagnosing test data impact, automation test runs, effort planning.

MTM tasks

• Create Test Plan• Create test suites• Create/add test cases.• Test effort distribution • Assign test cases to testers• Create/add machine configurations.• Assign automated tests

(contd.)

• Assign build definitions.• Run manual and automated tests.• Associates bugs to test cases• Analyze test results.• Create Exploratory testing scenarios.• Perform Exploratory testing.

MTM for manual tests

• Connect to Team Project Site• Create Test Plan• Add details to Test Plan• Add test cases• Run tests manually• Log bugs and associate them with the testcase• Keep tap on the test progress

MTM for automated tests

• Connect to Team Project Site• Create Test Plan• Add details to Test Plan• Create automated (Unit/Coded-UI) test cases

using VSTS 2010.• Add project to TFS• Generate a build.

(contd.)

• Queue this build• Associate the automated test cases with the

manual test cases already present in test plan.• Add build definition to test plan.• Run automated tests.• Analyze test results.