Module 05Test Infrastructure

Topics

The setup and maintenance infrastructure phase aims to provide the required test infrastructure and resources. A distinction is made between test environments, test tools and workplaces. Setting up and maintaining the infrastructure requires very specific expertise. Testers generally have limited knowledge in this area, but are dependent on it. No test can be executed without an infrastructure.

http://www.tmap.net/en/tmap/4-essentials/structured-test-process/acceptance-and-system/infrastructure-phase

The Infrastructure Phase

Test Types

Performance &

Load Tests

Security Tests

‘ility’ Tests

Automated

and ManualManual

AutomatedTools

Automated

Build & Deploy Infrastructure

Functional Test Environment

Load Test Rig

Acceptance Test Environment

Possible Test Environments

Continuous Acceptance Testing

Continuous integration

automatically creates a

build after check-in

Create multiple build definitions

to support various build scenarios

Use build check-in policies to

ensure broken code is fixed

before being checked in

Use either continuous integration builds

or rolling builds with build check-in

policies to ensure quality builds

Configure the gated check-in

build to fail if a test fails, ensuring

the code won’t be checked in

Identify specific test to be run

during the gated check-in build

Configuringfor Unit Testing

Unit Tests, Component Tests

Build Controller andBuild Agent Infrastructure

Is used often and needsto be flexible and fast

Development Test Infrastructure

Use it to retrieve the necessary source code for a build, compile the solution and (if required) execute unit tests and static code analysis tools as part of the build process.

Build Controllers1-1 with Team Project Collection OR 1-* with build agentsPerform lightweight tasks.Can easily be virtualized.

Build Agents1-1 with build controller.Performs heavyweight build tasks.Manage files in source control.Run testsCompile code.Allow load-balanced and/or parallel builds.Can be tagged to spread workload.

Build Infrastructure

Lists builds and provides information about test results, test coverage, code churn and quality notes for each build.

Build Summary Report

Demo

Configuring the BuildInfrastructure for Unit Testing

Configuringfor Functional Testing

“Working Systems”

Manual and automatedtest execution

Planning and Managementof test cases and test plans

Functional Test Infrastructure

Virtual Environments

Lab Management

Working Systems

Use it to retrieve necessary source code for a build, compile the solution and (if required) execute unit tests and static code analysis tools as part of the build process.

Test Controllers1-1 with Team Project Collection | 1-* with test agents.Perform lightweight tasks.Can easily be virtualized.

Test Agents1-1 with test controller.Performs heavyweight test tasks.Run tests Interactive.Collect Data Diagnostic Information.

Build Infrastructure

Configuringfor Acceptance Testing

Real Life Scenarios

“Working systems”

Not always connected tothe rest of the team

Acceptance Tests

Feedback Manager

Exploratory Testing

Alpha/Beta Testing

Lab Management

Testing Working Systems

Test Virtualization

Execute tests in Virtual MachinesLink 1-* VMs with Test Agents to execute tests in different environments.

Low Effort – easy to set upInstall Test Agent on VM.Sync to TFS.Set Up Build Configuration to deploy tests to VM automatically.

Virtualized Test Environments

Execute tests in Virtual Test EnvironmentsA virtual environment is a collection of virtual machines (VMs). Each Virtual Machine in a virtual environment represents a role required for the application that is to be tested, developed or run. Lab Management can be used to start all the virtual machines in a virtual environment to run an application, or test an application.

Automated environment set upLab Management uses System Center Virtual Machine Manager (SCVMM) toallow access to virtual machines or templates in a library as “golden masters”.These golden masters are created by using either Hyper-V or SCVMM.SCVMM is used to deploy the virtual machines and templates tothe environments on the specified host group.

Lab Management

Lab Management Automation

On-premise | Cloud

Create a new environment

using virtual machine

templates in SCVMM

Create an environment

using machines running

on SCVMM

Create new

environments on existing

machines

Quickly choose multiple

test suites to run after

deployment

Choose your environment, and

build, deploy and test in that

environment from a single dialog

Snapshot environments

during manual testing to

increase reproducibility

Spin up virtual test environments from

within Test Manager, or attach a virtual

environment to a manual testing session

Test run results are attached to

specific builds and environments

Collect all the data required

to recreate bugs from virtual

environments

Demo

End-to-End Test WorkflowEnvironment Virtualization

Configuringfor Load Testing

Load Test Clients createvariable load on the target

Load Tests produce lotsof data and information!

Load testing expertise isrequired to interpret results

Load Test Infrastructure

Visual Studio Ultimate can use a computer or a group of computers to generate simulated load for testing, and to run tests remotely and concurrently on several computers. This group of computers consists of a single test controller and one or more test agents. You can use the test controller and test agent to generate more load than a single computer can generate alone or just test remotely.

Load Test Agents

The Cloud can be used for Load Testing – unlimited Load can be generated this way without associated hardware investments.

Agents on Windows Azure

Demo

Load Testing usingWindows Azure