Alm 4 Azure with screenshots

1

26 mei 2011

ALM 4 Azure T F S 2 0 1 0 f o r A z u r e C l o u d D e v e l o p m e n t

• Sogeti | Management Consultant

www.ClemensReijnen.nl

@ClemensReijnen [email protected]

Clemens Reijnen

2

work together __with work packages __on alm artifacts

3

http://msdn.microsoft.com/en-us/library/bb385832.aspx

4





ALM 4 Azure Goals: Collaborative

High Quality Flexible Automated Repeatable Efficient C

lou

d S

yste

ms

5

Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations

6


7

Scenario 1: Developer only

Staging Production

Developers implementeren het systeem Local unittesting

TEAM

Source Repository

(TFS) C# (implementatie + unittests)

‘SprintReview’Build Compile Run Unit Tests

1

2

3

8

dry run – local storage and compute

10

test run – cloud storage and local compute

11

debug – Intellitrace deploy

13

debug – Get Intellitrace file

14

debug – Find exception

15

debug

16

debug – send to peer

Scenario 1: Developer only

Pro: Easy installation and configuration

Single click deployment from VS2010

Con: No collaboration Easy deployment errors (configuration) What about test and ops

17


18

Scenario 2: Developer with manual tester

Staging Production


Testers specificeren testcases “Dry” run van tests tegen Development Fabric

TEAM

Source Repository



1

2

3

4

19

Microsoft Test Manager

21

Staging Production

TEAM

Source Repository

(TFS)

1

2

3

4

Functional, system, manual tests executed in compute emulator

with MTM.

Build Verification Tests (unit tests) and other quality gates,

executed in the build.

Platform accepatance tests executed on

Azure staging - tested with MTM.

1 2 4

23

Microsoft Test Manager – Specify Test Cases

24

Link Requirements to Tasks and Test Cases

25

Specify test case

26

Test with or without debugging

27

without Visual Studio

4

2

3

Get the CSX folder and CSFG file 1

1

Challenges with changing URL’s

30

Shared Steps 2 the rescue

31

1 Shared Step : open application

32

1 Shared Step : used by all test cases

33

Fast Forward

Diagnostic Data Adapters

35

Capture information during test execution

Create your own adapter.

37

For example: WAD

38

Or Intellitrace Files

Capabilities • Link test cases to requirements • Fast Forward • Test data Iterations • Test case management • Test configurations

Interesting capabilities 4 Azure • Shared steps ( 4 VIPS Swap )

– common practice: new hosted service only change the name in the DNS…

• Diagnostic adapters ( Rich bug reports ) capturing the logs

Not working capabilities in Azure • Test Impact 4 manual tests • IntelliTrace 4 manual tests (although possible custom adapter)

39

Scenario 2: Developer with manual tester


Single click deployment from VS2010 Testers connected, same heartbeat as dev

Proven quality

Con: Easy deployment errors (configuration) Time consuming (deploy and test) Not repeatable (annoyed testers) Testers connected

40


41

Scenario 3: Developer with manual tester and deployment build

Staging Production


Testers specificeren testcases Dry run van tests tegen Development Fabric

TEAM

Source Repository



Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)

1

2

3

4

42

Management API CSPack CSRun CSManage PowerShell CmdLets Build Targets

Automatic Deployment

http://scottdensmore.typepad.com/blog/2010/03/azure-deployment-for-your-build-server.html http://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx

43

http://scottdensmore.typepad.com/blog/2010/03/azure-deployment-for-your-build-server.html












http://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx


















44

Change .ccproj project

45

Use PowerShell script

CSPack used in Target file

Configure build definition

1. Change .ccproj project • New build target which uses the built-in CorePublish target

2. Create PowerShell script • Called by build target and uses Azure CMDLets

3. Edit Build Config • Add the necessary parameters

4. Do something with certificates • Put the Azure certificate on the build server

Deploy during build

47


No click deployment from build Repeatable ‘proven’ deployments*

Testers connected, same heartbeat as dev Proven quality

*demo needs some additional environment tweaks for different csfg files

per environment

Con: Time consuming testing Application can contain ‘annoying’ bugs Build workflow knowledge necessary Powershell, ccproj tweaks, target files, certificates

Scenario 3: Developer with manual tester and deployment build

48


49

Scenario 4: Developer with automated regresion tests, manual tests and deployment build

Staging Production


Testers specificeren testcases Dry run van tests tegen Development Fabric Automation van testcases

(moeten gedraaide testcases zijn voor CodedUI) UI automation tests run tegen Development Fabric

en associëren met test plan

TEAM

Source Repository

(TFS) C# (implementatie + unittests) Test cases (automated UI tests)


Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)

1

2

3

4

5

50

CodedUI

53

Different URL’s

1

2

54

UIMap Editor

55

Execute from MTM

Execute from VS

56

Execute from Build

Execute from Test Agent

57

Automated Test Settings

a) At the Build server? (no … only unit test)

b) In the Development environment? (only dry run for autoamtion)

c) In the Test environment? (sure, functional tests.. You need emulator at the test env)

d) In the Azure environment? Some definitely not all (money)

Where to run the auto test?

58

Pro: No click deployment from build

Repeatable ‘proven’ deployments* Testers connected, same heartbeat as dev

Proven quality Automated BVT on different Environments

Comfortable Acc Testing Done Done

*demo needs some additional environment tweaks for different csfg files

per environment

Con: Build workflow knowledge necessary Powershell, ccproj tweaks, target files, Certificates Test Infrastructure knowledge necessary A balanced thinking of test automation

Scenario 4: Developer with automated regresion tests, manual tests and deployment build

59


60

Staging Production

Developers implement the system Local unit testing

Testers specify test cases Dry run tests cases against Development Fabric Automation of test cases

UI automation tests execute against Development

Fabric and associate with a test case Operations determine instrumentation data DFO, SLA Usages Metrics, payment

‘Release ’Drop Package (same as test env

package) Acc testing met MTM en

Diagnostic Data Collectors

Automated ALM for Azure Cloud development Efficient, repeatable with proven quality fast flexible delivering of useful functionality … Making optimal use of the characteristics of Azure: - Identical environments (Guest OS versioning). - Separate in depended staging production environments. - Easy up and downgrade of applications. - Unambiguous deployment mechanism - rich diagnostic capabilities - Simplified Application Deployment & Management - …

TEAM

BUILD (TFS)

Acceptation

C# (implementation + unittests) Test cases (automated UI tests)

‘SprintReview’Build Compile Run Unit Tests Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell) Run automated Test Suites (MTC.exe) with different

environments (test controller – agents infrastructure) Collect and publish test results “Build Verification Tests” green -> deployment 2 ‘test

'production

1 2

3

4

61

App fabric Application Manager (June)

Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations Scenario 6: TFS on Azure Scenario 7: Other Infrastructure on Azure

62

Alm 4 Azure with screenshots

Technology

Transcript of Alm 4 Azure with screenshots