Post on 13-May-2015
description
1
26 mei 2011
ALM 4 Azure T F S 2 0 1 0 f o r A z u r e C l o u d D e v e l o p m e n t
• Sogeti | Management Consultant
www.ClemensReijnen.nl
@ClemensReijnen Clemens.Reijnen@Sogeti.nl
Clemens Reijnen
2
work together __with work packages __on alm artifacts
3
http://msdn.microsoft.com/en-us/library/bb385832.aspx
4
ALM 4 Azure Goals: Collaborative
High Quality Flexible Automated Repeatable Efficient C
lou
d S
yste
ms
5
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
6
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
7
Scenario 1: Developer only
Staging Production
Developers implementeren het systeem Local unittesting
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
1
2
3
8
dry run – local storage and compute
10
test run – cloud storage and local compute
11
debug – Intellitrace deploy
12
13
debug – Get Intellitrace file
14
debug – Find exception
15
debug
16
debug – send to peer
Scenario 1: Developer only
Pro: Easy installation and configuration
Single click deployment from VS2010
Con: No collaboration Easy deployment errors (configuration) What about test and ops
17
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
18
Scenario 2: Developer with manual tester
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases “Dry” run van tests tegen Development Fabric
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
1
2
3
4
19
Microsoft Test Manager
21
Staging Production
TEAM
Source Repository
(TFS)
1
2
3
4
Functional, system, manual tests executed in compute emulator
with MTM.
Build Verification Tests (unit tests) and other quality gates,
executed in the build.
Platform accepatance tests executed on
Azure staging - tested with MTM.
1 2 4
23
Microsoft Test Manager – Specify Test Cases
24
Link Requirements to Tasks and Test Cases
25
Specify test case
26
Test with or without debugging
27
without Visual Studio
4
2
3
Get the CSX folder and CSFG file 1
1
Challenges with changing URL’s
30
Shared Steps 2 the rescue
31
1 Shared Step : open application
32
1 Shared Step : used by all test cases
33
Fast Forward
Diagnostic Data Adapters
35
Capture information during test execution
Create your own adapter.
37
For example: WAD
38
Or Intellitrace Files
Capabilities • Link test cases to requirements • Fast Forward • Test data Iterations • Test case management • Test configurations
Interesting capabilities 4 Azure • Shared steps ( 4 VIPS Swap )
– common practice: new hosted service only change the name in the DNS…
• Diagnostic adapters ( Rich bug reports ) capturing the logs
Not working capabilities in Azure • Test Impact 4 manual tests • IntelliTrace 4 manual tests (although possible custom adapter)
39
Scenario 2: Developer with manual tester
Pro: Easy installation and configuration
Single click deployment from VS2010 Testers connected, same heartbeat as dev
Proven quality
Con: Easy deployment errors (configuration) Time consuming (deploy and test) Not repeatable (annoyed testers) Testers connected
40
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
41
Scenario 3: Developer with manual tester and deployment build
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases Dry run van tests tegen Development Fabric
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)
1
2
3
4
42
Management API CSPack CSRun CSManage PowerShell CmdLets Build Targets
Automatic Deployment
http://scottdensmore.typepad.com/blog/2010/03/azure-deployment-for-your-build-server.html http://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx
43
44
Change .ccproj project
45
Use PowerShell script
CSPack used in Target file
Configure build definition
1. Change .ccproj project • New build target which uses the built-in CorePublish target
2. Create PowerShell script • Called by build target and uses Azure CMDLets
3. Edit Build Config • Add the necessary parameters
4. Do something with certificates • Put the Azure certificate on the build server
Deploy during build
47
Pro: Easy installation and configuration
No click deployment from build Repeatable ‘proven’ deployments*
Testers connected, same heartbeat as dev Proven quality
*demo needs some additional environment tweaks for different csfg files
per environment
Con: Time consuming testing Application can contain ‘annoying’ bugs Build workflow knowledge necessary Powershell, ccproj tweaks, target files, certificates
Scenario 3: Developer with manual tester and deployment build
48
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
49
Scenario 4: Developer with automated regresion tests, manual tests and deployment build
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases Dry run van tests tegen Development Fabric Automation van testcases
(moeten gedraaide testcases zijn voor CodedUI) UI automation tests run tegen Development Fabric
en associëren met test plan
TEAM
Source Repository
(TFS) C# (implementatie + unittests) Test cases (automated UI tests)
‘SprintReview’Build Compile Run Unit Tests
Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)
1
2
3
4
5
50
CodedUI
52
53
Different URL’s
1
2
54
UIMap Editor
55
Execute from MTM
Execute from VS
56
Execute from Build
Execute from Test Agent
57
Automated Test Settings
a) At the Build server? (no … only unit test)
b) In the Development environment? (only dry run for autoamtion)
c) In the Test environment? (sure, functional tests.. You need emulator at the test env)
d) In the Azure environment? Some definitely not all (money)
Where to run the auto test?
58
Pro: No click deployment from build
Repeatable ‘proven’ deployments* Testers connected, same heartbeat as dev
Proven quality Automated BVT on different Environments
Comfortable Acc Testing Done Done
*demo needs some additional environment tweaks for different csfg files
per environment
Con: Build workflow knowledge necessary Powershell, ccproj tweaks, target files, Certificates Test Infrastructure knowledge necessary A balanced thinking of test automation
Scenario 4: Developer with automated regresion tests, manual tests and deployment build
59
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
60
Staging Production
Developers implement the system Local unit testing
Testers specify test cases Dry run tests cases against Development Fabric Automation of test cases
UI automation tests execute against Development
Fabric and associate with a test case Operations determine instrumentation data DFO, SLA Usages Metrics, payment
‘Release ’Drop Package (same as test env
package) Acc testing met MTM en
Diagnostic Data Collectors
Automated ALM for Azure Cloud development Efficient, repeatable with proven quality fast flexible delivering of useful functionality … Making optimal use of the characteristics of Azure: - Identical environments (Guest OS versioning). - Separate in depended staging production environments. - Easy up and downgrade of applications. - Unambiguous deployment mechanism - rich diagnostic capabilities - Simplified Application Deployment & Management - …
TEAM
BUILD (TFS)
Acceptation
C# (implementation + unittests) Test cases (automated UI tests)
‘SprintReview’Build Compile Run Unit Tests Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell) Run automated Test Suites (MTC.exe) with different
environments (test controller – agents infrastructure) Collect and publish test results “Build Verification Tests” green -> deployment 2 ‘test
'production
1 2
3
4
61
App fabric Application Manager (June)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations Scenario 6: TFS on Azure Scenario 7: Other Infrastructure on Azure
62
63