Alm 4 Azure
-
Upload
clemens-reijnen -
Category
Technology
-
view
3.259 -
download
2
description
Transcript of Alm 4 Azure
![Page 1: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/1.jpg)
A u t o m a t e d A L M f o r A z u r e C l o u d D e v e l o p m e n t ALM 4 Azure
1
![Page 2: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/2.jpg)
• ALM • Azure • ALM 4 Azure
AGENDA
2
![Page 3: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/3.jpg)
• Sogeti | Management Consultant
www.ClemensReijnen.nl
@ClemensReijnen [email protected]
Clemens Reijnen
3
![Page 4: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/4.jpg)
application lifecycle management
http://www.clemensreijnen.nl/post/2008/02/18/ALM-Definitions.aspx
4
![Page 5: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/5.jpg)
5
ALM - Agile | Lean
Application Lifecycle Management Collaborate over Processes and Products with tools.
Agile, Lean and also TMap, DyA, ITIL Methodologies, principles customized, adopted and merged from all ALM roles.
optimized environments for supporting the ALM team goals
![Page 6: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/6.jpg)
work together __with work packages __on alm artifacts
6
![Page 7: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/7.jpg)
http://msdn.microsoft.com/en-us/library/bb385832.aspx
7
![Page 8: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/8.jpg)
identical – independent environments
Guest OS Versioning
8
![Page 9: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/9.jpg)
Config *.cscfg
Development & Deployment
SDK
*.cspkg
9
![Page 10: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/10.jpg)
Start
10
![Page 11: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/11.jpg)
ALM 4 Azure Goals: Collaborative
High Quality Flexible Automated Repeatable Efficient C
lou
d S
yste
ms
11
![Page 12: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/12.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
12
![Page 13: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/13.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
13
![Page 14: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/14.jpg)
Scenario 1: Developer only
Staging Production
Developers implementeren het systeem Local unittesting
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
1
2
3
14
![Page 15: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/15.jpg)
Stages of Service Deployment
Taken from:
![Page 16: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/16.jpg)
Deploying a Cloud Service from Visual Studio
DEMO
16
![Page 17: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/17.jpg)
Scenario 1: Developer only
Pro: Easy installation and configuration
Single click deployment from VS2010
Con: No collaboration Easy deployment errors (configuration) What about test and ops
17
![Page 18: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/18.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
18
![Page 19: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/19.jpg)
Scenario 2: Developer with manual tester
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases “Dry” run van tests tegen Development Fabric
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
1
2
3
4
19
![Page 20: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/20.jpg)
![Page 21: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/21.jpg)
Microsoft Test Manager
21
![Page 22: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/22.jpg)
Staging Production
TEAM
Source Repository
(TFS)
1
2
3
4
Functional, system, manual tests executed in compute emulator
with MTM.
Build Verification Tests (unit tests) and other quality gates,
executed in the build.
Platform accepatance tests executed on Azure stating with
MTM.
1 2 4
![Page 23: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/23.jpg)
4
2
3
Get the CSX folder and CSFG file 1
1
![Page 24: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/24.jpg)
Test a Cloud Service with Microsoft Test Manager
DEMO
24
![Page 25: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/25.jpg)
Diagnostic adapters
![Page 26: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/26.jpg)
Azure Logs Diagnostic Data Adapter
DEMO
26
![Page 27: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/27.jpg)
Capabilities • Link test cases to requirements • Fast Forward • Test data Iterations • Test case management • Test configurations
Interesting capabilities 4 Azure • Shared steps ( 4 VISP Swap ) • Diagnostic adapters ( Rich bug reports )
Not working capabilities in Azure • Test Impact 4 manual tests • Intelli Trace 4 manual tests (although possible custom
adapter) 27
![Page 28: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/28.jpg)
Scenario 2: Developer with manual tester
Pro: Easy installation and configuration
Single click deployment from VS2010 Testers connected, same heartbeat as dev
Proven quality
Con: Easy deployment errors (configuration) Time consuming (deploy and test) Not repeatable (annoyed testers) Testers connected
28
![Page 29: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/29.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
29
![Page 30: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/30.jpg)
Scenario 3: Developer with manual tester and deployment build
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases Dry run van tests tegen Development Fabric
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)
1
2
3
4
30
![Page 31: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/31.jpg)
Management API CSPack CSRun CSManage PowerShell CmdLets Build Targets
Automatic Deployment
http://scottdensmore.typepad.com/blog/2010/03/azure-deployment-for-your-build-server.html http://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx
31
![Page 32: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/32.jpg)
1. Change .ccproj project • New build target which uses the built-in CorePublish target
2. Create PowerShell script • Called by build target and uses Azure CMDLets
3. Edit Build Config • Add the necessary parameters
4. Do something with certificates • Put the Azure certificate on the build server
Deploy during build
32
![Page 33: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/33.jpg)
Deploy a Cloud Solution within the Build
DEMO
33
![Page 34: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/34.jpg)
Pro: Easy installation and configuration
No click deployment from build Repeatable ‘proven’ deployments*
Testers connected, same heartbeat as dev Proven quality
*demo needs some additional environment tweaks for different csfg files
per environment
Con: Time consuming testing Application can contain ‘annoying’ bugs Build workflow knowledge necessary Powershell, ccproj tweaks, target files, certificates
Scenario 3: Developer with manual tester and deployment build
34
![Page 35: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/35.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
35
![Page 36: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/36.jpg)
Scenario 4: Developer with automated regresion tests, manual tests and deployment build
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases Dry run van tests tegen Development Fabric Automation van testcases
(moeten gedraaide testcases zijn voor CodedUI) UI automation tests run tegen Development Fabric
en associëren met test plan
TEAM
Source Repository
(TFS) C# (implementatie + unittests) Test cases (automated UI tests)
‘SprintReview’Build Compile Run Unit Tests
Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)
1
2
3
4
5
36
![Page 37: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/37.jpg)
CodedUI
![Page 38: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/38.jpg)
Where? When? How? Who? Why? What?
![Page 39: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/39.jpg)
Where? When? How? Who? Why? What?
![Page 40: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/40.jpg)
40
Different URL’s
1
2
![Page 41: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/41.jpg)
41
UIMap Editor
![Page 42: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/42.jpg)
Create automatic Tests for the Cloud Application with CodedUI
DEMO
42
![Page 43: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/43.jpg)
a) At the Build server?
b) In the Development environment?
c) In the Test environment?
d) In the Azure environment?
Where to run the auto test?
43
![Page 44: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/44.jpg)
Visual Studio
Test Manager
Team Foundation Server
Build Server
Test Agent
Test Agent
Test Agent
44
![Page 45: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/45.jpg)
Configurations for automated test execution:
VS 2010
MTM 2010
TFS 2010
Build Controller
Test Controller
Build Agent
Test Agent
On physical environments (no lab)
45
![Page 46: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/46.jpg)
46
Automated Test Settings
![Page 47: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/47.jpg)
A
Flavor A: Execution from VS2010…
Configurations for automated test execution:
Real developer test / test automation dry run (Triggered by: right mouse click – run tests, in test project) http://msdn.microsoft.com/en-us/library/dd286580.aspx No additional configuration needed* * Data Diagnostic Settings in the Local.Testsettings file are configurable.
Pro: Easy setup
Debug-able test automation
Con: No collection of test results in TFS Run on developer environment 47
![Page 48: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/48.jpg)
B
Flavor B: Execution during build with Build controller…
Configurations for automated test execution:
Part of Build Verification Tests (triggered by: build) Set the Build Service to run in interactive mode http://blogs.msdn.com/b/mathew_aniyan/archive/2009/05/26/coded-ui-test-in-a-team-build.aspx Configure the build to run the tests http://msdn.microsoft.com/en-us/library/ms182465.aspx#CreateBuildType * Data Diagnostic Settings in the *.Testsettings file are configurable.
Pro: Easy setup
Test results in build result
Con: No collection of test results in TFS Build Controller needs to run in interactive mode Tests are executed on build environment Run on build environment
not recommended, strange to run UI tests on build
48
![Page 49: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/49.jpg)
C
Flavor C: Execution during build with Test controller…
Configurations for automated test execution:
Part of Build Verification Tests on multiple test agents (triggered by: build) Configure Test Controller (don’t register it with a project collection ) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestControllers Configure Test Agents on clients (interactive mode) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestAgents Configure *.Testsettings file in solution to use Test Controller and Test Agents Configure the build to run the tests (see B)
Pro: Test run on test environments
Tests run on multiple environments Test Results in Build result
Con: No collection of test results in TFS Harder to configure Need for specific test client environments Test Settings from VS
Preferred configuration above flavor B
49
![Page 50: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/50.jpg)
D
Flavor D: Execution from Microsoft Test Manager…
Configurations for automated test execution:
Part of Regression Tests (triggered by: MTM user, right mouse click on test case, run) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. http://msdn.microsoft.com/en-us/library/ee390842.aspx http://msdn.microsoft.com/en-us/library/dd293551.aspx Associate CodedUI test with WI Test Case from VS. http://www.richard-banks.org/2010/11/how-to-use-codedui-tests-watin-and-mtm.html
Pro: Test run on test environments
Tests run on multiple environments Test Result in MTM and TFS
Test Settings from MTM
Con: Manually triggered by Tester Hard to configure Need for specific test client (same as MTM?)
Other type of test than BVT
50
![Page 51: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/51.jpg)
E
Flavor E: Execution from MTM during Build…
Configurations for automated test execution:
Part of BVT (triggered by: Build) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. Associate CodedUI test with WI Test Case from VS. Create Build task to run MTC or MSTEST task for Test Plan http://blogs.microsoft.co.il/blogs/shair/archive/2010/10/30/how-to-run-coded-ui-tests-from-command-line.aspx
Pro: Test run on test environments
Tests run on multiple environments Test Result in MTM and TFS
Triggered by build Test Settings from MTM
Con: Hard to configure Need for specific test client (same as MTM?)
Preferred configuration above flavor C Flavor D and E can be configured together
51
![Page 52: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/52.jpg)
Automatic test a Cloud Service with Microsoft Test and Lab Manager infrastructure
DEMO
52
![Page 53: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/53.jpg)
Pro: No click deployment from build
Repeatable ‘proven’ deployments* Testers connected, same heartbeat as dev
Proven quality Automated BVT on different Environments
Comfortable Acc Testing Done Done
*demo needs some additional environment tweaks for different csfg files
per environment
Con: Build workflow knowledge necessary Powershell, ccproj tweaks, target files, Certificates Test Infrastructure knowledge necessary A balanced thinking of test automation
Scenario 4: Developer with automated regresion tests, manual tests and deployment build
53
![Page 54: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/54.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
54
![Page 55: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/55.jpg)
Staging Production
Developers implement the system Local unit testing
Testers specify test cases Dry run tests cases against Development Fabric Automation of test cases
UI automation tests execute against Development
Fabric and associate with a test case Operations determine instrumentation data DFO, SLA Usages Metrics, payment
‘Release ’Drop Package (same as test env
package) Acc testing met MTM en
Diagnostic Data Collectors
Automated ALM for Azure Cloud development Efficient, repeatable with proven quality fast flexible delivering of useful functionality … Making optimal use of the characteristics of Azure: - Identical environments (Guest OS versioning). - Separate in depended staging production environments. - Easy up and downgrade of applications. - Unambiguous deployment mechanism - rich diagnostic capabilities - Simplified Application Deployment & Management - …
TEAM
BUILD (TFS)
Acceptation
C# (implementation + unittests) Test cases (automated UI tests)
‘SprintReview’Build Compile Run Unit Tests Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell) Run automated Test Suites (MTC.exe) with different
environments (test controller – agents infrastructure) Collect and publish test results “Build Verification Tests” green -> deployment 2 ‘test
'production
1 2
3
4
55
![Page 56: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/56.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations Scenario 6: TFS on Azure Scenario 7: Other Infrastructure on Azure
56
![Page 57: Alm 4 Azure](https://reader031.fdocuments.in/reader031/viewer/2022020115/55526d92b4c9052e1f8b512f/html5/thumbnails/57.jpg)
57