Continuous Integration Processes and SCM To Support Test...

Continuous Integration

Processes and SCM

To Support Test Automation

© 2013 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

SIGIST Conference, July 2013

Gal Fatal

[email protected]

054-342-3864

AT&T Israel Center of Excellence

AT&T Worldwide

• One of the largest telecom companies in the world

• Over 250,000 employees worldwide

AT&T Israel R&D Center

• Formerly Interwise, Ltd.

• Purchased by AT&T in 2007

• Employment growth of 1,000% in 5 years

Main Lines of Business

• Connect

• Unified Communications

• Consumer Mobility

• IMS (Backend Components)

© 2013 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.2

Gal Fatal

MBA

Managing Configuration Management teams since 1999

Pre-AT&T: Intel, Siemens, Perigen

AT&T (Interwise) since 1997

Experience:

• Source control services

• Installer development

• Continuous integration processes


Moving from Waterfall to Agile (scrum) at AT&T

Continuous Integration processes

Bug lifecycle in scrum

How we provide fast and flexible builds

Sanity tests and automation

How we make multi-platform testing possible

Saving time with automatic environment installations

Demo

Agenda



Waterfall Development1994-2011

Waterfall


6

Dev group QA group

http://www.google.co.il/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=-xOvA76P8Uka1M&tbnid=Eom4TrM-VQFN-M:&ved=0CAUQjRw&url=http://avnei.ravid.org.il/1200/page/levenim&ei=Ziu4UZK1K8fOsgaqqIGQCA&bvm=bv.47810305,d.Yms&psig=AFQjCNGjX-QakvWXeE-vh8d98rmBHdY2Tg&ust=1371110593149518

http://www.google.co.il/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=-xOvA76P8Uka1M&tbnid=Eom4TrM-VQFN-M:&ved=0CAUQjRw&url=http://avnei.ravid.org.il/1200/page/levenim&ei=Ziu4UZK1K8fOsgaqqIGQCA&bvm=bv.47810305,d.Yms&psig=AFQjCNGjX-QakvWXeE-vh8d98rmBHdY2Tg&ust=1371110593149518

Waterfall


7

Dev group QA group

Waterfall Lifecycle at AT&T Connect®

Main Development


Dev

Build and Integration tests

QA runs tests and opens bugs

Milestone

3 weeks

1 week

Build took 4 days• Compilation problems• Manual processes• Build machine resources• Environments

Bugs and StabilityVersion Release

1 2 3 4 5 6 7 8

Waterfall Lifecycle at AT&T Connect

Stability and Bugs


Devresolves bugs

Build and Integration tests

QA verifies bugs

Milestone

3 weeks

1 week

Bugs and stabilityVersion release

1 2 3 4 5 6 7 8

Agile Scrum


10

Developers QA

Waterfall to Agile Scrum at AT&T Connect

Agile Scrum Development


Dev + QA

Sprint

3 weeks Continuous Integration

StabilityVersion Release

91 2 3 4 5 6 7 8 9 11 12 1310

Continuous Integration Processes


Source control build

Static code analysis

Run automated unit Tests

PackagingPrepare test environment

Deploy to test environment

Run automated sanity tests

Publish report


Bug Lifecycle in Scrum

In scrums, bugs have the highest priority and there’s a “Zero-bug-approach!”


Sometimes, to save time, QA don’t open bug entries

We have to rapidly supply builds as part of Continuous Integration

Complete user story

Build Find bug Fix bug Build Verify bug

How We Provide Fast and Flexible Builds

• We moved from physical-build machines to VMs

• Each product build runs on a separate VM

• Builds are run in parallel

• We use IncrediBuild for a 10-fold increase in compilation time

• Scrum Masters or authorized users can start a build, any time, with a single click

• We added flexible reporting

• We use Jenkins


Sanity Tests: We Automate Them

Sanity test =

• Product installation• Application load• Basic functionality working as expected• Reporting

We use JSystem (based on Java®) to write the tests

QA team write and maintain scenarios

We run nightly sanity tests, using PowerShell® scripts


How We Made Multi-platform Testing Possible

The Challenge

We don’t know our client configurations…

• OS: XP®/Windows 7®/Windows 8®, 32/64 bit• Browsers: Chrome™/Internet Explorer® 7- 10• MS Office®: 2003/2007/2010• Admin/None Admin

We can think of 100s of combinations

Manual testing by QA on all platforms is impossible

Early feedback is missing


How We Made Multi-platform Testing Possible


Win7 32 Win7 64 XP 32 Win 8 Restart VM

Go back to

snapshot

Install latest

product version

Run sanity tests

Report

Shutdown VM

Running nightly

Chrome FireFox IE9 IE10

Virtual Machines

We Save Time with AutomaticEnvironment Installations

• Our environment includes more than 10 different kinds of servers and DBs

• Environment preparation takes no more than 1 day

• We use VMs

• We prepare snapshots in advance

• CI environments go back to production versions every night and automatically upgrade to latest versions

• We have different environments for QA, Dev, CI, etc.


Demo


Report Example


Summary

Waterfall to Agile


Rapid, fast builds

Sanity tests

Multi-platform testing

Environment installations


Thank You


SIGIST Conference, July 2013

Gal Fatal

[email protected]

054-342-3864

Continuous Integration Processes and SCM To Support Test...

Documents

Transcript of Continuous Integration Processes and SCM To Support Test...