Manoj kolhe - Continuous Integration Testing

www.unicomlearning.com

World Conference on Next

Generation TestingBangalore, 17th and 18th July 2014

AUTOMATED INTEGRATION TESTING

MANOJ KOLHE

www.nextgentesting.org

http://www.unicomlearning.com/

http://www.nextgentesting.org/


About Me…

UNICOM Presents


Generation Testing www.nextgentesting.org

Manoj KolheProject Lead

* [email protected]

www.Lntinfotech.com

) +91 880 50 15 100

• 8+ years of IT experience Development & Testing areas• Certified in Java, PL/SQL & MongoDB• Technology Consultant in Technology Competency Group within

Testing Service Line BU• Team awarded with top rated entry for Innovation in Test Automation

from L&T• Domains: DW/BI, Insurance, Telecom, Automobile, Media &

Entertainment




AGENDA

UNICOM Presents



• Introduction

• Traditional way of Testing

• Challenges in existing Testing Model

• Addressing challenges using Open Source Solutions

• Introduction to Automated Test Integration

• Implementation of CIT / AIT

• Tool selection criteria for Test Automation

• Case Study

• Covering Up




Introduction – Typical Enterprise Landscape

UNICOM Presents



Sales & Marketing TeamBusiness Customers

Business Requirements

Development Team

Code Repository Staging / Production Server

Testing Team

Regression Testing

That’s a bug!!!

That was

working before!

Testing and Iterations

New Requirement




Challenges in existing Testing Model

UNICOM Presents



Traditional way of testing

Difficult to maintain stringent timelines

Drawbacks

Increased Project cost

Development oriented project instead of customer / quality oriented




Challenges in existing Testing Model

UNICOM Presents



Environment Challenges

• Multiple Test environments

• Continuous deployment

• Automated Deployment and Regression

Testing

People Challenges

• Lack of Skill set & Strong knowledge

• Lack of competence to work in additional

technologies - Scripting & Macros

Process Challenges

• Stringent and varying timelines

• Multiple layers and diversified testing at

various layers

Technology Challenges

• Unavailability of a technology agnostic

tool

• Increasing vendor stack in the testing

space




Addressing challenges using Open Source Solutions

UNICOM Presents



• Test Driven Development (TDD)

• Unit Testing Tools

• Mocking Frameworks

• Source Code Management Utilities

• Code Review and Analysis

• Continuous Integration Testing Tools

• Continuous Deployment




Addressing challenges using Open Source Solutions

UNICOM Presents



• Test Driven Development (TDD)

Write the Test Case Execute the Test Case

Write enough code to pass the test

1 2

3

Test Case Failed

Repeat

Test Case Passed

Write the next Test Case

1

Code Review and Continuous Integration

Testing




Introduction to Automated Test Integration

UNICOM Presents



Need for Automation Testing

• Can accurately record and test the time lapsed between two transactions

• Increased Test Speed, Test Efficiency and Software Quality

• Decreased Costs and shorter Time to Market

• Record and play feature

• Increased Confidence

• Greater Test Coverage

• Reusability

• Automated Reporting




Implementation of CIT / AIT

UNICOM Presents



Source Code

Version Control System

Source Code Build

Static Code AnalysisRun Automated

Unit TestsCode Coverage Analysis

Set-up Test ‘Fixtures’

Deploy to Test Environment

Run Automated Functional TestsPublish

Reports

Development Team

1

23

4

5 6 7

89

10

1112

Automated Testing is the key to AGILE

Arte

fact




Tool selection criteria for Test Automation

UNICOM Presents



• Small learning curve

• Environment Support

• Easy to use

• Generic across multiple platforms

• Easy Troubleshooting and debugging

• Minimum configuration

• Support for various types of Tests

• Precise Reporting




Case Study – Sparrow – M&E Leader in Nordic

UNICOM Presents



Sales & Marketing TeamBusiness Customers

Business Requirements

Development Team

Code Repository Web Server

Testing Team

Regression Testing

That’s a bug!!!

That was

working before!

Testing and Iterations

New Requirement




Case Study – Web Platform Existing Scenario

UNICOM Presents



Web Application Flow

Req

uir

em

en

ts P

hase

Requirement Analysis

Getting defects data

Co

din

gBrowser Specific Coding

Run Unit Tests

Man

ual

/ U

nit

Testi

ng

Prepare debug release

Deployment Activity

Testi

ng

Run regression suit

Validate bugs / defects

Update defect database

Prod

ucti

on

Prepare Production build

Deployment activity




Case Study – Mobile Platform Existing Scenario

UNICOM Presents



Mobile Application Development Flow

Req

uir

em

en

ts P

hase

Requirement Analysis

Getting defects data

Co

din

gDevice / Platform specific coding

Prepare Debug Release

Man

ual

/ U

nit

Testi

ng

Deploy Debug Package on Emulator

Run manual Unit Tests

Reg

ressio

n T

esti

ng

Prepare Beta Package

Validate bugs / defects

Update defect database

Prod

ucti

on

Prepare Release Package

Perform pre-release activities

Publish to the market / central repository




Case Study – CIT Setup

UNICOM Presents



Jenkins

QC

Data Testing

Framework

Software Test

Platform

BI Reports

Validation

Test Data

Management

Batch Job Layer

Rep

ort

Up

load

Mobile Platform

Simulator

Developer

Source Code

Repository

Defect Report

Check-In

Source code Pull Request

Lead

ing

Me

dia

& E

nte

rtai

nm

en

t C

om

pan

y in

No

rdic

Cross Browser

Testing




Case Study – CIT Setup

UNICOM Presents



Code Repository

Continuous Integration

Server

Job Configuration

1. Code Checkout

2. Run Unit Tests / Selenium Tests

3. Run Database Automation Scripts

6. Deploy on Staging server

8. Run Performance Tests

9. Publish Functional / Performance Test Result(s)

5. Build Deploy package

7. Trigger Regression Suit

4. Run GUI Tests, via Sikuli / QTP

Push Defects to QC

Development Team

Staging Server

Database




Case Study – Infrastructure Requirements

UNICOM Presents



Hardware Requirements• 4 GB RAM• 3 GHz CPU

Software Requirements• JRE 1.6• Microsoft Office 2007• Windows / Unix OS• Apache Tomcat 7.0• Android Emulator (v19 preferred)• HP Quality Center v11• CVS• Jenkins WAR and plugins• RADAR Framework tools• Apache Maven / Apache Ant / Gradle




Case Study

UNICOM Presents



Accuracy Dependability Value Benefits Quality

Reduced effort by

40%

Improved test

coverage by 80%

Automated and rule

based code review

Utilized various

components under

single umbrella

Performed integration

with HP-QC and CVS

using open source

plugins

End-to-end automated

data testing strategy

Reduced testing cycle

time by 70%

De-skilling of

resources

Improved time-to-

market by 30%

90% of the Test Cases

Automated End-to-end

Automated reports

generation

Reports status update

in QC instantly post

test execution

Automated Integration Testing for Sparrow Tech Pvt. Ltd.

Benefits Achieved through Automated Integration Testing




Covering Up…

UNICOM Presents



Limitations / Drawbacks of Automated Integration Testing

• Initial Setup time required

• Methodology can not be ‘one-size-fits-all’ Exception: Too Small Product life

• Automatic Compile, Deploy does not mean Automation

• It's about tests not builds

• Continuous Integration is an Attitude – Not a Tool

• Source code in the repository must always build and pass our tests

• Places where outcome of CI Reports does not mean to anyone

• Failover conditions’ analysis required




UNICOM Presents



Q&A




World Conference on Next Generation

Testing

Organized by

UNICOM Trainings & Seminars Pvt. Ltd.

[email protected]

MANOJ KOLHE

[email protected]


Manoj kolhe - Continuous Integration Testing

Software

Transcript of Manoj kolhe - Continuous Integration Testing