T12

Special Topics

5/8/2014 11:15:00 AM

DevOps: Where in the World Is

Test?

Presented by:

Erik Stensland

Pearson Learning Technology

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Erik Stensland

Pearson Learning Technology

Software quality architect for Pearson Learning Technologies, Erik Stensland is leading the technical testing efforts with a concentration on functional, performance, and integration testing. In the software test and quality engineering field for more than twenty years, Erik focuses on implementing innovative ideas around process and tools to help teams become more effective and efficient. His previous roles have included senior QA management positions and QA consultant for Internet startups and DoD industries. A board member for the Software Quality Association of Denver, Erik has been an international presenter at leading testing conferences in Europe and the United States. Follow Erik at testarch.blogspot.com/ and on Twitter @erikstensland60.

4/26/2014

1

DevOps: Where in the

World is Test?

StarEast 2014 Conference

Date: May 8th, 2014

Author: Erik Stensland

A Little about me

Blog: http://testarch.blogspot.com

LinkedIn : http://www.linkedin.com/in/erikstensland/

Twitter : https://twitter.com/erikstensland60

email: estensla@yahoo.com

4/26/2014

2

What “IF” ?

Agenda

• What is DevOps

• DevOps 1, 2, 3

• Challenges for QA

• Take Away’s

• Case Study

4/26/2014

3

First Thought!

The Backyard brawlers

Then Later-.

4/26/2014

4

Dev

Ops

4/26/2014

5

TEST/QA

Finally!!!

4/26/2014

6

What is DevOps

Where did it come from?

4/26/2014

7

DevOps Principles

Benefits of

DevOps

� Faster time to market (reduced cycle times

and higher deploy rates)

� Increased Quality (i.e., increased availability,

increased change success rate, fewer failures)

� Increased organizational effectiveness (increased time spent on value adding activities

vs. waste, increased amount of value being

delivered to the customer).

4/26/2014

8

Questions?

The Challenge for QA

High deployment rates typically associated

with DevOps work streams will often put

enormous pressure on QA.

4/26/2014

9

No longer to find defects,

necessarily.

4/26/2014

10

Prevent defects

Methodology

4/26/2014

11

Automation/Measurement

Culture / People

4/26/2014

12

Keys to Test/QA in

DevOps

� Your purpose is to Prevent Defects.

� Focus lies on Risks to the product instead

of just the functionality of the software

changes.

� QA must be testing early and often using

“Walk Away Automation” to Integrate QA

into the automated deployment flow.

� Team members need to have the “Right

Attitude”

The Case Study:

Code Name - Manhattan

4/26/2014

13

What is Manhattan

YESTERDAY

4/26/2014

14

Software Engineering• Few Builds a week. ~1 Deployment every week or two.

• Limited Unit testing

• Manually Build and Deploy to Pre-Prod

Software Quality Engineering• Very little architecture review for testability.

• Heavy Manual testing.

• Limited UI automated tests, Little web services automated tests.

• Automated test manually kicked off.

• Some performance testing.

IT Operations• Manual recompiling of Code for different source code repository

• Manual Deployments and Configurations of Production servers

• Limited monitoring of systems , No monitoring of Applications

• No review of Systems before Code is delivered for Deployment

Yesterday’s Environment

The “Discussion”

4/26/2014

15

Extend Eng

into

Production

• Created a Continuous Integration and Release

process using Jenkins.

• Deeply integrating QA into the release process by

creating “walk away automation”.

• Ensure production readiness of the Code as well as

the Environment by spinning up new

environments before code deployment and testing.

4/26/2014

16

Production

Feedback into

Eng

• Implemented Systems and Application Monitoring

in pre-prod and prod.

• Use one incident reporting system for Dev, QA

and Ops to monitor during production releases and

incidents.

• Created blameless postmortems for all releases

and incidents.

Embed Eng

into IT Ops

• Development and QA resources added to the

rotation for production support.

Embed IT Ops

into Eng

• Created reusable stories for non-functional such as

Monitoring, Environment creations.

4/26/2014

17

TODAY – Software Eng

• 100’s of Builds a day.

• Large amounts of Unit tests

• Multiple Deploys a day to pre-prod

• Push Button Deploys to all pre-prod

environments.

TODAY – Test/QA

• Same Repository for Tests as Code

• Large sets of Automated tests.

• New feature Exploratory testing.

• Low maintenance automation tests.

• Continuously running all Automated

Tests in All Pre-Prod environments.

4/26/2014

18

TODAY – IT Operations

• Same Repository for Code and

Environment scripts.

• Push Button Deployments to

Production.

• Monitoring at system and application

on all new deployments.

Ask Yourself What “IF” ?

4/26/2014

19

Give away questions?

�What is the name of the Band I

showed in the Video?

�How much did the Krupp Bagger

288 weigh?