Shifting left: Continuous testing for better app quality and security

Shifting left:Continuous testing for better

app quality & security

© Copyright 2016 NowSecure, Inc. All Rights Reserved. Proprietary information.

Connect

Twitter: @NowSecureMobile | @GuerrillaQA

—

Subscribe to #MobSec5, our weekly mobile security news digest

http://mobsec5.nowsecure.com/

—

Web: nowsecure.com | guerrillaqa.com

http://twitter.com/nowsecuremobile

https://twitter.com/guerrillaqa



https://www.nowsecure.com

Steven WinterFounder & Chief Strategist, GuerrillaQA


Andrew HoogCEO & Co-founder, NowSecure


Contents

● Why deploy more quickly?

● Going fast, achieving quality, & saving money

● What now? Must do’s!

● Continuous testing in practice

● Q & A


Why deploy mobileapps more quickly?


The business value of more frequent deployments

Happier customersNew features / improvements increase customer satisfaction & lead to faster

realization of revenue from new features.

Fix defects fasterIdentifying flaws earlier & shortening

the feedback loop leads to less expensive, faster fixes.

Reduce riskSmaller deployments include fewer

things that can go wrong, & those failures are easier to fix


Pain experienced as a result of infrequent releases

Dissatisfied customersApp users churn due to their perception

that the developer is not responsive with improvements & new features.

Slower reaction timeImprovements & fixes take longer to be released, are more expensive, &

leave customers dissatisfied longer.

High-risk, complex deploysMonolithic releases include more

dependencies & potential failures resulting in more expensive & time-consuming fixes.


What does the ideal look like?

Company Deploy Frequency Deploy Lead Time

Amazon 23,000 / day minutes

Google 5,500 / day minutes

Netflix 500 / day minutes

Facebook 1 / day hours

Twitter 3 / week hours

Typical enterprise Once every 9 months Months or quarters

Kim, Gene. "Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win" 2014.


How frequently are others deploying?

https://blog.newrelic.com/2016/02/04/data-culture-survey-results-faster-deployment/


Where are you in your journey? First steps

Automate what testing you can

Take advantage of Continuous Integration

Shift security & performance testing left


You can go fast, achieve quality, & save money


Earlier testing & remediation prevents technical debt

Requirements / Architecture

Coding Integration /Component

Testing

System /Acceptance

Testing

Production / Post-Release

Source: National Institute of Standards & Technology

The cost for fixing vulnerabilities is

30x higher after an app has been deployed


The cost of fixing a P1 mobile bug in production Case Study

Team Hours

Detection & communication 20

Verification 16

Fix 40

Build, test, certify the fix 60

Customer acceptance 40

Post-publish verification 20

Total hours 196

As well as

● Loss of client & app user confidence

● Negative app ratings

● Derailment of feature development & release

$35KIn total costs


Automation pays for itself with repeatability

Manual Testing

Automated Testing

Releases

Time / Effort

Time Savings


Development / Integration Staging Production

Dev TeamVersion Control

Build & Unit Tests

Automated Acceptance

TestsRelease

User Acceptance

Tests

Check-in

Check-in

Check-in

Trigger

Trigger

Trigger

Trigger

Trigger Approval

Approval

Feedback

Feedback

Feedback

Feedback

Feedback

Feedback

Engineer QA DevOps


Development / Integration Staging Production

Dev TeamVersion Control

Build & Unit Tests

Automated Acceptance

TestsRelease

User Acceptance

Tests

Check-in

Check-in

Check-in

Trigger

Trigger

Trigger

Trigger

Trigger Approval

Approval

Feedback

Feedback

Feedback

Feedback

Feedback

Feedback

Engineer QA DevOps

Shift security & performance Testing to the left


CI + CT = !!

Continuous Integration+

Continuous Testing=

Productivity multiplier

● Sets the stage for “set-it-and-forget-it” deployment

● Deliver higher quality code at lower risk in less time

● “Parallelizes” testing

○ Security, regression, performance, etc.

○ Simultaneously

● Repurpose test scripts

○ Write once

○ Use everywhere


What now? Must do’s!


1. Agree & commit to improving

Must do’s!


Must do’s!

1. Agree & commit to improving2. Plan testing & automation scripting up front


Must do’s!


3. Agree on test coverage


Must do’s!


3. Agree on Test Coverage4. Measure, measure, measure


Must do’s!


3. Agree on Test Coverage4. Measure, measure, measure

5. Plan for test script maintenance


Continuous testing in practice


Case study: Value realized in just a few hours


Steven’s experience at scaleFrom 4 months to Nightly


Scaling your automated testing based on maturity

Small Medium Enterprise

● Leverage open-source tools

● Build CI environment

● Create a basic smoke test

● Expand test coverage

● Leverage cloud platform services

● Plug security & performance

testing into CI

● Create smoke tests for each

feature (not the entire app)

● Prioritize by feature’s

success / risk

● Pick the top three & go!

Let’s talk

NowSecure+1 312.878.1100

@NowSecureMobilewww.nowsecure.com

GuerrillaQA+1 415.763.TEST

@GuerrillaQAwww.guerrillaqa.com

Subscribe to #MobSec5 - a collection of the week’s mobile news that matters - http://mobsec5.nowsecure.com/

Shifting left: Continuous testing for better app quality and security

Software

Transcript of Shifting left: Continuous testing for better app quality and security