Automated Testing

About Promet

Mission Statement: “Promet Solutions specializes in developing innovative, complex web applications with open source technology.”

30+ solutions architects worldwide

• Offices in the US • Chicago • San Jose

10th year in business

32 open source contributions

10+ mobile apps in the past 12 months

ABOUT JOHNNIE:

• Presenter • Johnnie Fox • @johnniefox

• Promet Source • Director of Development • Scrum Master • Project Manager • Recovering Developer • @prometsource

ABOUT CHRIS

• Presenter • Chris Calip

• Promet Source

• Application Developer • @prometsource

Insert picture of Chris

HISTORY OF DRUPAL TESTING

Drupal Testing: D6 core – Simpletest Application specific testing - Manual Tests - Written test cases – Word documents - The spreadsheet of pain. (Checklist)

Challenges -

Testing different browsers Reproducible results Providing a record Doesn’t test integration with 3rd party API (backend functionality)

CONTINUOUS INTEGRATION:

Continuous Integration (CI) implements continuous processes of applying quality control — small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development. Continuous Integration emerged in the Extreme Programming (XP) community, and XP advocates Martin Fowler and Kent Beck first wrote about continuous integration circa 1999.

Continuous integration - Wikipedia, the free encyclopedia : taken from - http://en.wikipedia.org/wiki/Continuous_integrationhttp://creativecommons.org/licenses/by-sa/3.0/

HACKATHON TEAM :

Five. Striving as one. Driven by a passion for continuous automated quality control.

Ruoran Wang Scott Massey Chris Calip Johnnie Fox Michelle Krejci

HACKATHON OBJECTIVE:

Develop, deploy, demonstrate, and identify challenges of making using ci infrastructure to implement acceptance tests (unit tests, selenium) on one existing site.

TECHNICAL DETAILS

phpqatools.org

provisions servers with testing components

admin gui to trigger test suites, or view reports

code metrics

user acceptance

Necessary components

•For necessary components and installation instructions, go to:

•Phing Drupal Template • http://reload.github.com/phing-drupal-template/ • Jenkins Drupal Template • http://reload.github.com/jenkins-drupal-template •Phing Drupal Template with User Acceptance Test • http://github.com/promet/phing-drupal-template • In addition for phing-drupal-template with user acceptance test, you need to install additional pear libraries

• For more instructions, http://phpqatools.org/

Installing Infrastructure

Initial assumption: your server has several initial software already 1. Go to https://github.com/promet/phing-drupal-template 2. Three files that you need to use to install infrastructure 3. Put in on your site Drupal root 4. If you want to override it, build.properties 5. Customize variables

EXAMPLE TEST SUITE FOR SITE SUPPORT

Jenkins: Scheduled 1AM User

Acceptance Test Jenkins: Clone

Production to Test

PHING: Autodiscover the Selenium test

PHING: Selenium Config Upload

Done

Jenkins: If errors found, notify stakeholders

PHING: Report Generated

Jenkins & PHING: User Acceptance PHPUnit +

Selenium

PHING: Trigger each Selenium test and generate report

PHING: PHPUnit Configuration Load

PHING: Trigger each PHPUnit Test and generate report

PHING: Autodiscover the PHPUnit test

Without Bootstrap

With Bootstrap and what Bootstrap file

(ability to use Drupal API)

What is Selenium?

•Selenium automates browsers. That's it. What you do with that power is entirely up to you. Primarily it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) also be automated as well.

•Selenium has the support of some of the largest browser vendors who have taken (or are taking) steps to make Selenium a native part of their browser. It is also the core technology in countless other browser automation tools, APIs and frameworks.

• Goes through every page • Makes sure header, footer, logos, etc • All branding elements in place

Functional Test PHPUnit

• Project Airbox • Checkout process that involves passing customer information to third party

banking API, passes back customer ID • Months after completion, API changes broke the application

• PHPUnit figures out how and where the problem is • More importantly, is there a problem?

• Major time saver for detective work • -- bootstrap option to make use of forked upal without SUT • https://github.com/chriscalip/upal • http://drupal.org/node/1567500 • actually installs a complete Drupal site like the regular Drupal installer for every test

being executed, using a random database prefix and some logic baked into Drupal's (regular runtime) core that re-routes all HTTP requests originating from WebTestBase into the "child site under test" (the db-prefixed one). Based on that, every test is able to specify additional modules to install into that child site, in order to test various combinations of modules and configuration and content/data and user input and

EXAMPLE TEST SUITE FOR DEVELOPMENT SITES

Guiding principle: Fast Iteration

Traditional Dev Staging Production and Workflow

Local Dev Staging Production

Manual Manual Manual

Lots of commits Significant Event (Less Frequent)

Dev Staging Production Work Flow with Automated Continuous Integration Tests

Local Dev Test Branch Production

Manual Manual Manual

Run Test Suite

Staging

Automatic push if passes

Jenkins: Clone Staging DB to test DB

Jenkins: Run code-metrics test

Jenkins: Run user acceptance test

Jenkins: Run Drupal Core and Contribution Simple test

Manual Test

Jenkins: Clone Staging DB to test DB (5 mins)

Jenkins: Run code-metrics test (10-15 mins)

Jenkins: Run user acceptance test (10-15 mins)

Jenkins: Run Drupal Core and Contribution Simple test (Needs work)

Jenkins: RUN ALL (35 mins)

All those test eats up server resources ...

.

Drupal

Sites

All

Modules

bathroom

conference room

Kitchen (Contrib)

bathroomSeleniumTest.php

bathroomUnitTest.php

conferenceroomUnitTest.php

conferenceroomSeleniumTest.php

# Chosen File Pattern by project for phpunit ie.. ExampleUnitTest.php SampleUnitTest.php project.phpunit.filepattern = UnitTest.php # Selenium # Chosen File Pattern by project for selenium ie.. ExampleSeleniumTest.php SampleSeleniumTest.php project.selenium.filepattern = SeleniumTest.php

ANATOMY OF USER ACCEPTANCE TEST ANATOMY OF USER ACCEPTANCE TEST

build.properties project.code.custom = bathroom conference_room

Example Jenkins Dashboard

Example of Code Metric Test PHP Mess Detector

EXAMPLE OF PHPUNIT REPORT

GIT POST HOOK TRIGGERING THE TEST SUITE.

Dev writes the code & test

•Dev pushes code

Client updates database • Finding a block with Selenium

http://demosource.prometdev.com/blog

Questions? www.prometsource.com

chriscalip@prometsource.com johnnie@prometsource.com

Twitter: @prometsource