Containerized End­2­End Testing

+

, Tobias Schneck ConSol Software GmbH

Test Pyramid

RobustIndependentIsolatedFast

Characteristics of End­2­End Testing

Different types of testing:Regression testsFunctional approval tests

Parallel tests with GUIs are complexStateful tests: user logins, sessions, historySetup and cleanup of test data

Manual effort > effort for test automation

Advantages of Container Technology

Isolation of environmentsRepository for versioning and distributionReproducible application environmentDockerfile, docker­compose.yml

Starting containers on­the­flyOptimized for parallel executionLess memory overheadShared Linux kernel

Containerized GUIs### start the docker container via x­forwardingdocker run ­it ­e DISPLAY=$DISPLAY ­v /tmp/.X11­unix:/tmp/.X11­unix:rw rasch/inkscape

### start the docker container with VNC interface# connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassworddocker run ­it ­p 5911:5901 ­p 6911:6901 consol/ubuntu­xfce­vncdocker run ­it ­p 5912:5901 ­p 6912:6901 consol/centos­xfce­vnc

What's provided by ?

CategoryWeb tests through HTML selectors

Restricted to browser content

Open Source & Java API

Headless execution

Test writing assistance(recorder, tag finder)

Automatable / Test result reporting(CI, DB, monitoring environment)

Nagios OMD Incinga

Monitoring Integration

Test Definition (Java Script)

Test Case Structure// testcase.js

/************************************** Initialization*************************************/_dynamicInclude($includeFolder);var testCase = new TestCase(60, 70);var env = new Environment();var appNotepad = new Application("gedit");var region = new Region();

/******************************* Description of the test case******************************/try

//...

/************************************************* Exception handling and shutdown of test case**********************************************/ catch (e) testCase.handleException(e); finally testCase.saveResult();

Call all Sahi Functions for Web Testing// testacase.js/************************* Call Sahi Functions***********************/

_navigateTo("http://sahi.example.com/_s_/dyn/Driver_initialized");_highlight(_link("SSL Manager"));_isVisible(_link("SSL Manager"));_highlight(_link("Logs"))_click(_link("Logs"))

testCase.endOfStep("Test Sahi landing page", 5);

Fluent API for UI Testing// testacase.js

/*** open calculator app ***/var calculatorApp = new Application("galculator").open();testCase.endOfStep("Open Calculator", 3);

/*** calculate 525 + 100 ***/var calculatorRegion = calculatorApp.getRegion();calculatorRegion.type("525") .find("plus.png") .click() .type("100");

calcRegion.find("result.png").click();calcRegion.waitForImage("625", 5);

testCase.endOfStep("calculate 525 +100", 20);

Custom Functions// e.g. excluded into a separate common.js

/*********** Combine click and highlight*********/function clickHighlight($selector) _highlight($selector); _click($selector);

/**************** Open PDF in native viewer**************/var PDF_EDITOR_NAME = "masterpdfeditor3";

function openPdfFile(pdfFileLocation) return new Application(PDF_EDITOR_NAME + ' "' + pdfFileLocation + '"').open();

Test Definition (Java)public class FirstExampleTest extends AbstractSakuliTest

private static final String CITRUS_URL = "http://www.citrusframework.org/"; private Environment env; private Region screen;

@BeforeClass @Override public void initTC() throws Exception super.initTC(); env = new Environment(); screen = new Region(); browser.open();

@Override protected TestCaseInitParameter getTestCaseInitParameter() throws Exception return new TestCaseInitParameter("test_citrus");

@Test public void testMyApp() throws Exception // ... testcode

For the Maven dependencies, take a look at the .Java DSL Documentaion

Test Definition (Java)public class FirstExampleTest extends AbstractSakuliTest

// ... initializing code

@Test public void testCitrusHtmlContent() throws Exception browser.navigateTo(CITRUS_URL); ElementStub heading1 = browser.paragraph("Citrus Integration Testing"); heading1.highlight(); assertTrue(heading1.isVisible());

ElementStub download = browser.link("/Download v.*/"); download.highlight(); assertTrue(download.isVisible()); download.click();

ElementStub downloadLink = browser.cell("2.6.1"); downloadLink.highlight(); assertTrue(downloadLink.isVisible());

@Test public void testCitrusPictures() throws Exception browser.navigateTo(CITRUS_URL); env.setSimilarity(0.8); screen.find("citrus_logo.png").highlight(); env.type(Key.END); screen.find("consol_logo.png").highlight();

Sakuli End­2­End Testing Container

Demo ­ Sakuli Container# start the docker containerdocker run ­it ­p 5911:5901 ­p 6911:6901 consol/sakuli­ubuntu­xfcedocker run ­it ­p 5912:5901 ­p 6912:6901 consol/sakuli­centos­xfcedocker run ­it ­p 5913:5901 ­p 6913:6901 consol/sakuli­ubuntu­xfce­javadocker run ­it ­p 5914:5901 ­p 6914:6901 consol/sakuli­centos­xfce­java

# start in parallel via docker­compose# use docker­compos.yml from https://github.com/ConSol/sakuli/tree/master/dockerdocker­compose up

Bakery Demo Setup

Bakery Demo Setup

Bakery Demogit clone https://github.com/toschneck/sakuli­example­bakery­testing.git

# start jenkinsjenkins/deploy_jenkins.sh# start OMD montioringomd­nagios/deploy_omd.sh

# start the build of the application imagesbakery­app/app­deployment­docker­compose/deploy_app.sh

#start testssakuli­tests/execute_all.sh#start tests for monitoringsakuli­tests/execute_all_4_monitoring.sh

What's next?

Headless execution ­ Linux: VNC & Docker Windows: ?Cloud­ready container for OpenShift (security, source­2­image)Video recording of the test execution (error documentation)Web UI to handle Sakuli test suitesConnect 3rd­party test management tools (HP QC, TestRail, ...)Improve test result presentation in CI toolsSelenium as an alternative to SahiImplement Junit 5 test runner

Links

ConSol/sakuli

Consol/sakuli­examples

toschneck/sakuli­example­bakery­testing

sakuli@consol.de

Thank you Mannheim!

Tobias Schneck

ConSol Software GmbHFranziskanerstraße 38D­81669 MünchenTel: +49­89­45841­100Fax: +49­89­45841­111

tobias.schneck@consol. toschneck

info@consol.dewww.consol.de

ConSol