AVOCADO AND JENKINS -...

AVOCADO AND JENKINSAVOCADO AND JENKINST E S T- A U TO M AT I O N A N D C O N T I N U O U ST E S T- A U TO M AT I O N A N D C O N T I N U O U SI N T E G R AT I O NI N T E G R AT I O N

YASH MANKAD LUKÁŠ DOKTORYASH MANKAD LUKÁŠ DOKTOR

Software Engineer, Red Hat Software Engineer, Red Hat

06-02-2016

DevConf 2016

OVERVIEWOVERVIEW( What are we going to learn ? )

Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor

Previous SetupProblem Statement

Patch handling

Development-Quality Engineeringhandoffs

Test-automation

OVERVIEWOVERVIEW( What are we going to learn ? )


Design GoalsWhere did we start ?

ApproachIntegrating test-automation

Challenges faced

Why Avocado ?Demo


PREVIOUS SETUPPREVIOUS SETUP( How did it work before ? )

Developer Patch Mailing List

Patch QueueBuildsQuality Engineering

Developer

Patch Mailing List Patch Queue Builds


Maintainer

PROBLEM STATEMENTPROBLEM STATEMENT( Why are we changing it ? )

ExtensiveGuidelines

Development



Quality Engineering

Development

Patch HandlingExtensive Patch guidelinesManual builds

Development-QE Handoffs

Different testing schedulesTime between patch submissionand testing is too long



Test-AutomationQuality Engineering teams used Autotestframework

Quality Engineering has to use lot of tools ontop of Autotest

Autotest is no longer maintained upstream


WHERE DID WE START?WHERE DID WE START?( Our initial efforts )

Developer Patch Mailing ListPatch Queue

Fail early, fail oftenCredit to Fam Zheng's Patchew(http://qemu.patchew.org/)Wrote a script that automates a part of the earlytesting process and runs functional sanity tests


WHERE DID WE START?WHERE DID WE START?( Making things run more efficiently )

Patch Queue

Had to focus on Development-Quality Engineeringhandoff

Script applies patches and triggers builds, whichtriggers the CI

Builds


INTEGRATING TEST-INTEGRATING TEST-AUTOMATIONAUTOMATION( How it is all coming together! )

Tests run based on keywords such as boot, migrate,network, virtio, etc.Different Jenkins slave based on architecture andcomponent

Development Quality Engineering

+Avocado

CHALLENGES FACEDCHALLENGES FACED( What we had to overcome )


Implement new features without affecting oldfeatures

libvirt CI supports upstream community anddownstream for RHEL

Jenkins jobs and configuration are automatedusing scripts

Currently, we are migrating from Autotest toAvocado

WHY AVOCADO ?WHY AVOCADO ?



The only possible alternative for virtualization:40017 qemu, 56745 libvirt tests (Also supportslibguestfs, lvsb, v2v, openvswitch, spice testproviders)3 (5) types of cpu architecture147 versions of operating systems864 hardware variants25412395680 theoretical combinations

Measured on 2016-02-03 using x86_64 + RHEL.7.devel to evaluate the number of tests and hw variants



Talks to JenkinsEven our developers are using it locallyYou can actually run “anything” with itGives you well defined results structure (withbenefits)Supports neat features


Questions?

Thank You!

More questions?

[email protected]

[email protected]

AVOCADO AND JENKINS -...

Documents

Transcript of AVOCADO AND JENKINS -...