Automated testing 101

Automated Testing 101

By Tabby Chapman

@tabbymarie

http://www.tabbychapman.com

About Tabby Chapman@tabbymarie

https://github.com/tabbymarie

About Me

• Work for the University of California, Irvine

• Computer Scientist

• Developer Cheerleader

• Education Junkie

About Me

• WordPress Skillz– Reaching my 10th anniversary developing

with WordPress – Started with 2.0.5, building a custom theme.– Ran my own company for four years– Joined Zeek for almost two years– Currently build highly specialized custom

themes and plugins for UCI.

About This Presentation

• Geared towards intermediate QA testers and Developers and also for stakeholders/clients/customers

• Assumptions:– You know what WordPress is.– You know what PHP is.– You have written PHP code several times.

Successfully. Or.– You test all the things.

About This Presentation

• We’ll go over– The Developer Workflow(s)– Different Types of Tests– Automated Testing– Demo– Last Words

How it Currently Works

• Generic Software Development Lifecycle– In the dinosaur days of computers, the

majority of its users were very technical people.

– As computers became more accessible (and smaller), more and more users began to use it

– Developers needed to create a process to think outside of their own processes to accommodate non-technical users

• Generic Software Development Lifecycle (cont)– Many various methods were created to

attempt to meet this need– One of these is the Software Development

Lifecycle (which was a similar process used in other disciplines)

– It is a top down approach: generic to specific– Specification, Design, Coding

• Generic Software Development Lifecycle (cont)– Waterfall method is created in the ‘70s. – Structured analysis, Structured design,

Structured programming (sounds a lot like specification, design, and coding)

– Most developers tend to use Define, Design, Develop, Deploy, Maintain.

Generic Modern Development

Image credit: https://www.owasp.org/images/8/84/SDLC.jpg

Freelance Development Cycle

Image credit: http://mohamadsubhan.files.wordpress.com/2009/06/linear.jpg

• Full Software Development Lifecycle– Many developers on teams tend to use some

form of the full development lifecycle, as there tends to need to be some form of accountability for teams.

– This includes: feasibility study, requirements definition, software design, implementation and unit testing, integration and system testing, operation and maintenance.

Team Development Cycle

Image credit: http://www.softwaretestinghelp.com/unit-testing/

Types of Tests

• Black box• White box• Unit • Integration• Functional • System• End-to-end

• Sanity• Regression• Acceptance• Load• Stress• Performance• Usability

Source: http://www.softwaretestinghelp.com/types-of-software-testing/

Testing Players

• Quality Assurance (QA) Engineers

• Developers

• Product Owners

• Clients

• End-Users

• Other Stakeholders

Black Box Testing?

Black Box Testing

• Tests against requirements and functionality. Is not concerned with the internal workings.

• “If I put in 2+4, I expect 6. Does that work?”

White Box Testing?

White Box

• Tests against requirements and functionality with knowledge of the inner workings.

• “I know that I this system has these conditionals, so I will test all possible outcomes.”

Unit Testing?

Unit Testing

• Tests specific methods or functions, created by a programmer

• “Does function addTwoIntegers($num1,$num2) work as expected?”

Integration Testing?

Integration Testing

• Testing of whole application after new work has been integrated as they interact with the client/server

• “Does this site’s individual modules still work as expected when we put it all together?”

Functional Testing?

Functional Testing

• Black Box testing for specific functional requirements.

• “If I tell it to add five text inputs and send an email, does it add five inputs and send an email as expected?”

Systems Testing

• Black Box Testing based on overall requirements.

• “Testing All the Things”. “Does this system meet all of the requirements as expected?”

End-to-end Testing

• Similar to Systems, but also tests interaction between various elements, protocols, layers and servers.

• “Testing All the Things and Moar”. “Does this system meet all of the requirements as expected and communicate as expected to its various parts?”

Sanity Testing

• A pre-test to make sure the new development is done enough to fully test.

• “Does it at least work a little bit?”

Regression Testing

• Testing the whole application after something has been modified to see if something was broken as a result of the modification.

• “Does it STILL work as expected?”

Acceptance Testing

• User or Customer does a round of manual tests to make sure that it meets their requirements/needs

• “Does the client think it works as expected?”

Stress Testing

• Testing Performance Requirements

• “Can more than one interaction happen at the same time? How many more than one?”

Usability Testing

• Testing for user-friendliness. This can be done BEFORE and AFTER development phase.

• “Yeah, it works, but is it easy to use?”

Which testing is important for developers?

For the Developers

• Which testing is important for developers?– All of them (know thy tester and thy tester’s

methods)– But especially

• Unit• Functional• Systems• Integration

Which testing is important for testers?

For the Testers

• Which testing is important for testers?– All of them (uphold they developer to unit

tests)– But especially

• Black Box• White Box• Integration• Systems• Sanity

Which testing is important for Clients/Stakeholders?

For the Clients/Stakeholders

• Which testing is important for clients/stakeholders?

• Black Box• Usability• Regression• Acceptance

For the Clients/Stakeholders

• Which testing is important for clients/stakeholders?

• Black Box• Usability• Regression• Acceptance

What does Automation of Testing mean?

• The traditional/easiest way to test what you develop is to make a change, and then load the app to see if that change was effective. Very Manual

• Automated Testing means to write code to test code and use testing software to test the front-end. Don’t rely on the old manual check.

How does this apply to WordPress?

• WordPress utilizes a ‘test suite’ “which have multiple test cases and a library and command line tool that runs the test suite or suites” according to WordPress’ automated testing core handbook page

• For this, they use phpUnit. https://phpunit.de

How does this apply to WordPress?

• PHPUnit has a series of commands that you can use to run all tests, specific tests, or specific types of tests.

Developer Workflow

• When you are ready to write your plugin:– Think of the outcomes you want for your

plugin– Think of the individual methods you want to

write. Should have one output.– Write a test that checks that the output is what

you expect.– Write your method.– Lather. Rinse. Repeat.

Developer Workflow

• Testing Tools

• phpUnit (official wordpress test suite)

• WordPress Developer Branch– https://make.wordpress.org/core/handbook/au

tomated-testing/

• Terminal or Editor with Testing Integrated

Developer Workflow

Editors (not limited to…)– TextMate - https://macromates.com/download

• (https://github.com/meritt/phpunit-tmbundle)

– Sublime – http://www.sublimetext.com/• https://packagecontrol.io/packages/PHPUnit

– phpStorm - https://www.jetbrains.com/phpstorm/

• Supports all OS’• Phpunit built in

Developer Workflow

• Put your tests IN your plugin

• /plugins/[plugindir]/tests/test-[pluginslug].php

• Add a phpunit.xml file:

• /plugins/[plugindir]/tests/phpunit.xml

Developer Workflowphpunit.xml

<?xml version=“1.0” encoding=“UTF-8” ?>

./tests/

</directory>

</testsuite>

</testsuites>

</phpunit>

./tests/

</directory>

</testsuite>

</testsuites>

</phpunit>

./tests/

</directory>

</testsuite>

</testsuites>

</phpunit>

./tests/

</directory>

</testsuite>

</testsuites>

</phpunit>

Tester Workflow

• When you are ready to test – Be familiar with the general requirements for

the site as a whole– Read the requirements of the change request– Focus on specifically what is supposed to

have changed.– Write a test while waiting for the change.– Run the test once you get the change from

the developer

Tester Workflow

• Selenium IDE, Firefox Plugin– http://www.seleniumhq.org/download/

• Browser Specific– Chrome

https://sites.google.com/a/chromium.org/chromedriver/

– Safari http://selenium-release.storage.googleapis.com/2.46/SafariDriver.safariextz

Tester Workflow

• Selenium– Installationhttp

://geekpad.ca/blog/post/automating-browser-testing-with-phpunit-and-selenium• First you must download the Selenium server and

the Internet Explorer Server from http://docs.seleniumhq.org/download/

• Then you must download the Chrome server from http://chromedriver.storage.googleapis.com/index.html

Tester Workflow

• Selenium– Installation (continued)

• You do not need to download a Firefox driver since it’s builtin the main Selenium file.

• Once all files have been downloaded you must add your selenium folder to your existing path see http://www.java.com/en/download/help/path.xml

• Then you can launch the server using java. With something that looks like this– Mac - java -jar selenium-server-standalone-[version].jar

Tester Workflow

Why is Selenium cool?

•Records your actions

•Let’s you test expected outcomes

•Let’s you test them in a series

Tester Workflow

Example

•The developer was given a task to make sure that all instances of the word “Orange County” is replaced with the word “OC” on the website, for “legal reasons”.

•Demo

Jenkins

• Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on the following two jobs:– Building/testing software projects continuously– Monitoring executions of externally-run jobs

https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins

Jenkins

Questions?

Resources

• http://www.dcs.bbk.ac.uk/~sven/cdm06/cdm3.pdf

• https://unit-tests.trac.wordpress.org/

• http://www.slideshare.net/ptahdunbar/automated-testing-in-wordpress-really

• http://johnregan3.co/2013/08/13/why-you-need-to-get-into-wordpress-unit-testing/

• http://blog.aizatto.com/2010/05/15/beginning-wordpress-automated-testing/

Resources

• https://www.owasp.org/index.php/Testing_Guide_Introduction

• https://make.wordpress.org/core/handbook/automated-testing/

• http://stackoverflow.com/questions/22414134/different-types-of-testing-eg-unit-functional-integration-etc-document

Resources

• http://www.seleniumhq.org/• http://geekpad.ca/blog/post/automating-browser-

testing-with-phpunit-and-selenium

• https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins

• https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins

Automated testing 101

Software

Transcript of Automated testing 101

Integrating automated acceptance testing with requirements ... · Integrating automated acceptance testing with requirements engineering ... Robotframework ... Integrating automated

Automated Testing Tools

Automated Testing .

Automated Browser Testing

Automated Durability Testing

Automated Testing Tool - Automated Regression Testing for ... · Automated Testing Tool For POS applications Connie Driscoll CEO CDA RHISCOM’s Representative connie@cdriscollassociates.com

Automated Performance Testing

Automated Testing Environment

Automated Website Testing

Automated Testing solution

Automated Testing Overview

Feuerstein Automated Testing

Automated Infrastructure Testing

Agile/Automated Testing

Automated Testing Plan

Automated UI Testing

TESTING 101

Automated testing

Automated Software Testing

Software testing techniques Automated software testing