Automated Unit Testing using PHPUnit, Selenium and Phing
Tricode Professional Serviceswww.tricode.nl
01-05-2008Sander van Beek & Patrick van Dissel
“One test is worth a thousand expert opinions”-- Bill Nye, The Science Guy
Automated Unit Testing
- Put Angels on Your Shoulders- Unit Testing
- Use It Before You Build It- PHPUnit
- Different Makes a Difference- Selenium
- Automate, Automate, Automate- Phing
Put Angels on Your Shoulders
Unit Testing is a software verification and validation method where the programmer gains confidence that individual units of source code are fit for use
A unit is the smallest testable part of an application, e.g. a method
Unit testing is regression testing, making sure made changes are not a step backwards
Unit Testing
Without Unit Testing:You write code and stick in some print statements to see the value of a few key variables. You run the code, maybe in a debug mode, and look at the results manually, fix problems that come up
Then you throw away the print statements or exit the debug mode, and move on to the next item
Unit Testing
With Unit Testing:With Unit Testing you take that familiar process and instead of throwing away debug code, you save it, and continue to run it automatically
Instead of manually inspecting the interesting variables, you write code to check for specific values in specific situations
Unit Testing
Benefits:- Provides instant feedback
Code gets exercised repeatedly. As you change and rewrite your code, test cases will check that you haven’t broken existing contracts. You can quickly identify and fix any problems
- Makes your code robustTesting helps you think through the behavior of the code, exercising the positive, negative and exceptional cases
Unit Testing
Benefits (continued):- Can be a helpful design tool
Unit testing can help you achieve a pragmatic and simpler design
- Is a confidence boosterYou’ve tested your code and exercised its behavior for a variety of different conditions:
this will give you the confidence when faced with new, high pressure tasks on tight deadlinesYou can change anything as long as the tests stay green!
Unit Testing
Benefits (continued):- Can act as probes when solving problems
Tests go red when something is wrong. If something is wrong and tests stay green, that means that new test-cases need to be created for that specific situation
- Are reliable documentationUnit tests can serve as accurate, reliable documentation
When following an easy naming-convention you can even create documentation based on unit tests
Unit Testing
Benefits (continued):- Are a learning aid
You can write unit tests against an API to facilitate your learning. The tests not only help you understand the behavior of the API but also help you quickly find any incompatible changes that might be introduced later
Unit Testing
Limitations:- Public API only
Unit tests are focused on the public API only!
Protected and private API can be tested, but wrappers are needed to get this to work. It’s a bit more work, but advised for very important/complex methods
- Tests are not created and updated by themselves
Creating and keeping unit tests up-to-date is not an automatic process. You must to have the discipline to create and update the tests yourself!
Unit Testing
Limitations (continued):- Not everything can be tested
By default, unit tests are focused on testing behavior of single units. Integration, performance, GUI and user acceptance tests are not the focus or supported by unit testing alone
Using unit testing in combination with other testing tools can extend the reach of the tests. More and more extensions are added to unit testing frameworks to support these tools
Unit Testing
Some best practices:- Don't assume the order in which tests within a test case
run- Avoid writing test cases with side effects- Do not load data from hard-coded locations on a file
system- Keep tests in the same location as the source code- Name tests properly- Keep tests small and fast- Test Behavior, Not Methods- Apply the “Too Simple to Break” Rule- Use Object-Oriented Best Practices In Your Test Cases
Use It Before You Build It
PHPUnit is a unit testing framework for PHP5
It is one of the xUnit family of frameworks that originated with Kent Beck's SUnit for Smalltalk
Wikipedia has a nice list of unit testing frameworks:http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
PHPUnit
Static Test structure
PHPUnit
Runtime Test structure
PHPUnit
Basic concepts of xUnit:– Test fixture
The set of preconditions or state needed for a test to succeed. Also known as a test context.
– Test caseA single test method within a test suite.The setUp() and tearDown() methods of the test suite are executed before and after each test case, respectively
– AssertionEvaluate the result of the test case
– Test suiteA collection of test cases, andA collection of test suites
A simple Test Suite
class ArrayTest extends PHPUnit_Framework_TestCase{ public function testNewArrayIsempty() { $array = array(); $this->assertEquals(0, count($array)); }}
A simple Test Suite
class ArrayTest extends PHPUnit_Framework_TestCase{ protected $fixture; protected function setUp() { parent::setUp(); $this->fixture = array(); } protected function tearDown() { parent::tearDown(); $this->fixture = null; } public function testNewArrayIsempty() { $this->assertEquals(0, count($this->fixture)); }}
Writing tests
- A test suite is a class that extends from PHPUnit_Framework_TestCase
- A test case is a method within a test suite, which:- Method name starts with ‘test’, or- Is annotated with ‘@test’
- The setUp() method of a test suite can be overriden and should be used to setup the fixture. setUp() is called before each test case
- The tearDown() method of a test suite can be overriden and should be used to cleanup the fixture.tearDown() is called after each test case
Executing tests
DEMO- From command-line- From Zend Studio
- Generating a result report- Code-coverage
Data providersclass DataTest extends PHPUnit_Framework_TestCase{ /** * @dataProvider provider */ public function testAdd($a, $b, $c) { $this->assertEquals($c, $a + $b); }
public function provider() { return array( array(0, 0, 0) , array(0, 1, 1) , array(1, 0, 1) , array(1, 1, 3) // fails ); }}
Testing exceptionsclass ExceptionTest extends PHPUnit_Framework_TestCase{ /** * @expectedException InvalidArgumentException */ public function testException() { throw new InvalidArgumentException(); }}
PHPUnit converts PHP errors, notices and warnings to exceptions, respectively:
- PHPUnit_Framework_Error- PHPUnit_Framework_Error_Notice- PHPUnit_Framework_Error_Warning
Testing performance
class PerformanceTest extends PHPUnit_Extensions_PerformanceTestCase{ public function testPerformance() { $this->setMaxRunningTime(2); // 2 seconds sleep(1); // 1 second }}
Testing output
class OutputTest extends PHPUnit_Extensions_OutputTestCase{ public function testExpectFooActualFoo() { $this->expectOutputString('foo'); print 'foo'; } }
PHP's Output Buffering feature is used to provide thefunctionality that is necessary for this
Advanced usage
- Code Coverage Analysis- Grouping of tests
@group- Mark tests incomplete- Mark tests skipped- Behavior-Driven development
stories- Agile documentation
--story, --tap, --testdox- Test doubles
Stubs and Mocks- Skeleton generation
- from test case to class- from class to test case
Documentation
Want to learn more?– PHPUnit pocket guide
www.phpunit.de/manual/ – Testing patterns
www.xunitpatterns.com
At the end you will receive:- A PHPUnit Cheatsheet for version 3.3- A Unit Testing best practices document- A Test-Driven-Development:
- Rhythm reference guide- Quick reference guid
SeleniumWeb application automated testing suite
Tricode Professional Serviceswww.tricode.nl
24-04-2009Sander van Beek
Selenium
Selenium is a suite of tools to automate web app testing across many platforms.
Selenium...• runs in many browsers and operating systems • can be controlled by many programming
languages and testing frameworks.
Selenium
Selenium suite has 3 main tools:• Selenium IDE
– Create tests
• Selenium RC (remote contol)– Run tests in different browsers
• Selenium Grid– Scale out, speed up
Selenium
Selenium IDE
• Firefox plug-in
• Record tests (demo)
• Export them to different formats– HTML– C#– Java– PHP– Ruby– Perl– Python
Selenium IDE
Commands:• Actions
open, click, refresh
• Accessors Get state and store it
• Assertions Verify state. Have 3 modes:
- assert (halt on error)- verify (generate warning on error)
- waitFor (wait for condition to become true)
Selenium RC
• Run distributed Selenium tests on one (remote) server
Selenium RC
• Supported browsers:– Firefox 2, 3– IE 7, 8– Safari 2, 3– Opera 8, 9– Others: partially
• Supported operating systems:– Windows– Linux– OS X,– Solaris– Others (as long as they run supported browsers)
Selenium Grid
• Selenium in parallel on multiple servers (or VM’s)
Selenium Grid
• http://saucelabs.com/: Selenium on EC2
• TestSwarm: http://ejohn.org/blog/javascript-testing-does-not-scale/
Questions
More info: http://seleniumhq.org/
Tricode Professional Serviceswww.tricode.nl
14-12-2008Sander van Beek
Phing An automated build system based on Apache ant
Phing
• PHing Is Not GNU make; it's a project build system based on Apache Ant. You can do anything with it that you could do with a traditional build system like GNU make, and its use of simple XML build files and extensible PHP "task" classes make it an easy-to-use and highly flexible build framework. Features include file transformations (e.g. token replacement, XSLT transformation, Smarty template transformations), file system operations, interactive build support, SQL execution, CVS operations, tools for creating PEAR packages, and much more.
Phing
• Automated build tool
• Types of automated builds:– On demand– Scheduled (nightly build)– Automated (continuous integration)
Phing
• Why ‘build’ PHP?– Improve product quality – Eliminate redundant tasks – Minimize "bad builds" – Eliminate dependencies on key personnel – Have history of builds and releases in order to
investigate issues– Save time and money - because of the reasons listed
above
Concepts
• Targets & taks
• The build file: build.xml– Project
• Target1– Task1– Task2
• Target2– Task1– Task2
• Target3– Task1– Task2
• Targets can be dependant on each other
• One default target (run when no task is specified on command line)
Task types
• Core– PhingCall, input, phpeval, condition, echo, exec
• File system– Copy, delete, move, mkdir, touch, available,
resolvePath
• Transformation– XsltTask, ReflexiveTask
• Packaging– pearPackageTask, zipTask, tarTask, dbdeploy,
ioncube
Task types
• Version control– svnCheckout, svnExport, svnUpdate, svnLastRevision
• Quality assurance– phpunitTask, zendCodeAnalyser, phpDocumentor,
phpCodesniffer, coverageXXX, phpLint
• Filters– Replacing, stripping comments/whitespaces, line
numbering, tidy and more
• Easy to add new tasks
Filesets
• Can be used in many tasks
<fileset dir="/etc" > <include name="httpd/**" /> <include name="php.ini" /> <exclude name=“doc/**" /></fileset>
• **/* <- Ant glob syntax
Property files
• Create environment specific builds
• E.g. build.properties, contains ‘key=value’ lines. E.g:database.user=upcpl
• Can be used to replace properties in source code - e.g. ${database.user}
<property file="build.properties" /> <reflexive> <fileset dir=“config"> <include file=“config.php”/> </fileset> <filterchain> <expandproperties /> </filterchain></reflexive>
Running phine
• Simply run ‘phing’ in the directory containing build.xmlOptionally specify a target as the first parameter
• Can also be run automated, e.g. from cruisecontrol or hudson
Top Related