Automated Testing with Umbraco

… or any other website you build

Who am I?

• Lars-Erik Aabech

• Professional web developer since 1998

• Working at MarkedsPartner AS since 2002

• Umbracian since 2012

• Certified V5 developer (!!!)

– Also V7 master

• Umbraco Core MVP x 3

• Avid fan of SOLID principles and Automated Testing

WHY AREN’T WE UNIT TESTING?

We don’t have the time!

Code Build(Re)Start

WebserverStart browser

Wait for runtime compiles

Click aroundVerify

Code Build(Re)Start

WebserverStart browser

Wait for runtime compiles

Click aroundVerifyDebug

AUTOMATEDTESTS!

Code Build

Run TestsAutomagically

verify

Code Build Run Tests

Automagically verify

Debug

We don’t have the time to learn how to unit-

test!

EFFO

RT

FEATURES / TIME

Tests No tests

We don’t have the time to learn how to do our

job properly!

We don’t want to learn how to do our job

properly!

HOW?

Example: Recently Used List

• A nice little Kata to train on

• Last in, First out

• If it were already there, it’s moved to the top

1990’s: «Drivers»

• Throw-away console apps for proof of concept

• Throw-away web-pages with buttons that invoke functions

• Throw-away windows with buttons that invoke functions

• Big-ass logfiles or console outputs logging shitloads of stuff to textboxes or output

1999: Self Testing Code

“Self-Testing Code is the name I used in Refactoring to refer to the practice of writing comprehensive automated tests in conjunction with the functional software. When done well this allows you to invoke a single command that executes the tests - and you are confident that these tests will illuminate any bugs hiding in your code”

- Martin Fowler, https://martinfowler.com/bliki/SelfTestingCode.html

2K: Automated Unit Tests

• Kent Beck publishes Test Driven Development: By Example (2002)

• A mindset for development and driving design

• The tests verify their own results

– AKA Assert.ThisAndThat()

• A set of classes and functions designed to run in unordered sequences

• All the «drivers» automated without apps or buttons to click

• Comprehensive list of results for each feature

• No need for logging and manual verification

BENEFITS?

SPEED!(Fast feedback)

SELF ESTEEM!(I know my code worked 2 minutes ago)

DOCUMENTATION!(Examples)

REFACTORING!(Refactoring without tests is just moving shit around

– Corey Haines)

What’s a unit?

A function

A class

A set of classes

A webpage

An application

A unit is whatever your team decides is a unit

- Martin Fowler

The simplest units

• A function using, manipulating and returning simple values

• A class manipulating its own inner state, exposing or returning some value(s)

POCOPlain Old C# Object

HERE BE DRAGONS!

Mr. AssertTooMuch

• Asserting several things in a tests is a mistake

• It’s actually a breach of the Single Responsibility Principle

• If the first assert fails, you won’t know about any successive failures

• It’s a lot of code to write

My name is Mr. AssertTooMuch

Well you’d better cut down a bit, then.

Meet ApprovalTests

Meet ApprovalTests

Method Purpose

Verify Text / Whatever

VerifyAll Lists of whatever

VerifyBinaryFile Unreadable, but approvable ☺

VerifyHtml Formats HTML nicely

VerifyJson Formats and compares JSON nicely

VerifyPdfFile Like binary, just more PDF’y

VerifyXml Formats XML nicely

Extendable! Whatever you want, however you want

Meet ApprovalTests

#H5YR@LlewellynFalco

Then there were two...

Then there were two...

• Two or more classes interacting, perhaps across modules

• The one doesn’t work without the other

• «System Under Test» and «Collaborators»

• Testing two or more artifacts together is per definition an integration test

The problems

IO Databases

Webservers Umbraco ☺

NON-POCO:’(

DEPENDENCY INVERSION!

Test Doubles

• AKA. Fakes, Mocks, Stubs

• Replaces the collaborating class with a dumb object satisfying the tested class’ needs

• Removes dependencies on IO, Databases, Webservers and others

• Either handwritten (fakes) or using a library that creates such types at runtime. (dynamic stubs/mocks)

HERE BE DRAGONS!

Mr. StubTooMuch

• One stub can be a lot of code to set up

• Two stubs are even more

• Every property and method call have to be declared

• Can become completely unmaintainable

• You have no idea whether the «real» collaborator will do the same thing

My name is Mr. StubTooMuch

Well you’d better cut down a bit, then.

The dragons are many!

• Using test doubles may require a lot of extra code

– Writing a fake may be the quickest way

• Stubbing without effective means of setting up data creates heaps of unreadable and unmaintainable code

– Stubbing is useful when the output can be deserialized instead of fleshed out in the test

• Mocks are not stubs!

– Mocking is only meant to verify external calls or calls to IO

• You might end up hating unit-testing.

INTEGRATE!

Go to the database if you want

• Make sure to create a fresh one for each test run

• Seed it with data that all tests expect

• Start a transaction and roll it back after each test

• Incidentally this is dead easy to do with Entity Framework + «Code First»

RazorGenerator

• Design-time generates «codebehind» for views

• Lets you render views without a webserver

• Combined with ApprovalTests is a definite win for verifying GUI

On acceptance tests

• NUnit isn’t the only framework

• Scenarios can be expressed in Wallet-Speak with a language called «Gherkin»

• Scenarios written in Gherkin are ran with «Cucumber»

• Regular expressions match yourtext to your code

• In .net «Cucumber» is called «SpecFlow»

• Properly layered they can be ran against your domain code, or through the GUI with Selenium on a whim.

The test pyramid and the types of automated tests

Acceptance(a few)

Integration (quite a few)

Unit Tests(a lot)

The ice cream cone of automated tests

Acceptance(a lot)

Integration (quite a few)

Unit Tests(a few)

What about legacy software without tests

• Start adding tests when making new features

• Introduce abstractions by «extract interface» to introduce «seams»

• Change private, protected and internal to public (!!!)

• Read «Working effectively within legacy code» by Michael Feathers

ENOUGH, LARS!GET TO

UMBRACO ALREADY!

EXPLOIT THE CORE TESTS!

Core Tests and Internals

• The core tests already stub everything you need to run Umbraco in a test context

• Available as a community build on Nuget (Thanks Christian Thillemann/@kedde!)

• 20+ base classes to inherit depending on how much you need support for

• All the annoying internal things can be exposed by introducing an «adapter assembly» called «Umbraco.UnitTesting.Adapter»

• Can still require a bit too much code

UmbracoSupport

• General class to include instead of inheriting Umbraco’s base tests.

– Actually inherits BaseRoutingTest

• All the stubs and core integrations you’ll need for Umbraco 7

• Extensive samples for most use-cases athttps://github.com/lars-erik/umbraco-unit-testing-samples

• (You still have to set things up)

Composition vs. Inheritance

• Inheriting Umbraco’s tests block your own inheritance

• Support classes can be brought in as you need them

• Typical support classes

– UmbracoSupport

– DatabaseSupport

– ExternalServiceSupport

• The support classes contains the stubbing so you don’t need to repeat it

• Simple API-calls on the support classes to set up stuff using POCOs

UMBRACO 8?

DEPENDENCY INVERSION

EVERYWHERE!\o/

EVERYTHING CAN BE

STUBBED! \o/

CurrentUmbracoContext.

NUCACHE AND THE CURSE OF

INTERNALS

UmbracoSupport for V8 TODOs

• All the base classes are still there

• We can still use XML, but there’s nowhere to copy from

• DI is all around now

• Yet Current is used all over

• Everything can be stubbed!

• We need test APIs to help reduce friction

Come join the discussion

• We have several paths to our goal

• There are fakes ready, but internal in the test project

• There are real classes ready to integrate, and they might just as well be public…?

• Proposal: Publicize everything, but tag candidates for breaking changes such that the Roslyn compiler throws an error when they’re used. Add an attribute in your test assembly to override this as an «EULA» for using volatile classes.

• Should published content be at the forefront of our testing?

• https://github.com/umbraco/Umbraco-CMS/pull/5325

Let’s find the sweet spot betweenintegration and stubbing to make us

as fast as possible when we buildcomplex sites.

QUESTIONS?