Specification by example with specflow 110217

Specification by examplewith SpecFlow

Marcus HammarbergAvega Group

www.marcusoft.net

söndag den 20 februari 2011 (v.)

http://www.stochasticgeometry.ie/wp-content/uploads/2009/10/snakeoil.jpg?w=257


Executable specifications - that verifies that the system is behaving as expected.

You want it - i can give it to you!

This presentation is about Specification by example with SpecFlow.

As a last teaser, a quote by Dan North.Hows that’s for something to live up to? :)



http://www.stochasticgeometry.ie/wp-content/uploads/2009/10/snakeoil.jpg?w=257 http://www.unit5.org/debordc/MPj04071740000[1].jpg








http://www.unit5.org/debordc/MPj04071740000%5B1%5D.jpg


http://www.stochasticgeometry.ie/wp-content/uploads/2009/10/snakeoil.jpg?w=257 http://www.unit5.org/debordc/MPj04071740000[1].jpg

“It’s not a silver bullet – but it seems a good start”

Dan North











My name is Marcus Hammarberg


I work for Avega Group as an agile coach and consultant


I have one wife


two hobbies


and three kids


I have been working as a programmer


since 1998 - mainly with Microsoft products


During the last 6 years I have been more and more interested in how you can work effective together


with agile methods - such as scrum

Who are you?


BDD


What’s this BDD thing?

Wasn’t it TDD? Or was it DDD? or xDD?

Here’s a short background

TDD


TDD (Test driven development) is a way to write code where you start from a test describing the functionality you’re about to write. The test drives the production code from the developer.

There’s a lot of pros from that way of working- you think about the behaviour of the code before you write it- you can take small steps through complex problems- you only write the code that is really needed- the tests describes the functionality of the system

- and, as a funny side-effect, you get tests for all the code you’re writing.

But there’s some problems as well:- where to start- where to stop- you easily end up writing test that describes HOW the production code is implemented- TDD is lowlevel inside-out. You start at a single method or class. How did you get there?

BehaviourDrivenDesign


All these problems can easily get to you when you start TDD’ing.

This was addressed in a blogpost by Dan North (http://blog.dannorth.net/introducing-bdd/) in which he coined the term BDD - Behavior Driven Design.

With BDD the behavior of the feature we’re about to implement is the focus.

Also the language we’re using is very important in BDD. The whole idea is to use a language that everyone concerned can read and understand the langauge. So that we can talk about WHAT the feature does - separated from HOW.

Outside-in development

http://www.adaptivepath.com/blog/wp-content/uploads/2009/02/oi_model_inward.jpg


On of the problems of TDD, I took up, was that you didn’t know where to start. Or stop. That has to do with you working inside-out.

BDD gives you the possibility to work from the outside-in.

You start with failing acceptance test that runs end-to-end. When that is implemented - this feature is done!

This is called ATDD by some - Acceptance test driven development



What’s all these DD’s?


This is my humble understanding. Today- TDD - helps you write better code by writing a test before the production code

- ATDD - start from a failing end-to-end acceptance test. Jenitta Andrea says this: ATDD is to describe a functional requirements of a story as concrete examples or expectations before development is started

- STDD - Story test driven development - don’t know much about this. Seems to be very like ATDD

BDD - focus on the behavior of the feature. Liz Keogh : BDD’s focus is to discover the things we didn’t know, especially in the contexts that the scenarios or examples is run in

Specification by example - a way to specify something by giving concrete examples on how they should work.


• TDD








• TDD

• ATDD








• TDD

• ATDD

• STDD








• TDD

• ATDD

• STDD

• BDD








• TDD

• ATDD

• STDD

• BDD

• Specification by example








Or you can use this great intro and skip the buzz-words altogether. From Gojko Adzic

This is a great sum up that I like by Janet Gregory:

Where we choose to call it BDD or ATDD or Specification by Example, we want the same result – a shared common understanding of what is to be built to try to build the ‘thing’ right the first time.

We know it never will be, but the less rework, the better.

Whether we choose to call it BDD or ATDD or Specification by Example, we want the same

result – a shared common understanding of what is to be built

to try to build the ‘thing’ right the first time.


Janet Gregory


Or you can use this great intro and skip the buzz-words altogether. From Gojko Adzic

This is a great sum up that I like by Janet Gregory:

Where we choose to call it BDD or ATDD or Specification by Example, we want the same result – a shared common understanding of what is to be built to try to build the ‘thing’ right the first time.


http://janetgregory.blogspot.com/

http://janetgregory.blogspot.com/

Theory


Some theory on the background of Specfication by example

User stories


User stories is a Extreme Programming way to keep requirement small and manageable.

It’s requirements in a sentence. Often using a template like this:

In order to [benefit]As a [role]I want to [feature]

That first part is seldom done in any use cases that i’ve read. And that’s the whole reason for the function to exist.

But this is, even though it’s short, still to wide and open for interpretations. Therefore user stories are often called “a placeholder for a further conversation”

User stories









And that discussion should lead to ...

In order to better understand each other one powerful technique is to give concrete example.

Just imaging how hard it would be to learn how to tie a Windsor-tie with just the theoretical text...

It’s much easier with a couple of pictures to guide us


So a good way to get this in a user story is to give a couple of examples on how it could be played out.

They could be written with the template; Given / When / Then.

Given [initial context]When [event]Then [an outcome]




Scenario 1: Account is in creditGiven the account is in credit And the card is valid And the dispenser contains cashWhen the customer requests cashThen ensure the account is debited And ensure cash is dispensed And ensure the card is returned




Scenario 2: Account is overdrawn past the overdraft limitGiven the account is overdrawn And the card is validWhen the customer requests cashThen ensure a rejection message is displayed And ensure cash is not dispensed And ensure the card is returned





This looks a lot like a test-case, right?Or maybe a detailed requirements document?And wouldn’t that be nice to code after?

There’s other pro’s too: - this foster the use of the same language for requirements, test and even maybe code - we can easily discuss the scenarios, add and remove stuff as we learn more. Before the system is implemented - it’s easy to see which functionality that is included (in a release for example) - and finally it’s VAD separated from HOW

If you now only didn’t have to run the test cases ...

Scenario 1: Account is in creditGiven the account is in credit And the card is valid And the dispenser contains cashWhen the customer requests cashThen ensure the account is debited And ensure cash is dispensed And ensure the card is returned





Wrap-up

Executable specifications that verifies that the system behaves as expected, readable by all in the team, and around it


This was the promise in the outset - is it kept?

It’s not in the tool

http://www.barbellsandbacon.com/wp-content/uploads/2010/02/mlss_mario-hammer.jpg


It’s very important to remember that the main benefit is not in the tool (which we will talk about from now on).

The benefit is that you get together and write down your common, shared knowledge in a well known format. In one place.

That’s why anyone would dare call this a Silver Bullet



Tools


When we have these scenarios in place it would be wonderful if we could use some kind of tool to verify that the system actually behaves as specified.

And of course you could - this presentation would not be very meaningful otherwise.

This was the idea of Cucumber that first was created in the Ruby-community. Cucumber can actually execute scenarios written in the Given/When/Then-syntax. This syntax is called Gherkin.

For .NET there’s a lot of BDD frameworks that emulates Cucumber, but in my opinion the best one is SpecFlow. You can think of SpecFlow as a port of Cucumber for .NET.

Tools

http://buyorganicvegetable.com/images/Cucumber.jpg


When we have these scenarios in place it would be wonderful if we could use some kind of tool to verify that the system actually behaves as specified.

And of course you could - this presentation would not be very meaningful otherwise.

This was the idea of Cucumber that first was created in the Ruby-community. Cucumber can actually execute scenarios written in the Given/When/Then-syntax. This syntax is called Gherkin.

For .NET there’s a lot of BDD frameworks that emulates Cucumber, but in my opinion the best one is SpecFlow. You can think of SpecFlow as a port of Cucumber for .NET.



How does this work?


But how can text be executable - it’s so open, you can write almost anything?

It’s pretty simple really:- SpecFlow creates an automated test for each scenario, with the test runner of your choice- Each row in the scenario (called a step) will then call a method, that you supply, called step definition. The method defines what the step means; How do you define “Given I am on the Activity list”

It might be easier with an example (nudge, nudge :)):

- For the scenario above - SpecFlow will create a test called NavigateToCreateNewActivity- in that test SpecFlow will look and call methods with attributes as the steps, shown in the example

How does this work?

[Given("I am on the Activity list")]public void ThisNameIsNotImportant()

[When("I click the Create New link")]public void SpecFlowSimplyUsesTheAttribute()

[Then("I should be on the Create Activity page")]public void AndCouldntCareLessOfYourMethodName()


But how can text be executable - it’s so open, you can write almost anything?

It’s pretty simple really:- SpecFlow creates an automated test for each scenario, with the test runner of your choice- Each row in the scenario (called a step) will then call a method, that you supply, called step definition. The method defines what the step means; How do you define “Given I am on the Activity list”

It might be easier with an example (nudge, nudge :)):

- For the scenario above - SpecFlow will create a test called NavigateToCreateNewActivity- in that test SpecFlow will look and call methods with attributes as the steps, shown in the example

No really, how does it WORK?

http://www.specflow.org/getfile/a67f8885-4900-4803-a972-5f3c57364097/artifacts.aspx


Here’s the technical part:1 - You write your scenarios in a .feature-file. Every time it’s saved SpecFlow will generate a test-class, that contains a test for each scenario

2 - The generated code (that you never care about that code, or rather don’t touch it) it’s just there to glue together your scenario with your step definitions.

3 - When the tests are run by the test runner (just as ordinary unit tests), it will call you step definitions in the order they are written in the scenario

4 - And your step definitions then call your application or domain.



Step definitions

http://www.treehugger.com/green%20footsteps%20generic.jpg


So the trick lies in the step definitions.

They are realizing the “promise” of the steps in the scenario.

So when we write a step definition for “When I click the Create New Link” we need to find that link on the web page (I guess) and then click it.



Step definitions

Scenario: Navigate to Create new Activity Given I am on the Activity list When I click the Create New link Then I should be on the Create Activity page



So the trick lies in the step definitions.

They are realizing the “promise” of the steps in the scenario.

So when we write a step definition for “When I click the Create New Link” we need to find that link on the web page (I guess) and then click it.



Automation

http://www.cultofmac.com/wordpress/wp-content/uploads/2010/09/automator-icon.png


That’s called automation. To write a program that automate the running of another program.

There’s a lot of framework that assists in this. But that’s not the purpose of this presentation.

I would recommend Watin (http://watin.sourceforge.net/) for web applications and White (http://white.codeplex.com/) for Windows based applications.



Gherkin


For a Swede Cucumber and Gherkins are more or less the same...

But apparently that’s not true - these are Gherkins.

And Gherkin is also the name of the language that scenarios are written in, invented by Alsak Hellesoy.

We will now examine it a bit closer.


The language is not big. This is the whole language.

And that means it can be translated. To Swedish. Arabic. Bulgarian.

And some more ... dubious languages too.

but it gives us possibilities to write our scenarios in a suitable language.

Feature

http://www.lushtshirts.co.uk/images/products/bug-feature.jpg


Features and scenarios are written in .feature. It contains an feature description and one or more scenarios.

It’s a plain text file and it checked in with and versioned together with the code of the project.

The feature is the description of the functionality. It’s plain text and doesn’t affect how the scenario is executed or interpreted. It’s very important though - in order to understand the scenarios.

Every feature has a title and is written after the keyword Feature:



Scenario

http://images2.fanpop.com/images/photos/7000000/Friends-33-friends-7046166-1280-1024.jpg


A scenario also has a title and one or more steps

You can use the Friends-rule to set a good name of the scenario. You know, the episodes of Friends was called “The one where Ross and Rachel became a couple”, about half of the episodes was called that if I remember correctly :)



GIVENWHENTHEN


The scenarios follows the template Given / When / Then with the following meaning:- Given - in this step you setup the system in the desired state. - When - this is the actual action the scenario is about- Then - asserts that the right things happend

The scenario is executed top down. And the Given / When / Then keyword can be repeated several times in the scenario, if you like. Not recommended but possible.

And speaking of not recommended - you can write the Given / When / Thens in any order you want. But then you lose any pros of the structure in the first place.

Scenario: Simple GWTGiven the intial state of the application is RunningWhen I ask what the application state isThen I should see Running as the answer





But? And?


There’s two more keywords AND and BUT to connect several steps with each other.

AND and BUT refers back to the word above it. They are used to get a better flow in the language.

Try not do use AND/BUT in the When-step. Let that be a single action.

Expressions


Sometime the steps are very similar to other existing steps.

And it would be horrible if we needed to write every step definition explicit.

Therefore Regular expressions can be used to have some part of the steps as variables.

That will be converted into parameters to our step definition methods. Please note that the order of the regular expressions are important.

If you, like me, don’t like RegExps you’ll be happy to known that these two will get you very far.

Also, since SpecFlow 1.4, you get intellisense for the step definitions you already written

Expressions

When I write 'My test string' in the textbox 'Test textbox'








Expressions


[When(@"I write '(.*)' in the textbox '(.*)'")]public void WriteInTextBox(string textToWrite, string wellKnownName)








Expressions


[When(@"I write '(.*)' in the textbox '(.*)'")]public void WriteInTextBox(string textToWrite, string wellKnownName)

(.*) - en sträng => string(\d+) - ett heltal => integer








Background


With the keyword background you can run some steps before each scenario in the feature.

It’s just like a ordinary scenario, it’s just run before the scenarios.

<CLICK>In this example we set up the system under test in a common state for all the scenarios

Hooks


From Cucumber SpecFlow borrows the concept of hooks.

Hooks are events that is fired in the following order.

And you can “hook in” code here to do useful things: - Restore a database - Dump a log on error - Init variables before steps - etc

Tags


Hooks are cool but they become really powerful combined with Tags.

Tags are simple small markers that can be written in your feature.

Only one tag is supported by SpecFlow out of the box; @ignore that simply ignores the scenario. Great for stuff that isn’t ready yet.

A tag becomes a test category and can be used to categorize your tests (acceptance test, dailes, slow running etc)

But you can use tag to overload the hooks. So that a certain hook only is run when the scenario or feature has a certain tag.

@ignore








You can use table to manage multiple data. The data will be passed to you step definition as a SpecFlow Table that has values for columns and rows. From that table you can read a data.

[Given("I have the following persons")]public void IHaveTheFollowingPersons (Table personsTable)


You can use table to manage multiple data. The data will be passed to you step definition as a SpecFlow Table that has values for columns and rows. From that table you can read a data.

SpecFlow.Assist


Working with tables can be quite messy. But in this namespace (SpecFlow.Assist) there’s some helpers to help us get around that.

Check these examples out: - https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/Specs/05TablesAndAssist - https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/Specs/06CompareToAssist

Scenario outline


By using a scenario outline you could reuse the same scenario with several different data.

In this way you can easily write several variants (permutations) of test data for the same scenario.

This example will create 5 test that performs the calculation. How this is done depends on the testrunner you’re using; NUnit creates TestCases that sends data to the unit test while MsTest creates several tests.

Langauge


You can change the language in your feature by using the #language : sv (for Swedish).

But probably you want to do that for all your features, in the .config-file of the specification project.

ScenarioContext.Current


ScenarioContext is a small dictionary that is used to send data between your steps. It’s much better than to use private variables in your step classes since that hinders step reuse etc.

So better is to group your step definitions per logical unit; controller, page, function

See https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/Specs/031ScenarioContext

AvegaCoach Activity Log


I have developed a small example site that shows how these techniques can be used.

It’s a small site to keep track of the activities that persons in my group, AvegaCoach, undertake.

This is the domain model and I’ve scaffolded the whole site.

I have written two sets of specifications- against the controller level (https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/SpecsAgainstController)- against the web pages (https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/SpecsAgainstGUI)

to show how they differ.

On with the code!

http://burningbird.net/photos/thumb-victwait.jpg


On to the code!



Learn more


Learn more• http://blog.dannorth.net/introducing-bdd/



• http://www.specflow.org/




• https://github.com/techtalk/SpecFlow





• http://github.com/techtalk/SpecFlow-Examples






• http://github.com/aslakhellesoy/cucumber/wiki


http://github.com/aslakhellesoy/cucumber/wiki/gherkin







• http://www.marcusoft.net/search/label/SpecFlow




http://www.marcusoft.net/search/label/SpecFlow











• http://www.marcusoft.net/search/label/SpecFlow

• https://github.com/marcushammarberg/ProgressiveNetDemos










And even more


[email protected]


Thanks for listening

http://www.marcusoft.net





The end

Specification by example with specflow 110217

Technology

Transcript of Specification by example with specflow 110217