Staying Close to Experts with Executable Specifications
-
Upload
vagif-abilov -
Category
Software
-
view
46 -
download
0
Transcript of Staying Close to Experts with Executable Specifications
![Page 1: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/1.jpg)
Vagif Abilov, Peder Søholt
Staying close to experts
with executable specifications
![Page 2: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/2.jpg)
Vagif Abilov, Peder Søholt
Staying close to experts
with executable specifications
![Page 3: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/3.jpg)
Case study: NRK player
Norwegian Broadcasting Corporation (NRK)
• On-demand TV/Radio media player
• Access to the whole program archive
• Copyright restrictions
• Rich metadata information
![Page 4: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/4.jpg)
NRK player
![Page 5: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/5.jpg)
NRK player
![Page 6: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/6.jpg)
Distributed domain knowledge
• End-user experience
• Program player
• Metadata database
• Media file distribution
• TV and radio programs are handled by different
subsystems
• Program index management
• Subtitle management
![Page 7: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/7.jpg)
System complexity
![Page 8: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/8.jpg)
Knowledge gathering challenges
• Experts are engaged in multiple projects
• Developers are divided into multiple groups and placed
in different locations
• Unified terminology is defined but not fully adopted
• External systems bring their own terms and standards
• Specifications are frequently revised
• Interactive sessions with experts may reveal additional
design constraints
![Page 9: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/9.jpg)
Program metadata
• Series -> Seasons -> Programs -> Indexes
• Metadata aggregate may contain
– Titles
– Descriptions
– Subjects
– Locations
– Contributors
• Original program metadata are usually not sufficient for
on-demand access, therefore part of the project scope
is extending metadata
![Page 10: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/10.jpg)
Obtaining metadata
• Metadata come from 4 different data sources
(programs/transmissions, indexes, subtitles, radio)
• Metadata sources belong to different subsystems
managed by different experts
• Protocols include SOAP and REST, both push and pull
methods
• Data transformation rules are established through a
series of interviews with experts and are revised with
each iteration
![Page 11: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/11.jpg)
Metadata validation workflow
Metadata
bank
Program
transmissions
Subtitles
TV program
metadata and
indexes
Radio program
metadata and
indexes
Given
Then
When When
![Page 12: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/12.jpg)
BDD framework selection
• Main development is on .NET platform
• Early consideration: from Fitness to StoryTeller
• Gherkin appeared to be a good communication
language choice
• Collected scenarios became a foundation for
executable specifications and acceptance tests
• Considered: Cuke4Nuke
• Selected: SpecFlow
![Page 13: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/13.jpg)
Example of the specification
Scenario: Update rights for a TV program
Given ODA database contains raw rights for program "NNFA20400099"
| VodPatternId | PublishStart | PublishEnd |
| 83223739100 | 2012-11-01 | 2012-12-01 |
And ODA database does not contain raw rights with pattern "832237391"
When MgxGranitt adapter service receives pattern with data
| ChangeType | AlternateId | Id | StartDateTime| EndDateTime |
| NewOrUpdate | NNFA20400099 | 832237391 | 2012-11-11 | 2012-12-11 |
Then MgxGranitt adapter service should return OK
And ODA database should contain rights for program "NNFA20400099"
| PublishStart | PublishEnd |
| 2012-11-01 | 2012-12-11 |
![Page 14: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/14.jpg)
Feature completion reports
![Page 15: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/15.jpg)
Extending SpecFlow
Validating data projections and anonymous types
![Page 16: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/16.jpg)
Using LINQ projections in steps
![Page 17: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/17.jpg)
SpecFlow.Assist extension methods
![Page 18: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/18.jpg)
Support for LINQ operations
• We may need to specify match type:
• Equivalence (same items in both collections, order is
insignificant);
• Equality (same items in the same order);
• Subset (one collection is a subset of another collection);
• Intersection (collections have common items).
![Page 19: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/19.jpg)
Lessons and discoveries
• Initial temptation to focus on end-to-end testing
• “Don’t use executable specifications as end-to-end
validations“ (Gojko Adzic)
• “Acceptance tests written for pure testing sake are
normally a sign that the code is badly designed” (Matt
Wynne)
• Even though Gherkin adds extra
development/maintenance cost, some developers
found convenient to use it as specification/document
tool among technical people
![Page 20: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/20.jpg)
NRK player – architecture layout
![Page 21: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/21.jpg)
User stories and criterias
• Product owner works together with interaction designer
and us developers to write User Stories with
acceptance criterias
– team work during sprint
– Criterias in Given, When, Then format
• People with strong opinions
• Report showing all specifications.
• Report showing current sprint specifications
![Page 22: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/22.jpg)
Gherkin and SpecFlow
• Write tests using SpecFlow
• Divided in two ways of writing specs
• Acceptance criteria tests from customer
• Specifications of how and where we have solved the criteria, this might
be technical specs
![Page 23: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/23.jpg)
SpecFlow and SpecRun
• Running tests using SpecRun
• Can run parallel tests
• Creating reports using SpecRun
• Unfortunately no SpecLog usage yet
![Page 24: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/24.jpg)
SpecRun report
![Page 25: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/25.jpg)
SpecRun report
![Page 26: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/26.jpg)
SpecRun report
• fdsa
![Page 27: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/27.jpg)
SpecRun report
• fdsa
![Page 28: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/28.jpg)
SpecFlow tips
• Use tags
• @acceptance
• @deleporter
• @webdriver
• @sprint35
![Page 29: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/29.jpg)
SpecFlow tips
![Page 30: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/30.jpg)
Web testing - RazorGenerator
• Generates the views in the MVC model
• Test the actual HTML output
• Fast
![Page 31: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/31.jpg)
Web testing - Selenium
• Web testing with Selenium / Webdriver
• if you need to test specific stuff on the web
• Test javascript/AJAX functionality in the
• Slow, but tests actual functionality in the webbrowser
![Page 32: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/32.jpg)
Web testing - Selenium
• Use SauceLabs
• Selenium/Webdriver testing in the cloud
• Can run parallel tests
• Lots of browsers and OS to choose between
• Can test closed in house servers using SSH tunneling
![Page 33: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/33.jpg)
Web testing tips
• Use abstraction layer, easier maintenance
• Ex. page-objects.
• Do not test specifically a tag with hardcore xpath or css
• Ex.Test that HTML-body has the text you are looking for
• Do not use for example xpath:
• /html/body/div/div[2]/div/div/section/article/hgroup/h1
• Do not use for example CSS-selector:
• html.gecko body#program.a-showprogramsync div#sfWrap
div#sfMain div#main div.box section#programMetaData.container
article#episode.span-10 hgroup h1
![Page 34: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/34.jpg)
Deleporter
• Cross-Process Code Injection for ASP.NET
• For mocking data, when running webtests on a different
machine than where you run your tests
• Developed by Steven Sanderson
![Page 35: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/35.jpg)
Deleporter
![Page 36: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/36.jpg)
Deleporter
![Page 37: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/37.jpg)
Deleporter
![Page 38: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/38.jpg)
Deleporter
![Page 39: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/39.jpg)
Deleporter
• For mocking data, when running webtests on different
machine than where you run your tests
• SECURITY RISK - DO NOT DEPLOY TO
PRODUCTION!
![Page 40: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/40.jpg)
Cucumber tests
![Page 41: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/41.jpg)
Cucumber tests
![Page 42: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/42.jpg)
Conclusions
• Work together with your customer to create good specs
• Please include a Interaction designer and a developer
• Use tags in SpecFlow
• Generate HTML from views instead of web tests –
much faster
• When using web tests, use Deleporter for mocking the
server
![Page 43: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/43.jpg)
Conclusions
• Executable specifications have proven their efficiency
when gathering and maintaining expert knowledge
• Wide adoption requires enthusiasts and patience
• Use of open source software enables adjusting it for
team’s specific needs
• Even within the same organization practice and
interpretation of terms may differ significantly
![Page 44: Staying Close to Experts with Executable Specifications](https://reader034.fdocuments.in/reader034/viewer/2022042716/55a2be651a28ab54268b4682/html5/thumbnails/44.jpg)
Thank you!
• Vagif Abilov
• @ooobjects
• http://bloggingabout.net/blogs/vagif/
• Peder Søholt
• @psoholt
• http://highfivedrivendevelopment.org