BDD USING SPECFLOW (CUCUMBER IN .NET)files.meetup.com/2625872/PowerpntBDD(Behavior Driven...
Transcript of BDD USING SPECFLOW (CUCUMBER IN .NET)files.meetup.com/2625872/PowerpntBDD(Behavior Driven...
PROBLEM
• ACCORDING TO A NUMBER OF STUDIES, AROUND HALF OF ALL SOFTWARE PROJECTS FAIL TO
DELIVER IN SOME SIGNIFICANT WAY.
• THE 2011 EDITION OF THE STANDISH GROUP’S ANNUAL CHAOS REPORT FOUND THAT 42%
OF PROJECTS WERE DELIVERED LATE, RAN OVER BUDGET, OR FAILED TO DELIVER ALL OF THE
REQUESTED FEATURES, AND 21% OF PROJECTS WERE CANCELLED ENTIRELY.
• SCOTT AMBLER’S ANNUAL SURVEY ON IT PROJECT SUCCESS RATES USES A MORE FLEXIBLE
DEFINITION OF SUCCESS, BUT STILL FOUND A 30–50% FAILURE RATE, DEPENDING ON THE
METHODOLOGIES USED. THIS CORRESPONDS TO BILLIONS OF DOLLARS IN WASTED EFFORT,
WRITING SOFTWARE THAT ULTIMATELY WON’T BE USED OR THAT DOESN’T SOLVE THE
BUSINESS PROBLEM IT WAS INTENDED TO SOLVE. - BDD IN ACTION
BEHAVIOR-DRIVEN DEVELOPMENT (BDD) IS A SET OF SOFTWARE ENGINEERING PRACTICES DESIGNED TO HELP TEAMS BUILD AND DELIVER MORE VALUABLE, HIGHER QUALITY SOFTWARE FASTER.
IT DRAWS ON AGILE AND LEAN PRACTICES INCLUDING, IN PARTICULAR, TEST-DRIVEN DEVELOPMENT (TDD) AND DOMAIN-DRIVEN DESIGN (DDD).
BUT MOST IMPORTANTLY, BDD PROVIDES A COMMON LANGUAGE BASED ON SIMPLE, STRUCTURED SENTENCES EXPRESSED IN ENGLISH (OR IN THE NATIVE LANGUAGE OF THE STAKEHOLDERS) THAT FACILITATE COMMUNICATION BETWEEN PROJECT TEAM MEMBERS AND BUSINESS STAKEHOLDERS.
KEY BENEFITS
• LIVING DOCUMENTATION – DOCUMENTATION CHANGES AS BEHAVIOR CHANGES AND IS IN
SOURCE CONTROL
• *COLLABORATION (3 AMIGOS QA/BA/DEV)
• EASY FOR NEW TEAM MEMBERS TO LEARN THE SYSTEM
• REAL TIME PROGRESS REPORTING – FEATURES ARE COMPLETE WHEN TESTS TURN GREEN
• END OF SPRINT DEMOS – MORE PRODUCTIVE AS WE CAN SHOW EXACTLY WHAT SYSTEM IS
DOING
• FASTER RELEASE CYCLE – TIES INTO CONTINUOUS INTEGRATION ENVIRONMENT AND COVERS
REGRESSION SCENARIOS
SCENARIO
•GIVEN DESCRIBES THE PRECONDITIONS FOR THE
SCENARIO AND PREPARES THE TEST ENVIRONMENT.
•WHEN DESCRIBES THE ACTION UNDER TEST.
• THEN DESCRIBES THE EXPECTED OUTCOMES.
GIVEN
THE PURPOSE OF GIVENS IS TO PUT THE SYSTEM IN A KNOWN STATE BEFORE
THE USER (OR EXTERNAL SYSTEM) STARTS INTERACTING WITH THE SYSTEM (IN
THE WHEN STEPS). AVOID TALKING ABOUT USER INTERACTION IN GIVENS. IF
YOU WERE CREATING USE CASES, GIVENS WOULD BE YOUR PRECONDITIONS.
WHEN
THE PURPOSE OF WHEN STEPS IS TO DESCRIBE THE KEY ACTION THE USER PERFORMS (OR,
USING ROBERT C. MARTIN’S METAPHOR, THE STATE TRANSITION).
THEN
THE PURPOSE OF THEN STEPS IS TO OBSERVE OUTCOMES. THE OBSERVATIONS SHOULD BE
RELATED TO THE BUSINESS VALUE/BENEFIT IN YOUR FEATURE DESCRIPTION. THE OBSERVATIONS
SHOULD ALSO BE ON SOME KIND OF OUTPUT – THAT IS SOMETHING THAT COMES OUT OF THE
SYSTEM (REPORT, USER INTERFACE, MESSAGE) AND NOT SOMETHING THAT IS DEEPLY BURIED
INSIDE IT (THAT HAS NO BUSINESS VALUE).
AND, BUT
THE AND AND BUT KEYWORDS CAN BE USED TO JOIN SEVERAL GIVEN, WHEN, OR THEN STEPS
TOGETHER IN A MORE READABLE WAY:
GIVEN I HAVE A CURRENT ACCOUNT WITH $1000
AND I HAVE A SAVINGS ACCOUNT WITH $2000
EXAMPLE FEATURE/SCENARIO
FEATURE: SCORE CALCULATION
IN ORDER TO KNOW MY PERFORMANCE
AS A PLAYER
I WANT THE SYSTEM TO CALCULATE MY TOTAL SCORE
SCENARIO: GUTTER GAME
GIVEN A NEW BOWLING GAME
WHEN ALL OF MY BALLS ARE LANDING IN THE GUTTER
THEN MY TOTAL SCORE SHOULD BE 0
CURRENT STATE OF AUTOMATION AT NORDSTROM
EXECUTABLE SPECIFICATIONS
• SPECFLOW AND SOAP SERVICE LAYER AUTOMATION FRAMEWORK
• SPECFLOW AND SELENIUM WEBDRIVER AUTOMATION FRAMEWORK
• SPECFLOW AND RESTSHARP/REST SERVICE LAYER AUTOMATION FRAMEWORK
LOW LEVEL EXECUTABLE SPECIFICATIONS
• SPECFLOW UNIT TESTS ARE CREATED/MAINTAINED BY DEVELOPERS BUT REVIEWED BY 3
AMIGOS
SCENARIO OUTLINE
SCENARIO OUTLINE: EARNING INTEREST
GIVEN I HAVE AN ACCOUNT OF TYPE <ACCOUNT-TYPE> WITH A BALANCE OF <INITIAL-BALANCE>
WHEN THE MONTHLY INTEREST IS CALCULATED
THEN I SHOULD HAVE EARNED AT AN ANNUAL INTEREST RATE OF <INTEREST-RATE>
AND I SHOULD HAVE A NEW BALANCE OF <NEW-BALANCE>
EXAMPLES:
| INITIAL-BALANCE | ACCOUNT-TYPE | INTEREST-RATE | NEW-BALANCE
| 10000 | CURRENT | 1 | 10008.33
| 10000 | SAVINGS | 3 | 10025
| 10000 | SUPERSAVER | 5 | 10041.67
EMBEDDED TABLES
SCENARIO: TRANSFERRING MONEY BETWEEN ACCOUNTS WITHIN THE BANK
GIVEN I HAVE THE FOLLOWING ACCOUNTS:
| ACCOUNT | BALANCE |
| CURRENT | 1000 |
| SAVINGS | 2000 |
WHEN I TRANSFER 500.00 FROM CURRENT TO SAVINGS
THEN MY ACCOUNTS SHOULD LOOK LIKE THIS:
| ACCOUNT | BALANCE |
| CURRENT | 500 |
| SAVINGS | 2500 |
BACKGROUND
BACKGROUND:
GIVEN I HAVE A "SALE" TRANSACTION TYPE
AND I HAVE A "POS" TRANSACTION DATA SOURCE
AND I HAVE A "USD" CURRENCY CODE
AND I HAVE A VALID STORE NUMBER
HOOKS
• [BEFORETESTRUN]/[AFTERTESTRUN]
• [BEFOREFEATURE]/ [AFTERFEATURE]
• [BEFORESCENARIO] OR [BEFORE]/ [AFTERSCENARIO] OR [AFTER]
• [BEFORESCENARIOBLOCK]/[AFTERSCENARIOBLOCK]
• [BEFORESTEP]/[AFTERSTEP]
SHARING BETWEEN STEPS
•CONTEXT INJECTION
•FEATURECONTEXT
•SCENARIOCONTEXT
SPECRUN
• (HTML) REPORT AS PRIMARY OUTPUT
• FAST INTEGRATION TEST EXECUTION BY PARALLELIZATION
• ADVANCED EXECUTION METRICS: EXECUTION TIME, MEMORY USAGE, MEMORY LEAKING, BENCHMARKING
• DETECTION OF “RANDOM FAILURES”
• TEST VARIATIONS (E.G. RUNNING THE SAME SUITE FOR IE AND FIREFOX)
• SUPPORT FOR APPLYING DIFFERENT TEST CONFIGURATIONS
• PROVIDE FAST FEEDBACK ABOUT FAILURES (PRIORITIZE FAILING TESTS FIRST, STOP AFTER A NUMBER OF
FAILURES, ETC.)
PICKLES
*PICKLES* CAN PRODUCE OUTPUT IN DIFFERENT FORMATS:
• STATIC HTML: A SET OF HTML FILES WITH MINIMAL JAVASCRIPT
• DYNAMIC HTML: A JAVASCRIPT-RICH SINGLE PAGE APP WITH INTEGRATED SEARCH
• WORD: MICROSOFT WORD
• EXCEL: MICROSOFT EXCEL
• JSON: A CUSTOM JSON FORMAT
SOURCESHTTP://WWW.SPECFLOW.ORG/
HTTP://MARTINFOWLER.COM/BLIKI/UBIQUITOUSLANGUAGE.HTML
HTTP://DANNORTH.NET/INTRODUCING-BDD/
HTTPS://GITHUB.COM/TECHTALK/SPECFLOW-
EXAMPLES/BLOB/MASTER/BOWLINGKATA/BOWLINGKATA-
NUNIT/BOWLING.SPECFLOW/SCORECALCULATION.FEATURE
HTTP://GUIDE.AGILEALLIANCE.ORG/GUIDE/UBIQUITOUS.HTML
HTTP://MARTINFOWLER.COM/BLIKI/TESTPYRAMID.HTML
HTTPS://WWW.MANNING.COM/BOOKS/BDD-IN-ACTION
HTTP://WWW.SPECFLOW.ORG/PLUS/RUNNER/
HTTPS://WWW.NUGET.ORG/PACKAGES/PICKLES/
HTTPS://SITES.GOOGLE.COM/SITE/UNCLEBOBCONSULTINGLLC/THE-TRUTH-ABOUT-BDD