Software testing and quality assurance

of 30 /30
Software Testing & Quality Assurance Why using Specflow (Gherkin) and Selenium ? Benjamin Baumann @Zentiltoutou October 21st 2016

Embed Size (px)

Transcript of Software testing and quality assurance

PowerPoint Presentation

Software Testing & Quality AssuranceWhy using Specflow (Gherkin) and Selenium ?Benjamin [email protected] 21st 2016

How much costs a bug?1962 NASA - Mariner 1Omission of a hyphen in instructions. Auto destruct. $18M1996 Ariane 5Integer Overflow on old legacy code that is not needed. No flight simulator to spare money. $370M for satellites alone

Coder punchline #1

How much costs a bug?2009 ToyotaLag in the anti-lock-brake system. Recall 9M cars. $3G estimated2012 Knight Capital GroupCode deployed to all but one server. This server sent out 4M+ wrong stock orders in one hour. $440M

But were no Spacecraft or trading firm!2015 Eurosport Player V6Video lags, fps drop, overheating PC, angry clients.Due to incompatibility between technology and requirements. Detected by beta testing but gone live before the test results

Ok. Lets test! White-box? Black-box?White-box testingTests the internal structure or workings of a program :Unit testsFunctional testingIntegration testingRegression testingLoad & Stress testingSanity & Smoke testingBlack-box testingSoftware is a black box. Tester knows nothing about how it works.Acceptance testingBeta testing

Unit tests are idiots.

Unit tests are idiots.

Unit tests are idiots but legion!Think about code coverageAssess code quality, not softwaresThink about edge cases, error handling

Unit tests are idiots, legion and hard to maintainEach code change/writing begin with test change/writingIm a developer, these tests are idiots! Why do I have to write them?The tests failed, such a pain. Ill just comment them out

Break #1 : Thats not me. All internet is saying it!

Thats not me. All internet is saying it!

Thats not me. All internet is saying it!

Thats not me. All internet is saying it!

Still, I want to do unit testingPlease, do unit testing only if :You have a decent number of developers in your teamYou have time to do it and to maintain itYou can run them and validate them automatically (through CI)You run a business critical projectYou have detailled technical specsCoding unit test is still coding, treat your tests like your codeDont write the code AND the tests, make someone else write the tests.Golden rule : Each unit test case should be very limited in scope

Ok I have a ton of unit tests. Am I OK?

Ok I have a ton of unit tests. Am I OK?

Focus on functional,regression and acceptance testsTesting software features above code methodsLess detailed, less to do, less to maintainFocus on testing the requirements/specsSafeguard for developers

Ok lets test a feature !Describe the feature : Who? What? Why?In order to watch the player As a Eurosport Player user I want to be able to connectHelp describing the needFeature-centricTechnology agnosticNeed for testing scenarios

Ok lets write a scenario !Given I go to the Eurosport Player home pageAnd I open the connection modalAnd I enter my credentialsWhen I submit the formThen I must be logged in

White-box? Black-box?

Ok lets write a scenario!Given I go to the Eurosport Player home pageAnd I open the connection modalAnd I enter my credentialsWhen I submit the formThen I must be logged in

White-box! Describing a feature should be blackboxed.

Ok lets write a better scenario!Given I go to the Eurosport PlayerWhen I loginThen I must be able to watch Eurosport Player

No tech. No design stuff. Just the feature.

Feature does not change when code is refactored.You should rewrite the code behind, but not the test scenario

Break #2: Understanding others

Break #2: Understanding othersHey, this is not the behaviour I asked for!, But thats what you told me in your mail, You did not understand, thats not what I wanted Domain, Network, AD, I mix up all these, its gibberish to meSolutionsLearn the other language so that you understand themEveryone learns a new language so that we understand each other (AWS)Dont go into details, just do high levelSpeak like youd speak to the average man in the street

What about Gherkin?Weve already used GherkinGherkin is the language that Specflow understands. It is aBusiness Readable, Domain Specific Languagethat lets you describe softwares behaviour without detailing how that behaviour is implemented.Gherkin serves two purposes documentation and automated tests. The third is a bonus feature when it yells in red its talking to you, telling you what code you should write.

Documentation by GherkinList featuresList testing scenariosReadable by EVERYONEHard to write, easy to understandA heaven for maintenance

http://qa.sti.eurosport.intra/Home/SpecflowList?path=d%3A%5Csites%5CQA%5CProjects%5CEurosport%20Player%20PROD%5CScenarios&projectName=Eurosport%20Player%20PROD

Automated tests with SpecflowEach scenario line is linked to a code methodPass scenario data to the methodScenario Context (browser, language, environment)Runnable in classic test runners (NUnit, MsTest )Explicit errorsWhen failed, need to go deeper with unit tests or by hand

And what about Selenium?Selenium WebDriver is a library that can drive a web browserMethod on the webdriver results on real user action

Let code!

Merci!