Transport For LondonKeeping London on the move

!Anand Ramdeo / @testinggeek

Inspiration“Tester who helped TFL in

revamping their web presence and presented his

work at one of the best testing conference - Let’s

Test 2014”

Good to know!

• 150+ years old • 24 million Journeys every day • 1.17 billion passenger

journeys in a year • 75% Londoners use TFL

website regularly • 250 million visits a year

!

“Lifeline of London”

Responsive design with better!usability and consistent! platform for developers.

Revamp

Journey!Planner Disruptions Lines Station

InfoTimeTable

And lots of other Interesting and useful services..

External developers who build interesting apps based on the

data available from TFL.

Tight coupling, inconsistent experience

Journey!Planner Disruptions Station

Info

And lots of other Interesting and useful services..

Consistent Restful Interface

External developers who build interesting apps based

on the data available from TFL.

Loosely coupled, RESTful!interface

Timetable Lines

Bad agile, good waterfallBad Agile!

• No testers in the team

• Not enough unit tests

• No Continuous Integration

• Sprint and Scrum

Good Waterfall!

• Excellent user research

• Almost finalised design

Assignment“You have got 21 days

to do automated regression testing for Journey Planner. Look at the stories we have

already finished.”

Challenge #1 Journey Planner

New, responsive web design which consumes RESTful

interface

Context Revealing Questions• Who are the stakeholders?

• What’s important for them?

• What is already tested by the team?

• When is it supposed to be released?

• What would be my oracle?

• What could possibly go wrong?

• Do I have access to data, environment, people who can help me test?

http://www.developsense.com/blog/2010/11/context-free-questions-for-testing/

Clear PrioritiesKey stake holder!

• Technical Architect

Key concerns!

• Quality of results, reliability.

• User experience, on web, mobile & devices.

• Validate architecture & deployment on cloud.

Scope!

• API isn’t important for beta release

Quality of Results

How do I know if route is right or wrong?

Simple Oracle

“Results should be identical to the live

website.”

Process1. Plan a journey on the existing

website (Source, destination, time, date, mode, preference etc.)

2. Note results (Journey time, departure, arrival, options, walking, escalators, earlier, later etc.)

3. Plan same journey with same options on the new website

4. Make a note of results

5. Compare

Key Problems!!

• Different UI • Too Many things to check • Time consuming • Error prone • Tiring • Difficult to scale

Automation!for the !Rescue

Journey j = new Journey(Victoria, Waterloo)!!routesFromExistingSite = JourneyPlanner.planJourneyOnOldSite(j)!!routesFromNewSite = JourneyPlanner.planJourneyOnNewSite(j)!!compareRoutes(routesFromExistingSite, routesFromNewSite)

Domain!Objects

Abstraction Encapsulation Clean code

Robust, Reliable, Maintainable & Useful automation!

http://www.testinggeek.com/a-software-tester-s-journey-from-manual-to-political

Progress so far..

• Who are my stakeholders?

• What’s important for them?

• How am I going to test?

Which routes do we need to test?

• Popular journeys

• Which modes are missing?

• Expand coverage

• Bug bash

Challenge #2!Status and Disruptions

• Realtime status of all the tube lines.

• Information about disruptions across all the modes.

• Not testable - could not inject disruptions.

• Oracle - compare with the live

What’s the biggest challenge?

SolutionContinuous monitoring using

Jenkins, PhantomJS and GhostDriver.

Challenge #3!Lines

Data Migration - Every bus, train, tram, river etc.

WaterlooNorthern Bakerloo Jubilee

WestministerJubilee Circle District

StanmoreJubilee

Stratford Jubilee Central DLR London OG

EppingCentral

North Acton Central

West Rulslip Central

StockwellNorthern Victoria

BrixtonVictoria

Green park Victoria

Jubilee Piccadilly

VictoriaVictoria Circle District

TFL Spider• Start from one station

• Get all lines from this station

• Get first line and check - is data present and correct?

• If yes, get list of all stations and mark this line as processed.!

• Add unique stations in the list of stations to process.

• If no, record missing data for this line and mark this line as processed.

Follow this for all lines on current station and move to the next

station

Challenge #4!Public API, Internal customer

• Assumed domain knowledge

• Team’s access to API developers

• Lack of consistency

• No versioning and regard for contract

PAPAS BE SFO!DEED HELP GC!

DVLA PC

Functionality Consumability MaintainabilityPaging Domain specific names DiagnosticAuthentication Examples VersioningParameters / Payloads Easy to learn LoggingAuthorization Documentation AccessibilitySecurity

Hard to misuse PurposeBehave Ease to use ConsumerError handling Lead to readable code

Principle of least surpriseStateFilter GuessabilityOrder Consistency

http://www.testinggeek.com/testing-restful-webservices-or-api-testing-remember-papas-be-sfo-deed-help-gc-and-dvla-pc

API Testing Mnemonic!

Lesson LearnedPro tips?

Pro Tip #1Automation does not have to be binary.

Use tools in the best possible way to help you test.

Pro Tip #2Break problem into manageable chunks.

Define layers, focus on one thing at a time.

Pro Tip #3Be context driven

Question and do the right things

Pro Tip #4know your stakeholders. What’s important for them

and why?

Pro Tip #5Use automation as a safety net, but be aware of its limitations.

Pro Tip #6Be vocal

!Blocked? Making progress?

Done something well? !

Don’t assume people would know.

Pro Tip #7Find ways to make progress

Pro Tip #8Embrace throwaway automation and utility

scripts.

Pro Tip #9Make testing visible

!http://www.satisfice.com/presentations/dashboard.pdf

Pro Tip #10Keep asking

Are we doing the right thing? !

Remember !

Projects unfold over time in ways that are often not

predictable.

Thank YouAnand Ramdeo !@testinggeek!

!anand@testinggeek.com

!TestingGeek.com TestSpicer.com

PlannedDeparture.com !