Eurostar Past, Present, Future v4timkoomen.nl/wp-content/uploads/Eurostar-Past-Present...What to...
Transcript of Eurostar Past, Present, Future v4timkoomen.nl/wp-content/uploads/Eurostar-Past-Present...What to...
-
Past, present, future …
1
Tim Koomen Eurostar 2008
-
Journey from different viewpoints
� What to test
� People
� Techniques
� Tools
� Process� Process
� Organisation
-
What to test …
-
What to test …
� Past: testing of functionality
� Present: bias of testing more and more shifts toward non-functional aspects like performance, suitability, usability and security
-
What to test …
� The enormous amount of what can be tested has forced testers to choose very carefully what (not) to test and how thoroughly
� These choices are usually based on perceived risks
Risk techniques: TMap, RRBT, PRISMA, � Risk techniques: TMap, RRBT, PRISMA, SmarTEST, Failure mode and effects analysis (FMEA)
-
What to test: future?
� New IT developments (architecture, services, integration issues, standard solutions)
� Growing importance of non-functionals
� Art of testing remains: “choosing what to test and how thoroughly”“choosing what to test and how thoroughly”
-
People
� Past:
� from ad-hoc testers to …
� professional testers to …
� Present:
� SIGIST’s and � SIGIST’s and
� certified testers (like ISEB, ISTQB, TMap)
� non-functional test experts
-
Discussions on certificationPro:
� Knowledge of terminology
� Proof of test expertise
� Entry-criterion for professional tester
� Part of the tester’s career path
� Recognition of software testing as a professional specialisation within IT
� Commercial necessity when more than 1 customer
Con:
� Testing as a profession is still immature, therefore certification programs serve as an obstacle to innovation and learning
� Test certification is very light compared to other certified professionals like java programmer (18 months), software architect (15 months), this does not help recognition of our profession
� Commercially misused
� Having a test performed by a certified tester does not guarantee quality of work
� Having a test performed by a certified tester does not guarantee quality of work
� A good tester has knowledge / skills in the following areas:� IT in general,
� testing,
� analytical,
� soft skills,
� business knowledge
Only testing knowledge is part of the certification …
� Certification insufficiently covers non-functional testers
� Once certified, always certified
Sources: I. Evans, M. Bolton, K. Hanselman, R. Kochuyt, R. Marselis
-
People: future?
� Maybe a formal testing discipline?
� Professionals taking personal responsibility for their work
� Certification level(s) with entry-criteria and continuous professional development
� Governing body� Governing body
� Ethical rules and professional standards
� Disciplinary action for not observing standards or bad professional work
(I. Evans)
(however: far away future …)
� Business knowledge vs. technical knowledge?
� Soft skills certification?
-
Techniques
� Techniques: early!
� Well-known techniques:
� equivalence partitioning,
� boundary value analysis,
� domain testing, domain testing,
� graph testing,
� decision/condition testing,
� …
� Newer techniques?
-
Pairwise testing
� Buying a train ticket from the vending machine Parameters:
The aim of pairwise testing is to test all the possibilities of any combination of 2 factors
Parameters:� Trip : one-way; return
� Class : 1st , 2nd
� Payment : credit card; debit card; cash
-
Pairwise testing
Trip Class Payment
1 One-way 1st Credit card
2 One-way 1st Cash
3 One-way 2nd Debit card
4 Return 1st Debit card4 Return 1st Debit card
5 Return 2nd Credit card
6 Return 2nd Cash
-
Pairwise testing
Trip Class Payment
1 One-way 1st Credit card
2 One-way 1st Cash
3 One-way 2nd Debit card
4 Return 1st Debit card4 Return 1st Debit card
5 Return 2nd Credit card
6 Return 2nd Cash
-
Pairwise testing
Trip Class Payment
1 One-way 1st Credit card
2 One-way 1st Cash
3 One-way 2nd Debit card
4 Return 1st Debit card4 Return 1st Debit card
5 Return 2nd Credit card
6 Return 2nd Cash
-
Pairwise testing
Trip Class Payment
1 One-way 1st Credit card
2 One-way 1st Cash
3 One-way 2nd Debit card
4 Return 1st Debit card4 Return 1st Debit card
5 Return 2nd Credit card
6 Return 2nd Cash
-
Some more parameters …
� Parameters:� Trip: one-way, return
� Class: 1st , 2nd
� Payment: credit card,debit card, cash
� Discount: yes, no
� Place: lower, upper, standing, roof� Place: lower, upper, standing, roof
� Train type: ICE, slow train, dieselloc, intercity, steam
� Age:
-
Trip Class Payment
Discou
nt Place Train type Age Catering Color
1 one-way 1st debit card yes upper steam 65+ coffee red
2 return 2nd credit card no standing intercity
-
Exploratory testing
� … a technique?
Exploratory testing is the simultaneous learning, designing and executing of tests, in other words every form of testing in which the tester designs his tests during the test execution and the
information obtained is reused to design new and improved test cases.(James Bach, 2002)
� … a technique?
� keyword: “… the tester designs his tests…”
-
Model-based testing
Model-based testing is software testing in which test cases are derived inwhole or in part from a model that describes some (usually functional)
aspects of the system under test (Wikipedia)
Modelrequirements
functional
SystemExecutable
tests
Test design
test tool
Keywords:Automated generation and execution of test cases
functional design
technical design
-
Techniques … �
� But … which ones do we really use?
-
Classic: triangle case …
� Input: 3 integer values
� Output: message saying the values represent a scalene, isosceles or equilateral triangle
� How many test cases would be adequate…?
-
Techniques: future …
Room for more?
� … for non-functionals
� … for interfaces, integration
-
Tools
Tools for planning and controlling the test
� Testware management tool
� Defect management tool
� Planning and progress monitoring tool
� Workflow tool
Tools for designing the test
� Test data tool
� Test design tool
text based
GUI’s� Test design tool
� Model-based testing tool.
Tools for executing the test
� Automated test execution tool
� Performance, load and stress test tool
� Monitoring tool
� Code coverage tool
� Comparator
� Database manipulation tool
� Simulator
� Stubs and drivers
web-based
service-based
Automated test execution tool
• GUI• object recognition• action words / data-driven• framework / testsuite / components
-
Tools: future …
� Keeping up with IT developments
� Non-functionals
� Test design?
-
Process
� Standards� IEEE829 (software test documentation)� IEEE1008 (software unit testing)� BS7925-1 (Software Testing – Vocabulary)� BS-7925-2 (Standard for Software Component Testing) � IEEE 1028 (Reviews and Inspections)� ISO 9126 (Software Quality) � ISO 9126 (Software Quality) � ISO/IEC 25000:2005 (SQuaRE), supersedes ISO9126. � + specials (FDA, AECL)
� Books� 80’s: books on test design techniques (Beizer, Myers), � 90’s: several books on the process of testing (Black, Kaner, Kit, Pol)
� 00’s : more emphasis on agile, context-driven, flexible, adaptive aspects
-
Process
� Methods / process descriptions
� 90’s: structured test methods (STEP, TMap, Testframe/RRBT) emerged
� 00’s: many organisations invented / implemented their own variant of such a methodmethod
� Process Improvement
� 90’/00’s: specific test process improvement models (TPI, TMM, TMMi, V2M2)
� Certification schemes (ISEB, ISTQB, TMap)
� aimed at people, best practice processes
-
Process: future
Agile
Integrated in Division between
Formal
Integrated indevelopment best practices
Division between IT and Business testing
-
Organisation
Development Implementation
Development Test Implementation
Development Test ImplementationDevelopment Test Implementation
Development Test Implementation
Test expertise centre
-
Organisation
Development Test ImplementationDevelopment Test Implementation
Development Test Implementation
Test department(resource pool)
Test factory(result)
or
Development & test
(resource pool) (result)or
Test Policy
-
Organisation
Development Test ImplementationDevelopment Test Implementation
Development Test Implementation
Test factory(result)
Supplier organisation
-
Organisation: future
� Certification of services?
� Certification of test services?
� TaaS?
-
Past, present, future
� Many things
� “nothing’s changed, really…”
� Biggest changes
� people, organisation, non-functionals, tools
� Achievement
testing keeps up with IT developments� testing keeps up with IT developments
� Prediction
� higher variety … and long way to go
Software complexity (and therefore that of bugs) grows to the limits of our ability to manage that complexity
(Boris Beizer’s law of the complexity barrier)
-
M. +31 (0)6 34139260
I. www.timkoomen.nlI. www.timkoomen.nl
Copyright Tim Koomen Testmanagement en -advies