Post on 03-Jan-2016
description
A Walk on the Web SideA Walk on the Web Side
Differences and Differences and Similarities Between Web Similarities Between Web and non-Web Application and non-Web Application
TestingTesting
WISQA Special WISQA Special PresentationPresentation
June 11, 2003June 11, 2003
OverviewOverview
Functionality and Reliability testing: Functionality and Reliability testing: making sure a website works as requiredmaking sure a website works as required
Compatibility: making sure a website works Compatibility: making sure a website works as required for the users’ many operating as required for the users’ many operating systems and browserssystems and browsers
Usability: making sure a website works to Usability: making sure a website works to meet users’ goals with effectiveness, meet users’ goals with effectiveness, efficiency, and satisfactionefficiency, and satisfaction
Functionality and Reliability Testing
Functionality & Functionality & Reliability: The Usual Reliability: The Usual
SuspectsSuspects Forms and form validationForms and form validation Error messagesError messages Proofreading the contentProofreading the content Checking calculationsChecking calculations Verifying that new entries are saved Verifying that new entries are saved
correctlycorrectly Verifying that changes are saved correctlyVerifying that changes are saved correctly Verifying that deletions are really deletedVerifying that deletions are really deleted Find the weaknesses in system under testFind the weaknesses in system under test Regression testingRegression testing
Functionality & Functionality & Reliability: Web-wiseReliability: Web-wise
Update releases: when live, users see Update releases: when live, users see instantlyinstantly Navigation: users have choices outside of what Navigation: users have choices outside of what
the site itself offers; page orders aren’t strictthe site itself offers; page orders aren’t strict Browser settings: users can control many things Browser settings: users can control many things
about their browser… how many combinations about their browser… how many combinations will you test? how many directly affect the site?will you test? how many directly affect the site?
Cookies: should the site work without them? Cookies: should the site work without them? Search: what can Search: what can this site’sthis site’s search functions do? search functions do? Are there web-unique functions for the site… like Are there web-unique functions for the site… like
uploading or downloading files? (snapfish.com) uploading or downloading files? (snapfish.com) or sending email? (cnn.com)or sending email? (cnn.com)
Functionality & Functionality & Reliability: Complications Reliability: Complications
of eCommerceof eCommerce Business rules vary from online seller to Business rules vary from online seller to
online seller, so understanding them quickly online seller, so understanding them quickly is vitalis vital
Is registration (saving of name, address, and Is registration (saving of name, address, and possibly credit card number) required before possibly credit card number) required before checkout? Can it be done during or even checkout? Can it be done during or even after checkout?after checkout?
Are shipping and billing addresses saved Are shipping and billing addresses saved separately or together? Can they be edited, separately or together? Can they be edited, or must they be deleted and re-entered?or must they be deleted and re-entered?
What are the rules for sales tax calculation?What are the rules for sales tax calculation? What parts of the site must be secure?What parts of the site must be secure?
Functionality & ReliabilityFunctionality & ReliabilityPlaying with Test CasesPlaying with Test Cases
www.rei.comwww.rei.com
Let’s buy a roof-mounted bike rack for the Let’s buy a roof-mounted bike rack for the car. What test cases do we see along the car. What test cases do we see along the way?way?• NavigationNavigation• SearchSearch• RegistrationRegistration• CheckoutCheckout• Site-specific featuresSite-specific features
Functionality & ReliabilityFunctionality & ReliabilityHandy Tools We UseHandy Tools We Use
Many tools for automated testing of web Many tools for automated testing of web sites exist (see sites exist (see www.testingfaqs.orgwww.testingfaqs.org) )
Defect tracker: TeamTrack by TeamShareDefect tracker: TeamTrack by TeamShare SnagIt by TechSmith, super screen captureSnagIt by TechSmith, super screen capture Realistic Random Data generatorRealistic Random Data generator Microsoft ExcelMicrosoft Excel Paper techniques:Paper techniques:
• Page flows – helps plan workflow casesPage flows – helps plan workflow cases• Page hierarchy diagrams – helps plan navigation Page hierarchy diagrams – helps plan navigation
casescases• Page mockups – identifies page elements Page mockups – identifies page elements
requiring casesrequiring cases
Compatibility Testing
Compatibility TestingCompatibility Testing
It’s a browser jungle out there:It’s a browser jungle out there:• Internet Explorer 6, 5.5, 5, 4…Internet Explorer 6, 5.5, 5, 4…• Netscape Navigator 7, 6.2, 6.1, 6, 4.79, 4.78…Netscape Navigator 7, 6.2, 6.1, 6, 4.79, 4.78…• OperaOpera• MozillaMozilla• SafariSafari• AOLAOL• More! More! www.download.comwww.download.com offers 192 other offers 192 other
browsers!browsers! Don’t forget the operating systems: Don’t forget the operating systems:
Windows (95, 98, 2000, ME, XP, NT), Mac Windows (95, 98, 2000, ME, XP, NT), Mac (OS 9, OS X), Linux…(OS 9, OS X), Linux…
How could you possibly test them all?How could you possibly test them all?
Compatibility TestingCompatibility TestingMaking Good ChoicesMaking Good Choices
First step is to help the customers choose First step is to help the customers choose which OS/browser combinations to supportwhich OS/browser combinations to support• Do they have web logs for a current site?Do they have web logs for a current site?• Do they have a specialized customer base?Do they have a specialized customer base?• Get national statistics, watch the newsGet national statistics, watch the news
Consider support for more users vs. Consider support for more users vs. technology: if you technology: if you mustmust use a technology use a technology (for example, Cascading Style Sheets), who (for example, Cascading Style Sheets), who might this disenfranchise? (And does the might this disenfranchise? (And does the customer care?)customer care?)
Compatibility TestingCompatibility TestingCreating a Test LabCreating a Test Lab
Create a planCreate a plan• What browsers and operating systems do you What browsers and operating systems do you
need?need?• What limitations are there (e.g., can’t install What limitations are there (e.g., can’t install
multiple IE versions on one machine)?multiple IE versions on one machine)?• What hardware do you need? (machines, browsers, What hardware do you need? (machines, browsers,
hubs, KVM switches for monitors & keyboards)hubs, KVM switches for monitors & keyboards)• What software do you need? (browsers, screen What software do you need? (browsers, screen
capture, instant messenger, etc.)capture, instant messenger, etc.)• Try for a modem and phone lineTry for a modem and phone line
Go scavengingGo scavenging• Old PCs useless to company employees are Old PCs useless to company employees are
invaluable to your labinvaluable to your lab
Compatibility TestingCompatibility TestingMuppet LabsMuppet Labs
““Muppet Labs” lives in a corner of our buildingMuppet Labs” lives in a corner of our building• 9 PCs9 PCs• 3 Macs3 Macs• 9 Operating Systems9 Operating Systems• 21 Browsers21 Browsers
Lab area is labeled with useful information:Lab area is labeled with useful information:
IE VersIE Vers Netsc Netsc VerVer
AOL AOL VerVer
OSOS NameName NotesNotes
5.005.00 4.614.61 66 W98W98 KermitKermit scannescannerr
5.55.5 NoneNone 77 W98W98 Miss Miss PiggyPiggy
No No netscnetsc
Usability Testing
First, a few definitions…First, a few definitions…
User InterfaceUser Interface - The language by which a - The language by which a user and system communicate with each user and system communicate with each other.other.
UsabilityUsability - The extent to which a product - The extent to which a product can be used by specified users to achieve can be used by specified users to achieve specified goals with effectiveness, specified goals with effectiveness, efficiency, and satisfaction in a specified efficiency, and satisfaction in a specified context of use.context of use.
(ISO 9241-11 - Ergonomic requirements for office work with visual (ISO 9241-11 - Ergonomic requirements for office work with visual display terminals (VDTs) - guidance on usability)display terminals (VDTs) - guidance on usability)
More definitions…More definitions…
User Centered Design (UCD)User Centered Design (UCD) - any of a wide - any of a wide variety of methods and techniques for variety of methods and techniques for designing usable products. Key features:designing usable products. Key features:
• Understand users - who they are & what they Understand users - who they are & what they needneed
• Understand the context of useUnderstand the context of use• Involve users in the (iterative) design processInvolve users in the (iterative) design process• Evaluate designs against requirementsEvaluate designs against requirements
(ISO 13407: Human-centered Design Processes for Interactive Systems)(ISO 13407: Human-centered Design Processes for Interactive Systems)
And more definitions…And more definitions…
Usability Evaluation Usability Evaluation - any of a class of methods - any of a class of methods used to assess/evaluate/review how usable a used to assess/evaluate/review how usable a particular product is.particular product is.• Expert evaluationExpert evaluation• Competitive evaluationCompetitive evaluation• Task analysisTask analysis• Cognitive walkthroughCognitive walkthrough• User testUser test• etcetc
User Test User Test - Facilitated evaluation of the usability - Facilitated evaluation of the usability of a product by asking real users to perform of a product by asking real users to perform real tasks with real products in real contexts.real tasks with real products in real contexts.• Or as close as you can get to “real”Or as close as you can get to “real”
Web vs. DesktopWeb vs. Desktop
Ease-of-learning vs. ease-of-useEase-of-learning vs. ease-of-use
Much more holistic user experienceMuch more holistic user experience
Nobody reads on the webNobody reads on the web
““use, then buy” vs. “buy, then use”use, then buy” vs. “buy, then use”
What to Use WhenWhat to Use When
Competitive EvaluationCompetitive Evaluation• Product definitionProduct definition• Planning for redesignPlanning for redesign• Helps identify areas of concern, strategic Helps identify areas of concern, strategic
advantages, etc.advantages, etc.• Define Define whatwhat you want to build you want to build
Task Analysis, et. alTask Analysis, et. al• Analysis & designAnalysis & design• Helps ensure you’re solving the problem the Helps ensure you’re solving the problem the
right wayright way• Define Define howhow you want to build it you want to build it
What to Use When (cont)What to Use When (cont)
Expert EvaluationExpert Evaluation• Early (and often) during initial design phasesEarly (and often) during initial design phases• As soon as you have anything to evaluateAs soon as you have anything to evaluate• Very large bang for the buck - find most of big Very large bang for the buck - find most of big
gotchas earlygotchas early
User TestingUser Testing• Not-quite-as-early (but still early) and oftenNot-quite-as-early (but still early) and often• Certainly before launchCertainly before launch
You’re going to user test it anyway, might as well You’re going to user test it anyway, might as well do it before launchdo it before launch
• A bit harder, but only way to get at some issuesA bit harder, but only way to get at some issues
Lo-fi vs. Hi-fiLo-fi vs. Hi-fi
Don’t wait until you have a working system Don’t wait until you have a working system (or even prototype) to test(or even prototype) to test
You can test a piece of paperYou can test a piece of paper Sometimes, low-fidelity is better to test Sometimes, low-fidelity is better to test
(fewer misconceptions)(fewer misconceptions) Fixing mistakes during construction costs Fixing mistakes during construction costs
ten times as much as fixing during designten times as much as fixing during design You wouldn’t dream of not testing what you You wouldn’t dream of not testing what you
build… why would you ever NOT test what build… why would you ever NOT test what you design?you design?
Let’s Practice!Let’s Practice!
www.intellicast.comwww.intellicast.com• What’s the forecast for Buffalo, NY be tomorrow?What’s the forecast for Buffalo, NY be tomorrow?• Find a current radar image for HonoluluFind a current radar image for Honolulu
www.usatoday.comwww.usatoday.com• Find the box score for the Milwaukee Brewers Find the box score for the Milwaukee Brewers
baseball game yesterdaybaseball game yesterday• Find some current news about the NHL playoffsFind some current news about the NHL playoffs
Recommended ReadingRecommended Reading
Basics of web testing: Basics of web testing: The Web Testing The Web Testing HandbookHandbook by Steven Splaine and Stefan P. by Steven Splaine and Stefan P. JaskielJaskiel
For software with a web front end: For software with a web front end: Testing Testing Applications on the Web Applications on the Web by Hung Q. Nguyenby Hung Q. Nguyen
Usability: Usability: Don't Make Me Think: A Common Don't Make Me Think: A Common Sense Approach to Web UsabilitySense Approach to Web Usability by Steve Krug by Steve Krug
Handbook of Usability Testing: How to Plan, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective TestsDesign, and Conduct Effective Tests by Jeffrey by Jeffrey Rubin Rubin
See See www.stickyminds.comwww.stickyminds.com for more books and for more books and articlesarticles