Software Testing - University of...
Transcript of Software Testing - University of...
![Page 1: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/1.jpg)
Software TestingTheory and Practicalities
![Page 2: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/2.jpg)
Purpose
• To find bugs
• To enable and respond to change
• To understand and monitor performance
• To verify conformance with specifications
• To understand the functionality of code
![Page 3: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/3.jpg)
Why Study Testing
• Testing is an integral part of professional software engineering
• Know how, when and why to apply testing techniques
![Page 4: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/4.jpg)
Classifying Testing
• There are many testing techniques -- a taxonomy is useful
• White box vs. Black box
• Automated vs. Manual
![Page 5: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/5.jpg)
White vs. Black Box
• How deeply do you look at the implementation of the system?
• Black box: do not want to look at implementation
• “Grey” box - understands assumptions and limitations of system
• White box: full knowledge of code, deliberate use of implementation choices
![Page 6: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/6.jpg)
Automated vs. Manual
• Automated: “make test”
• Manual “poke poke poke”
• Semi-automated: configure, setup, test, evaluate, retest
![Page 7: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/7.jpg)
Test Techniques
API Regression Unit
GUI Automation
Performance Integration Conformance
Smoke
Load Security
FunctionalUsability
Understanding
BB WB
Man
Auto
![Page 8: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/8.jpg)
Where to Apply These
• Roughly, the “lower” the level of the component, the more amenable to automation
• Higher level components often require significant “scaffolding” to test effectively
• Complete functional coverage of a GUI application can be very resource intensive
![Page 9: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/9.jpg)
Unit Tests
• Ensure completeness and correctness of implementation
• Cover all public APIs and significant internal APIs
• Begin writing tests as soon as basic structure and some APIs are known
• Regression tests: bugs as a source of test
![Page 10: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/10.jpg)
Unit Tests
• Use or create your own infrastructure to make writing unit tests trivial.
• Goal: “make test” yields true or false
![Page 11: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/11.jpg)
Unit Tests (Example 1)# ... main part of module ...
import unittestclass TestURLCanonicalization(unittest.TestCase):
def test_basic( self ): u = URL( "http://www.example.com" ) self.assertTrue( u.scheme == "http" ) self.assertTrue( u.host == "www.example.com" ) self.assertTrue( u.port == 80 ) self.assertTrue( u.path == "/" ) self.assertTrue( u.params == "" ) self.assertTrue( u.fragment == "" ) self.assertTrue( u.url == "http://www.example.com/" )
if __name__ == "__main__": unittest.main()
![Page 12: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/12.jpg)
Unit Tests (Example 2)# ... main part of module ...
import unittestclass TestURLCanonicalization(unittest.TestCase):
def test_nonascii_escaped_fragment( self ): u = URL( "http://www.facebook.com/?ref=home#!/pages/โครงการหาบานใหมใหสุนัขจรจัด-มก/117976974964923?sk=wall" ) self.assertTrue( u.scheme == "http" ) self.assertTrue( u.host == "www.facebook.com" ) self.assertTrue( u.port == 80 ) self.assertTrue( u.path == "/" ) self.assertTrue( u.params == "ref=home&_escaped_fragment_=/pages/..." ) self.assertTrue( u.fragment == "" ) self.assertTrue( u.url == "http://www.facebook.com/?ref=home&_escaped_fragment_=/pages/%E0%B9%82%E0...%81/117976974964923?sk=wall" )
if __name__ == "__main__": unittest.main()
![Page 13: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/13.jpg)
API Testing
• A variant of Unit testing
• An important component of module documentation
• Added emphasis on edge-cases, exceptional conditions, parameter verification, abuse
• Well defined APIs work well with a test-first strategy
![Page 14: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/14.jpg)
API Test Example"""This is the "example" module.
The example module supplies one function, factorial(). For example,
>>> factorial(5)120"""
def factorial(n): """Return the factorial of n, an exact integer >= 0.
If the result is small enough to fit in an int, return an int. Else return a long.
>>> [factorial(n) for n in range(6)] [1, 1, 2, 6, 24, 120]”””
... import math if not n >= 0: raise ValueError("n must be >= 0")
...
if __name__ == "__main__": import doctest doctest.testmod()
![Page 15: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/15.jpg)
Complex Integration and Unit Tests
• Test core application logic
• Often requires setting up test environment
• Invest in stubs, mocks, and scripted test scaffolding to automate complex testing
![Page 16: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/16.jpg)
Stubs, Mocks, and Scaffolding
• Stubs: downstream fn that gives a reasonable response (null, True, ...)
• Mock: somewhat intelligent stub. e.g. Must call fnX before fnY.
• Scaffolding: potentially complex test environment. VMs, DBs, configurations
![Page 17: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/17.jpg)
Stubs and Mocks
• More of a development than testing tool
• Focus development on one component at a time
• Enable unit and integration tests to be written
![Page 18: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/18.jpg)
Example Stub
class DummyCache( object ): def lookup( self, key, default_value = None ): return default_value def remove( self, key ): pass def store( self, key, val ): pass def clear( self ): pass
![Page 19: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/19.jpg)
Test Scaffolding
• Goal: a controlled version of an approximation of some “real world”
• Automate or die
• Scripting, tooling, system administration skills are required
![Page 20: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/20.jpg)
Complex example
Game Model Scoring
GUI
Utilities
Game Model Scoring
GUI
Utilities
Game Model Scoring
GUI
Utilities
Game Model Scoring
GUI
Utilities
Game Model Scoring
GUI
Utilities
![Page 21: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/21.jpg)
Scaffolding Env ExampleVagrant::Config.run do |config| config.vm.box = "precise64" config.vm.provision :chef_solo do |chef| chef.cookbooks_path = "cookbooks" chef.add_recipe "mysql" chef.add_recipe "mysql::server" end config.vm.provision :shell, :path => "./setup.sh"end
#!/bin/sh# setup test DBmysql -u root -p password -e “"create database test;GRANT ALL PRIVILEGES ON test.* TO user@localhost IDENTIFIED BY 'password';FLUSH PRIVILEGES;“# fetch test datacurl http://example.com/test_data.csv > /tmp/test_data.csv# load test datamysqlimport -u root -p password --local test /tmp/test_data.csv
$ make test# cd tests/vagrant# vagrant up# cd ../tests/integration# python test.py ...
![Page 22: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/22.jpg)
High-Level GUI Automation Testing
• “Robot” user - automate the use of the application
• Understand what you are testing
• Can be useful, but may be brittle
![Page 23: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/23.jpg)
Selenium Examplefrom selenium import webdriverfrom selenium.common.exceptions import TimeoutExceptionfrom selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Firefox()driver.get("http://www.google.com")inputElement = driver.find_element_by_name("q")inputElement.send_keys("Cheese!")inputElement.submit()
try: # wait for the page to refresh WebDriverWait(driver, 10).until(lambda driver : driver.title.lower().startswith("cheese!"))
# make assertions about the context of the page here...finally: driver.quit()
![Page 24: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/24.jpg)
Functional Testing
• Humans (finally!)
• Written scripts and checklists
• Levels of detail: smoke, iteration, alpha, beta, release acceptance
![Page 25: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/25.jpg)
Often Ignored
• Logs
• Internationalization
• Stubbed exception handling
![Page 26: Software Testing - University of Washingtoncourses.cs.washington.edu/.../cse403_dzc_testing.pdfStubs, Mocks, and Scaffolding •Stubs: downstream fn that gives a reasonable response](https://reader033.fdocuments.in/reader033/viewer/2022060311/5f0adf567e708231d42dc1bc/html5/thumbnails/26.jpg)
Summary
• Some testing is the developer’s responsibility
• Automation makes powerful testing relatively simple