Object-Oriented Design and Programming (Java)

2

Topics Covered Today

• Overview of Software Testing• Software Testing Methods• Software Test Types• 2.1 Implementing Classes

– 2.1.4 Unit Testing

3

Overview of Software Testing

• Purpose of Software Testing • Role of Testers • Truths and Realities

4

Software Testing

• Any activity designed to evaluate an attribute or capability of a program to determine that it meets required standards.

5

Purpose of Software Testing

• Measure Quality– Validation 确认 : Prove it works (Do the right thing)– Verification 证明 : Do it right

• Provide information– To development and management– For risk evaluation

• Watching the process not just the product

6

Software Quality

• Doing the right things right at the right time

• Conformance to applicable standards– Customer focus

• Features vs. Flaws

– Engineering focus• Maintainability, sustainability 可维护 , testability, etc.

• Quality is defined as conformance一致 to requirements, not as “goodness” or “elegance”

• Quality also means “meet customer needs”

7

Role of Testers

• Find Bugs • Break Software • Keep Developers Honest • Defect Prevention • Quality Measurement • Drive quality upstream • Customer focus

Goal: ensure the quality

8

Truths and Realities

• The earlier a bug is fixed, the less costly it is.– Before code review– By code review– By testing– By Beta testers– After product is released

• Prevention vs. Detection– “The mere act of designing a test is a powerful bug

preventer”

9

Truths and Realities

• Thorough Testing vs. Risk-based testing– Thorough Testing: testing for everything– Risk-based Testing

• Identify and analyze risks to enable informed and calculated decisions

• Look for the best use of test resource• Risks can be categorized by severity and likelihood.

10

Software Testing Methods

• Black-box Testing • White-box Testing • Risk-based Testing

11

Black-box Testing

• Doesn’t require explicit knowledge of the internal structure of the system

• Primarily focuses on functional requirements– User-perspective

– Validates the spec

Input Output

12

White-box testing

• Requires knowledge of the internal structure

• Primarily focuses on code coverage– e.g. write tests to “cover” internal paths– Good for testing individual functions– Developer’s tests

Input Output

13

Risk-based Testing

• Used to determine:– What to test– Test priority– Test coverage

14

Risk-based Testing

15

Risk-based Testing

• Two basic factors for determining risk:– Impact to the customer– Probability of customer experiencing issues

• Other factors affecting risk assessment:– History, complexity, usability, dependencies, new

features or technologies, modifications, etc.

• Value of Risk Assessment– Prioritize work– Scope testing effort

16

Software Test Types

• 基于是否需要执行被测试软件– Dynamic Test 动态测试、 Static Test 静态测试；

• 基于是否关注软件结构与算法– Black Box Test 、 White Box Test

• 基于测试的不同阶段– Unit testing 单元测试、 Integration testing 集成测试、

System testing 系统测试、 Acceptance testing 验收测试；• 其它

– Smoke Test 冒烟测试、 Regression Test 回归测试、 Functional Test 功能测试、 Stress Test 负载测试 ( 压力测试 ) 、 Performance Test 性能测试、 Accessibility Test 易用性测试、 Install/Uninstall Test 安装与反安装测试、 Security Test 安全性测试、 Application Compatibility Test 兼容性测试、 Alpha 测试、 Beta 测试等

17

Test Stages

System specification

( 规格定义 )

Design ( 设计 )

Code ( 编码 )

System testing

( 系统测试 )

Integration testing ( 集成测试 )

Unit testing ( 单元测试 )

Requirement specification

( 用户需求 )

Acceptance testing

( 验收测试 )

18

BVT(Build Verification Test)

• An automated test suite run on each new build to validate the integrity of the build and basic functionality before the new build is released for general testing. – Also known as build acceptance test （构建验证测试

） .

• Advantages– Save testing time– Validate basic functionality

• Disadvantages– Minimum level of test coverage

19

Application Compatibility Test

• Test 3rd-party application behavior and integration with new software

• Advantages– Exercise application interaction– Customer focused

• Disadvantages– Testers not usually experts in all application software– Impossible to test all applications or interactions

20

Scenario Tests

• Tests that simulate customers’ real usage

• Advantages– Customer focused

• Disadvantages– Difficult to simulate

21

Software Testing

• Important and expensive activity– Not unusual to spend 30-40% of total project effort on

testing– For critical systems (e.g. flight control): cost can be

several times the cost of all other activities combined

22

Test Planning

• Why Test Plan

• What’s in a Test Plan

• How to write a test plan

23

Benefits of Writing Test Plan

• Improves test coverage and efficiency

• Improves communication among peers, teams and with management– Feedback

• Helps manage testing

• Improves individual accountability

24

What’s in a Test Plan

• Who will be doing the work

• What will be tested

• What will NOT be tested

• Test environments (hardware/software)

• Test strategy

• Test dependency

• Test groups/categories

• Risks/Open Issues

25

Test Development

• Characteristics of Good Tests– Reasonable probability of catching an error

– Not redundant

– Be focused

• Development Techniques– Boundary Value Analysis

– Error Guessing/Exploratory Testing

• Test Case Documentation

26

Test Case

• A test case(测试用例 ) is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly

• Test case: specifies– Inputs + pre-test state of the software– Expected results (outputs and state)

27

Bug Report – Bad Example

Title: AV occurs.

Description:

An AV occurs when running test 2641.

Repro Steps:

1) Run test 2641

28

Mistakes to Avoid

• Incomplete repro steps

• Lack of details and investigation

• Missing environment/configuration info

• Missing expected result

• Duplicate bugs

29

Good Testers

• Good knowledge of computer science• Knowledge of one or more programming languages• Debugging skills• Attention to details• Ability to prioritize work• Team player• Good communication skills

30

Topics Covered Today

• Overview of Software Testing• Software Testing Methods• Software Test Types• 2.1 Implementing Classes

– 2.1.4 Unit Testing

31

Unit Testing

• scope = individual component

• Focus: component correctness

• Responsibility of the developer

• White-box and black-box techniques

32

Example: Hello World Program

• Use main method to test class “HelloWorld”

• Testing plan:– Instance of HelloWorld.java should not be null– Output should be “Hello World”

33

HelloWorld.java code

public class HelloWorld {

public String sayHello() {

System.out.println("Hello World");

return “Hello World”;

}

public static void main( String[] args ) {

HelloWorld world = new HelloWorld();

world.sayHello();

}

}

34

Create a New Class for Testingpublic class HelloWorld {

String sayHello() {

return "Hello World!";

} }

public class TestHelloWorld {

public static void main(String[] args) {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

if (result.equals("Hello World!") == false) {

System.out.println("Bad result: " + result);

} }

}

35

Example 2

• Another example to create a new class for testing (see 2.1.4 Unit Testing).– BankAccount.java– TestBankAccount.java

36

Class BankAccountpublic class BankAccount { private double balance; public BankAccount() { balance = 0.0; } public double getBalance() { return balance; } public boolean deposit(double amount) { if (amount > 0) { balance += amount; return true; } else { return false; } } public boolean withdraw(double amount) { if (amount > 0 && balance >= amount) { balance -= amount; return true; } else { return false; } } }

37

Class TestBankAccountimport java.io.*; public class TestBankAccount { private static PrintWriter stdOut = new PrintWriter(System.out, true); private static PrintWriter stdErr = new PrintWriter(System.err, true);

public static void assertTrue(String message, boolean condition) { if (! condition) { stdErr.print("** Test failure "); stdErr.println(message); } } public static void main(String[] args) { boolean result; // Testing constructor and accessor BankAccount accountOne = new BankAccount(); assertTrue("1: testing method getBalance()", accountOne.getBalance() == 0);

38

Class TestBankAccount //Testing method deposit BankAccount accountTwo = new BankAccount(); result = accountTwo.deposit(100); assertTrue("2: testing method deposit", result); assertTrue("3: testing method deposit", accountTwo.getBalance() == 100);

result = accountTwo.deposit(50); assertTrue("4: testing method deposit", result); assertTrue("5: testing method deposit", accountTwo.getBalance() == 150);

result = accountTwo.deposit(0); assertTrue("6: testing method deposit", ! result); assertTrue("7: testing method deposit", accountTwo.getBalance() == 150);

result = accountTwo.deposit(-25); assertTrue("8: testing method deposit", !result); assertTrue("9: testing method deposit", accountTwo.getBalance() == 150);

39

Class TestBankAccount //Testing method withdraw BankAccount accountThree = new BankAccount(); accountThree.deposit(100);

result = accountThree.withdraw(60); assertTrue("10: testing method withdraw", result); assertTrue("11: testing method withdraw", accountThree.getBalance() == 40); result = accountThree.withdraw(50); assertTrue("12: testing method withdraw", ! result); assertTrue("13: testing method withdraw", accountThree.getBalance() == 40); result = accountThree.withdraw(0); assertTrue("14: testing method withdraw", ! result); assertTrue("15: testing method withdraw", accountThree.getBalance() == 40); result = accountThree.withdraw(-10); assertTrue("16: testing method withdraw", ! result); assertTrue("17: testing method withdraw", accountThree.getBalance() == 40); stdOut.println("done"); } }

40

A (slightly) better TestHelloWorld program

public class TestHelloWorld {

private int nbErrors = 0;

public void testSayHello() {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

if (result.equals("Hello World!") == false) {

throw new RuntimeException("Bad result: " + result);

}

}

41

A (slightly) better TestHelloWorld program

public static void main(String[] args) {

TestHelloWorld test = new TestHelloWorld();

try { test.testSayHello(); }

catch (Throwable e) {

test.nbErrors++;

e.printStackTrace(); }

if (test.nbErrors > 0) {

throw new RuntimeException

("There were " + test.nbErrors + " error(s)");

}

} }

42

Advantage

• Move the test into its own method. – It’s now easier to focus on what the test does. – You can also add more methods with more unit tests

later, without making the main block harder to maintain.

• Change the main block to print a stack trace when an error occurs and then, if there are any errors, to throw a summary exception at the end.

43

JUnit

• JUnit is a framework for performing unit testing on programs– A unit test is a test of a single class

• A test case( 测试用例 ) is a single test of a single method• A test suit( 测试系列 ) is a collection of test cases

• Available as a stand-alone application and build into Eclipse

• Framework executes the test cases and records the results– Displays results in a GUI

44

Why JUnit

• Allow you to write code faster while increasing quality• Elegantly simple • Check their own results and provide immediate feedback • Tests is inexpensive • Increase the stability of software • Developer tests • Written in Java • Free • Gives proper understanding of unit testing

45

New->Other->Java->JUnit->JUnit Test Case

46

Create New JUnit Test Case

47

TestHelloWorld program written with JUnit 3.x

import junit.framework.TestCase;

public class TestHelloWorld extends TestCase {

public void testSayHello() {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

assertEquals("Hello World!", result);

}

}

48

import junit.framework.TestCase;

public class TestHelloWorld extends TestCase {

public void testSayHello() {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

assertEquals("Hello World!", result);

}

}1. You start by extending the test class from the standard Junit junit.framework.TestCase. This base class includes the framework code that JUnit needs to automatically run the tests.

TestHelloWorld program written with JUnit 3.x

49

TestHelloWorld program written with JUnit 3.x

import junit.framework.TestCase;

public class TestHelloWorld extends TestCase {

public void testSayHello() {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

assertEquals("Hello World!", result);

}

} 2. You make sure that the method name follows the pattern testXXX(). Following the testXXX naming convention is not strictly required, but it is strongly encouraged as a best practice.

50

TestHelloWorld program written with JUnit 3.x

import junit.framework.TestCase;

public class TestHelloWorld extends TestCase {

public void testSayHello() {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

assertEquals("Hello World!", result);

}

}3. You start the test by creating an instance of the HelloWorld class (the “object under test”),

51

TestHelloWorld program written with JUnit 3.x

import junit.framework.TestCase;

public class TestHelloWorld extends TestCase {

public void testSayHello() {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

assertEquals("Hello World!", result);

}

} 4. You execute the test by calling the method to test, no parameter is passing in this case.

52

TestHelloWorld program written with JUnit 3.x

import junit.framework.TestCase;

public class TestHelloWorld extends TestCase {

public void testSayHello() {

HelloWorld world = new HelloWorld();

String result = world.sayHello();

assertEquals("Hello World", result);

}

}5. To check the result of the test, you call an assertEquals method, which inherited from the base TestCase.

static void assertEquals([String message],expected,actual)

53

JUnit for Eclipse

54

Successful Test

55

Failed Test

Hello World. 多了一个句点符号

56

JUnit Methods

• assertEquals(x, y) – Test passes if x and y are equal– x and y can be primitives or any type with an appropriate equals

method– Three argument versions exist for floating point numbers

• assertFalse(b) – Test passes if boolean value b is false• assertTrue(b) – Test passes if boolean value b is true• assertNull(o) – Test passes if object o is null• assertNotNull(o) – Test passes if object o is not null• assertSame(ox, oy) – Test passes if ox and oy refer to the

same object • assertNotSame(ox, oy) – Test passes if ox and oy do not

refer to the same object

57

JUnit Test Runner Sequence

• Test runner( 测试运行程序 ) is given a list of test classes• For each test class

• Test runner produces a report

Create an instance of the test class

For each test*() method

Run setUp() method // code run before every test

Run test method steps and checks

If a check fails, an assertion is thrown and the testmethod fails

Run tearDown() method

58

Things to Notice

• The setUp() method ensures you entered the test method with a virgin set of objects.– Officially called “test fixtures( 测试固件 )”.

– It is just a some code you want run before every test

• The tearDown() method frees memory, prevents results of one test from affecting the next.– It will be run after every test case

• Only the first failure in a test method is reported– Don’t do too much in a single test

59

Implementing setUp() method

• Override setUp() to initialize the variables, and objects

• Since setUp() is your code, you can modify it any way you like (such as creating new objects in it)

• Reduces the duplication of code

60

Implementing the tearDown() method

• In most cases, the tearDown() method doesn’t need to do anything– The next time you run setUp(), your objects will be

replaced, and the old objects will be available for garbage collection

– Like the finally clause in a try-catch-finally statement, tearDown() is where you would release system resources (such as streams)

61

the Structure of a Test Method

• A test method doesn’t return a result

• If the tests run correctly, a test method does nothing

• If a test fails, it throws an AssertionFailedError

• The JUnit framework catches the error and deals with it; you don’t have to do anything

62

Test suites

• In practice, you want to run a group of related tests .To do so, running test suites.import junit.framework.Test;

import junit.framework.TestSuite;

public class AllTests {

public static Test suite() {

TestSuite suite = new TestSuite("Test for default package");

suite.addTestSuite(TestStringUtil.class);

suite.addTestSuite(TestHelloWorld.class)

return suite;

} }

63

Creating a test class in JUnit

• Define a subclass of TestCase • Override the setUp() method to initialize object(s) under

test. • Override the tearDown() method to release object(s) under

test. • Define one or more public testXXX() methods that exercise

the object(s) under test and assert expected results. • Define a static suite() factory method that creates a

TestSuite containing all the testXXX() methods of the TestCase.

• Optionally define a main() method that runs the TestCase in batch mode.

64

Start to Use it

• Download the latest version of JUnit from http://download.sourceforge.net/junit/

• Installation – unzip the junit.zip file – add junit.jar to the CLASSPATH.

65

the Framework of JUnit

66

the Framework of JUnit

Package junit.framework

67

the Framework of JUnit

Package junit.runner

68

the Framework of JUnit

A set of assert methods. Messages are only displayed when an assert fails.

69

the Framework of JUnit

A TestResult collects any errors or failures that occur during a test.

A TestFailure collects a failed test together with the caught exception.

70

the Framework of JUnit

A Test can be run and passed a TestResult.

71

the Framework of JUnit

A TestListener is apprised of events that occur during a test, including when the test begins and ends, along with any errors or failures.

72

the Framework of JUnit

A TestCase defines an environment (or fixture) that can be used to run multiple tests.

73

the Framework of JUnit

A TestSuite runs a collection of test cases, which may include other test suites. It is a composite of Tests.

74

the Framework of JUnit

A test runner is a user interface for launching tests.

BaseTestRunner is the superclass for all test runners.