Test Patterns in Java - NetBeans...return...
Transcript of Test Patterns in Java - NetBeans...return...
2006 JavaOneSM Conference | Session BOF0220 |
Test Patterns in Java
Jaroslav Tulach, Jesse Glick, Miloš Kleint
Sun Microsystemshttp://www.netbeans.org
2006 JavaOneSM Conference | Session BOF0220 | 2
Automated Testing
Learn why to invest in automated testing and let us give you overview of the wildest test patterns that NetBeans project found useful.
Test is a Kind of Documentation
2006 JavaOneSM Conference | Session BOF0220 | 3
Agenda
What is the problem?Why do automated tests help?How to write tests?Questions and Answers
2006 JavaOneSM Conference | Session BOF0220 | 4
The Quality of an Application
http://openide.netbeans.org/tutorial/testpatterns.html
• quality = implementation ∆ specification • quality may not be the right term
• maybe term confidence is better • regressions cost too much • the "amoeba" model
2006 JavaOneSM Conference | Session BOF0220 | 5
The Amoeba Model
http://openide.netbeans.org/tutorial/testpatterns.html
2006 JavaOneSM Conference | Session BOF0220 | 6
The Amoeba Model
http://openide.netbeans.org/tutorial/testpatterns.html
2006 JavaOneSM Conference | Session BOF0220 | 7
Why write automated tests?● Make sure the code works● Things do not get wrong
● concurrent access ● garbage collection and finalizers
● Algorithms and data structures are sane● fast and small
● Mistakes do not reappear● race conditions● deadlocks
2006 JavaOneSM Conference | Session BOF0220 | 8
Do tests influence code?
● methods to verify state ● ways to prevent asynchronous behavior ● separation to units
● using lookup and discovery● code against interfaces not implementation
● Java is not single threaded● tests may need cooperation with code
http://www.netbeans.org/download/dev/javadoc/orgopenideutil/org/openide/util/Lookup.html
2006 JavaOneSM Conference | Session BOF0220 | 9
Test Patterns in Java
● Testing exception states● Hard to invent● Easy to copy and use● Premise: Test everything
2006 JavaOneSM Conference | Session BOF0220 | 1 0
Demo ApplicationExtraordinary Complex and Useless Calculator
2006 JavaOneSM Conference | Session BOF0220 | 1 1
Code Against Interfaces
http://openide.netbeans.org/tutorial/testpatterns.html
public abstract class DialogDisplayer { public abstract Object notify(javax.swing.JOptionPane what); public static DialogDisplayer getDefault() { return Lookup.getDefault().lookup(DialogDisplayer.class); // or in Mustang return java.util.ServiceLoader.load(DialogDisplayer.class).iterator.next(); }}
● define abstract interfaces and “lookup” implementation:
● real implementation in the application● “mock object” in the test
2006 JavaOneSM Conference | Session BOF0220 | 1 2
Setup Testing Environment
http://openide.netbeans.org/tutorial/testpatterns.html
public class MyTest extends org.netbeans.nbjunit.NbTestCase { protected void setUp() { org.netbeans.nbjunit.MockServices.setServices(MockDialogDisplayer.class); }}public class MockDialogDisplayer extends DialogDisplayer { public static JOptionPane lastPane; public static Object toReturn; public Object notify(JOptionPane p) { lastPane = p; return toReturn; }}
● fill lookup with own “mock” implementation● replace UI with headless displayer
2006 JavaOneSM Conference | Session BOF0220 | 1 3
Memory Leak Tests
● Memory management is important aspect of “amoeba”● easy to regress● hard to describe and verify
● NbTestCase.assertGC(String, Reference);● Cooperates with Insane library
NetBeans JUnit Extensions http://xtest.netbeans.org/servlets/ProjectDocumentList
class SetupAndGCTest extends NbTestCase { public void testTheyCanDisappear() { Calculator same = Calculator.create("powerful"); WeakReference ref = new WeakReference(same); same = null; assertGC("Caches can be GCed", ref); }}
2006 JavaOneSM Conference | Session BOF0220 | 1 4
Memory Leak Tests II● Controlling size of data structures
● hard to describe and verify● NbTestCase.assertSize(String, int, Object);
● virtual size of data structure trees● 8/16/16/24 bytes● Based on Insane library
● How big is an empty string?
NetBeans JUnit Extensions http://xtest.netbeans.org/servlets/ProjectDocumentList
Object dataStructure = “”; assertSize( "Less than 16bytes?", 16, dataStructure);
class SizeRegisterFailingTest extends NbTestCase: Calculator calc = Calculator.create("powerful"); Calculator.Register reg = calc.getRegisters().get(0); assertSize("Register contains one integer field => 16 bytes. Will it fit?", Collections.singleton(reg), 16,
new Object[] { calc }); // last array allows to exclude some objects
2006 JavaOneSM Conference | Session BOF0220 | 1 5
Speed Test● NbTestSuite.speedSuite and linearSpeedSuite● Size of the test● Comparing speed of various algorithms● Need to eliminate influence of GC and hotspot
http://openide.netbeans.org/tutorial/testpatterns.html
protected void setUp () { calc.ensureRegistersSize(getTestNumber()); list = calc.getRegisters();}public void test10() { doTest (); }public void test100() { doTest (); }public void test1000() { doTest (); }public void test10000 () { doTest (); }private void doTest () { for (int i = 0; i < 10000; i++) { list.get(size / 2);}
2006 JavaOneSM Conference | Session BOF0220 | 1 6
Randomized Tests
http://openide.netbeans.org/tutorial/testpatterns.html
Test for the unknown● Hard to test where “amoeba” does more than expected● Use random sequence of operations:
int op = random.nextInt();
switch (op) {
case 0: list.add(new Integer(random.nextInt()); break;
case 1: list.removeAt(random.nextInt(list.size())); break;
case ....
}
● Record the seed for case of failureseed = System.currentTimeMillis();
random = new Random(seed);
2006 JavaOneSM Conference | Session BOF0220 | 1 7
Deadlock Test I
http://openide.netbeans.org/tutorial/testpatterns.html
● Deadlocks are biggest contributors of amoeba's shaking● no reasonable theory● hard to simulate from outside
2006 JavaOneSM Conference | Session BOF0220 | 1 8
Foreign Code In Critical Section
http://openide.netbeans.org/tutorial/apidesign.html
Deadlock prone code
● Calling foreign code under lock leads to deadlocks● Sometimes hard to prevent private HashSet allCreated = new HashSet(); public synchronized JLabel createLabel() { JLabel l = new JLabel(); allCreated.add(l); return l; }
● java.awt.Component grabs AWT tree lock● HashSet.add calls Object.equals
2006 JavaOneSM Conference | Session BOF0220 | 1 9
Deadlock Test II
http://openide.netbeans.org/tutorial/testpatterns.html
● Regression test● Reading thread dumps
● Reproducing thread locks● Need more threads● Using time out for the test
● Block them in the right moment● overriding virtual methods● introducing artificial hooks
2006 JavaOneSM Conference | Session BOF0220 | 20
Race Condition Test
http://openide.netbeans.org/tutorial/testpatterns.html
● Similar to deadlock tests● Missing thread dump information● Need more threads● Block them in the right moment
● overriding virtual methods● introducing artificial hooks
if (System.getProperty(“simulate.race.condition.2”) != null) {
Thread.sleep(500);
}
2006 JavaOneSM Conference | Session BOF0220 | 21
The Beauty of Logging
http://openide.netbeans.org/tutorial/testpatterns.html
● Analyze random failuresprotected Level logLevel() { return Level.FINE; }
CharSequence msgs = Log.enable(“logname”, Level.ALL);
● Natural in source codeLogger LOG = Logger.getLogger(“logname”);
LOG.finest(“simulate.race.condition.2”);
● Tests can plug their own Handlersclass TestHandler extends Handler {
public void publish(LogRecord r) {
if (r.getMessage().equals((“simulate.race.condition.2”))
Thread.sleep(500);
}
}
2006 JavaOneSM Conference | Session BOF0220 | 22
Flow Control using Logging
http://openide.netbeans.org/tutorial/testpatterns.html
● Automatic breakpoints● Allows replay of captured log files
● plugs in own handler● blocks and wake the threads up
● Complexity hidden behind simple NetBeans JUnit API:
Log.controlFlow(
“THREAD: main MSG: simulate.race.*
“THREAD: second MSG: cause problems”
);
2006 JavaOneSM Conference | Session BOF0220 | 23
Summary
● Prevent Amoeba from shaking● from one side – e.g. what is not working
● Automate testing, its cheaper● Treat tests as a complement to documentation● Use our test patterns● Invent new test patterns
2006 JavaOneSM Conference | Session BOF0220 | 24
References
● BOF2559 – Thu 19:30 Gateway 105● Discovery and Dependency Injection Patterns in
Modular Architectures● TS6218 – Fri 14:30 Gateway 102/103
● How to Write APIs That Will Stand the Test of Time● http://openide.netbeans.org/tutorial/testpatterns.html● download from http://jpackage.org● download from http://xtest.netbeans.org
2006 JavaOneSM Conference | Session BOF0220 | 25
Q&AJesse GlickMiloš Kleint
Jaroslav Tulach