Workshop TDD4ES
28/06/2010
EP 1
Workshop
TDD4ES Test-Driven Development of Embedded software
28/06/2010
dr. ing. Jeroen Boydens
ing. Piet Cordemans
ing. Sille Van Landschoot
http://ep.khbo.be
iwt-090191
EP research group
2
Workshop TDD4ES
28/06/2010
EP 2
User committee TDD4ES IWT/090191
3
www.edna.eu www.tesco.be
www.qstar.be
www.marelec.com
www.unitron.be
www.summa.be
www.sirris.be
www.televic.com
www.newtec.eu
www.vandewiele.com
www.dspvalley.com www.kdg.be
www.kuleuven.be www.fmtc.be
www.iwt.be
Content
1. Test-Driven Development
2. TDD strategies on embedded
3. Development environment
4. DIY (Do It Yourself)
4
Workshop TDD4ES
28/06/2010
EP 3
Test-Driven Development
TDD cycle
1. Write failing test
• New behavior
• Minimal skeleton to get through compilation
2. Write code to pass test
• Minimal implementation
3. Refactor
• No new behavior
• Clean code
• Keep tests passing
5
Red
Green
Refactor
Test-Driven Development
Advantages
1. Find bugs early
2. Quality
3. Confidence
4. Regression
5. Third party code
6. Final testing
Disadvantages
1. 2x code
increase development time
6
Lifecycle
Co
st
of
Re
pa
ir
Specification Programming ProductionDesign
1000
0
100
10
© B.W. Boehm, 1981
Software Engineering Economics
Investment
LifeCycleBenefit
NetReturn
Development
Development Quality Assurance
Conventional
TDD
© Müller & Padberg, 2003
Workshop TDD4ES
28/06/2010
EP 4
Test-Driven Development
Testing = repetitive
Manual
1. Error – prone
2. Boring
3. Slow
7
Test-Driven Development
Automated Testing Advantages
• Repeatable
• Documentation
• Regression testing
• Less debugging
• Smoke-bot
8
Workshop TDD4ES
28/06/2010
EP 5
Test-Driven Development
Unit test framework
• Organization
• Execution
• Reuse
Many exist for C/C++ (>40)
minUnit, CppUnitLite, UnitTest++, GoogleTest, …
Write your own (TDD-style)
9
Test-Driven Development
Unit test framework (xUnit)
10
Workshop TDD4ES
28/06/2010
EP 6
Test-Driven Development
Testing process goals:
• Validation testing
• Software meets requirements
• Conformance specification
• Defect testing
• Incorrect behavior
11
Test-Driven Development
Black box tests vs. White box tests
12
• Internal perspective • Implementation • Focus on developer • 20% coverage • Test Afterwards
• External perspective • Specification • Focus on end-user • 80% coverage • Test First
Also: customer tests, learning tests
Workshop TDD4ES
28/06/2010
EP 7
Test-Driven Development
Write test first
• Inspiration
• Specifications
• Use cases
• Notes
• …
• Strategy
• Define interface
• Select values for test
« By writing a test before implementing the item under test, attention is focussed on the item’s interface and observable behavior » Beck (2003)
13
Test-Driven Development
Testing pitfalls
14
• Broken window syndrome
– One broken test, which remains unresolved, leads quickly to many broken tests.
• Test cancer [Fowler]
– Removing broken tests is a bad habit.
Maintaining tests is necessary
«Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. » Dijkstra (1972)
Workshop TDD4ES
28/06/2010
EP 8
Test-Driven Development
Legacy code
• No tests to start with
• Code is not fit to be tested
• Start small
• Find a testable section (= seam)
• Write a test
• Gradually add more tests
• Before each bug fix
• Write test that detects bug
• Fix the bug
15
Refactoring Integration
of tests
Test-Driven Development
eXtreme Programming & Agile
• Simplicity is essential
– Do not try to anticipate future elements
– Never add functionality you do not need yet
• Pair Programming
– Two programmers working on the same machine
– The driver types and thinks tactically about the current issue
– The navigator thinks strategically about the bigger picture
– Switch back & forth
16
Workshop TDD4ES
28/06/2010
EP 9
Content
1. Test-Driven Development
2. TDD strategies on embedded
3. Development environment
4. DIY (Do It Yourself)
17
TDD on embedded
Obstacles
1. Memory footprint
2. Slow cycle (load to target)
3. Tools (C)
4. Hardware
1. Unavailable
2. Non-deterministic
3. Manual interaction
18
Workshop TDD4ES
28/06/2010
EP 10
TDD on embedded
Mocking hardware
• Software stub replacing the hardware
• Isolates the unit under test
• Mock object -> also validates interaction with the stub
19
TDD on embedded
Mocking hardware ..
1. Polymorphism
2. Preprocessor directives
3. Link-time polymorphism
4. Function pointers
20
Workshop TDD4ES
28/06/2010
EP 11
TDD on embedded
MCH-pattern
• Hardware independent (Model)
• Hardware aware (Conductor)
• Hardware specific (Hardware)
21
©Karlesky et al, 2006
(3-tier)
TDD on embedded
3 strategies
22
Host Target
Code
Program
Code
Test
Workshop TDD4ES
28/06/2010
EP 12
TDD on embedded
Test on target
23
+ Real hardware
• No cross-compiling
• Real drivers
- Slow deployment
- Memory issues
TDD on embedded
Test on host
24
+ Fast cycle
+ Hardware
• Unavailability
• Processing power
• No memory issues
- Driver verification
- Cross-compiling issues
Workshop TDD4ES
28/06/2010
EP 13
TDD on embedded
Test with testwrapper
25
+ Real hardware
• No cross-compiling
• Real drivers
- Slow deployment
- Extra code
TDD on embedded
3 strategies: wrapping up
26
• No single strategy is the all-encompassing solution • Each of the strategies has its own merits • The strategies can fill in at different times in the development cycle / process
©Grenning, 2004
Workshop TDD4ES
28/06/2010
EP 14
Content
1. Test-Driven Development
2. TDD strategies on embedded
3. Development environment
4. DIY (Do It Yourself)
27
Development environment
• Virtual machine
Admin rights
Slow
• IDE: Eclipse CDT
Open source
Customizable
Popular
• Unit test frameworks
• UnitTest++
• MinUnit
28
Workshop TDD4ES
28/06/2010
EP 15
Development environment
• EasyARM development board
– Mikroelektronika
– Temperature sensor (DS1820)
– 2x16 LCD
– RS232
– Drivers library (Keil compiler)
• Two versions (1.01 & 1.51)
– Hardware changes (switch on/off, …)
– Two different programming tools
29
Content
1. Test-Driven Development
2. TDD strategies on embedded
3. Development environment
4. DIY (Do It Yourself)
30
Workshop TDD4ES
28/06/2010
EP 16
Step 1
01_TDDonHost_TempLibrary
Temperature Library
– Stores internally a temperature value in Kelvin
– Offers an interface to get and set values in Kelvin, Celsius & Fahrenheit
– TDD without hardware involvement
– Getting acquainted with TDD
31
Step 1
32
Hello unit test framework: UnitTest++
RunTests.cpp -> Thermometer.cpp
Workshop TDD4ES
28/06/2010
EP 17
Step 1
33
• Grouping
• Setup & Teardown
• Assertions
Step 1
Some inspiration…
Kelvin = Celsius + 273,15
Celsius = (Fahrenheit - 32) * 5/9
KelvinTest
CelsiusTest
FahrenheitTest
Celsius2KelvinTest
Kelvin2CelsiusTest
Fahrenheit2KelvinTest
Kelvin2FahrenheitTest
Fahrenheit2CelsiusTest …
34
Workshop TDD4ES
28/06/2010
EP 18
Step 2
02_TDDonTarget_TempLibrary
• MinUnit: Minimalistic (2 C macro’s)
• Adapted (minunit.h) -> Reporting to UART
• Putty: check COM port / Baudrate 38400
• Hint: press CTRL + space + space
35
Step 2
02_TDDonTarget_TempLibrary
Write your TEST functions
RUN your tests
Insert run_all_tests() in the main
See failing tests in PuTTY
36
Workshop TDD4ES
28/06/2010
EP 19
Step 3
03_TDDonTarget_MockTempSensor
37
Step 3
03_TDDonTarget_MockTempSensor
1. Keep hardware independent code in TempSensor
2. Make the hardware dependent methods virtual
3. Move the hardware implementation to TempSensorReal
4. Implement stubs in TempSensorMock
5. Add helper functions in TempSensorMock
38
Workshop TDD4ES
28/06/2010
EP 20
Step 3
03_TDDonTarget_MockTempSensor
39
Step 4
04_TDDonHost_MockedHardware
Add preprocessor directives to extract the target code from the host code.
Minunit.h
TestTempSensor.h
Main.cpp 40
Workshop TDD4ES
28/06/2010
EP 21
Step 5
05_TDDWrapper
Implementing the testwrapper test
1. Implement Tempsensor_getTempDouble
Hint: Look at Tempsensor_getTemp
2. Add command to the testwrapper runner
Hint: double2uart
3. Test in putty
Hint: SI_050_000_000; //result: 25
41
Step 5
05_TDDWrapper
TestWrapper Application (proof of concept)
42
Workshop TDD4ES
28/06/2010
EP 22
TDD4ES Workshop
Remarks ?
43
Literature
• K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2003.
• K. Beck and C. Andres. Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley Professional, 2004.
• B.W. Boehm. Software Engineering Economics (Prentice-Hall Advances in Computing Science & Technology Series). Prentice Hall PTR, October 1981.
• J. Boydens, P. Cordemans, and E. Steegmans. Test-driven development of embedded software. In European Conference on the Use of Modern Information and Communication Technologies, 2010.
• E. Dijkstra. The humble programmer. Commun. ACM, 15(10):859866, 1972.
• M. Feathers. Working Effectively with Legacy Code. Prentice Hall PTR, 2005.
• J. Grenning. Progress before hardware. Agile Alliance Newsletter, 4:74_79, 2004.
• M. Karlesky, W. Bereza, and C. Erickson. Effective test driven development for embedded software. In IEEE 2006 Electro/Information Technology Conference, 2006.
• M. Müller and F. Padberg. About the return on investment of test-driven development. In International Workshop on Economics-Driven Software Engineering Research EDSER-4, 2003.
44
Top Related