TDD4ES · « By writing a test before implementing the item under test, attention is focussed on...

Workshop TDD4ES

28/06/2010

EP 1

Workshop

TDD4ES Test-Driven Development of Embedded software

28/06/2010

dr. ing. Jeroen Boydens

ing. Piet Cordemans

ing. Sille Van Landschoot

http://ep.khbo.be

iwt-090191

EP research group

2

http://ep.khbo.be/

Workshop TDD4ES

28/06/2010

EP 2

User committee TDD4ES IWT/090191

3

www.edna.eu www.tesco.be

www.qstar.be

www.marelec.com

www.unitron.be

www.summa.be

www.sirris.be

www.televic.com

www.newtec.eu

www.vandewiele.com

www.dspvalley.com www.kdg.be

www.kuleuven.be www.fmtc.be

www.iwt.be

Content

1. Test-Driven Development

2. TDD strategies on embedded

3. Development environment

4. DIY (Do It Yourself)

4

FMTC2009_TDD4ES.ppt

Workshop TDD4ES

28/06/2010

EP 3

Test-Driven Development

TDD cycle

1. Write failing test

• New behavior

• Minimal skeleton to get through compilation

2. Write code to pass test

• Minimal implementation

3. Refactor

• No new behavior

• Clean code

• Keep tests passing

5

Red

Green

Refactor


Advantages

1. Find bugs early

2. Quality

3. Confidence

4. Regression

5. Third party code

6. Final testing

Disadvantages

1. 2x code

increase development time

6

Lifecycle

Co

st

of

Re

pa

ir

Specification Programming ProductionDesign

1000

0

100

10

© B.W. Boehm, 1981

Software Engineering Economics

Investment

LifeCycleBenefit

NetReturn

Development

Development Quality Assurance

Conventional

TDD

© Müller & Padberg, 2003

Workshop TDD4ES

28/06/2010

EP 4


Testing = repetitive

Manual

1. Error – prone

2. Boring

3. Slow

7


Automated Testing Advantages

• Repeatable

• Documentation

• Regression testing

• Less debugging

• Smoke-bot

8

Workshop TDD4ES

28/06/2010

EP 5


Unit test framework

• Organization

• Execution

• Reuse

Many exist for C/C++ (>40)

minUnit, CppUnitLite, UnitTest++, GoogleTest, …

Write your own (TDD-style)

9


Unit test framework (xUnit)

10

Workshop TDD4ES

28/06/2010

EP 6


Testing process goals:

• Validation testing

• Software meets requirements

• Conformance specification

• Defect testing

• Incorrect behavior

11


Black box tests vs. White box tests

12

• Internal perspective • Implementation • Focus on developer • 20% coverage • Test Afterwards

• External perspective • Specification • Focus on end-user • 80% coverage • Test First

Also: customer tests, learning tests

Workshop TDD4ES

28/06/2010

EP 7


Write test first

• Inspiration

• Specifications

• Use cases

• Notes

• …

• Strategy

• Define interface

• Select values for test

« By writing a test before implementing the item under test, attention is focussed on the item’s interface and observable behavior » Beck (2003)

13


Testing pitfalls

14

• Broken window syndrome

– One broken test, which remains unresolved, leads quickly to many broken tests.

• Test cancer [Fowler]

– Removing broken tests is a bad habit.

Maintaining tests is necessary

«Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. » Dijkstra (1972)

Workshop TDD4ES

28/06/2010

EP 8


Legacy code

• No tests to start with

• Code is not fit to be tested

• Start small

• Find a testable section (= seam)

• Write a test

• Gradually add more tests

• Before each bug fix

• Write test that detects bug

• Fix the bug

15

Refactoring Integration

of tests


eXtreme Programming & Agile

• Simplicity is essential

– Do not try to anticipate future elements

– Never add functionality you do not need yet

• Pair Programming

– Two programmers working on the same machine

– The driver types and thinks tactically about the current issue

– The navigator thinks strategically about the bigger picture

– Switch back & forth

16

Workshop TDD4ES

28/06/2010

EP 9

Content





17

TDD on embedded

Obstacles

1. Memory footprint

2. Slow cycle (load to target)

3. Tools (C)

4. Hardware

1. Unavailable

2. Non-deterministic

3. Manual interaction

18

Workshop TDD4ES

28/06/2010

EP 10

TDD on embedded

Mocking hardware

• Software stub replacing the hardware

• Isolates the unit under test

• Mock object -> also validates interaction with the stub

19

TDD on embedded

Mocking hardware ..

1. Polymorphism

2. Preprocessor directives

3. Link-time polymorphism

4. Function pointers

20

Workshop TDD4ES

28/06/2010

EP 11

TDD on embedded

MCH-pattern

• Hardware independent (Model)

• Hardware aware (Conductor)

• Hardware specific (Hardware)

21

©Karlesky et al, 2006

(3-tier)

TDD on embedded

3 strategies

22

Host Target

Code

Program

Code

Test

Workshop TDD4ES

28/06/2010

EP 12

TDD on embedded

Test on target

23

+ Real hardware

• No cross-compiling

• Real drivers

- Slow deployment

- Memory issues

TDD on embedded

Test on host

24

+ Fast cycle

+ Hardware

• Unavailability

• Processing power

• No memory issues

- Driver verification

- Cross-compiling issues

Workshop TDD4ES

28/06/2010

EP 13

TDD on embedded

Test with testwrapper

25

+ Real hardware

• No cross-compiling

• Real drivers

- Slow deployment

- Extra code

TDD on embedded

3 strategies: wrapping up

26

• No single strategy is the all-encompassing solution • Each of the strategies has its own merits • The strategies can fill in at different times in the development cycle / process

©Grenning, 2004

Workshop TDD4ES

28/06/2010

EP 14

Content





27

Development environment

• Virtual machine

Admin rights

Slow

• IDE: Eclipse CDT

Open source

Customizable

Popular

• Unit test frameworks

• UnitTest++

• MinUnit

28

Workshop TDD4ES

28/06/2010

EP 15

Development environment

• EasyARM development board

– Mikroelektronika

– Temperature sensor (DS1820)

– 2x16 LCD

– RS232

– Drivers library (Keil compiler)

• Two versions (1.01 & 1.51)

– Hardware changes (switch on/off, …)

– Two different programming tools

29

Content





30

Workshop TDD4ES

28/06/2010

EP 16

Step 1

01_TDDonHost_TempLibrary

Temperature Library

– Stores internally a temperature value in Kelvin

– Offers an interface to get and set values in Kelvin, Celsius & Fahrenheit

– TDD without hardware involvement

– Getting acquainted with TDD

31

Step 1

32

Hello unit test framework: UnitTest++

RunTests.cpp -> Thermometer.cpp

Workshop TDD4ES

28/06/2010

EP 17

Step 1

33

• Grouping

• Setup & Teardown

• Assertions

Step 1

Some inspiration…

Kelvin = Celsius + 273,15

Celsius = (Fahrenheit - 32) * 5/9

KelvinTest

CelsiusTest

FahrenheitTest

Celsius2KelvinTest

Kelvin2CelsiusTest

Fahrenheit2KelvinTest

Kelvin2FahrenheitTest

Fahrenheit2CelsiusTest …

34

Workshop TDD4ES

28/06/2010

EP 18

Step 2

02_TDDonTarget_TempLibrary

• MinUnit: Minimalistic (2 C macro’s)

• Adapted (minunit.h) -> Reporting to UART

• Putty: check COM port / Baudrate 38400

• Hint: press CTRL + space + space

35

Step 2

02_TDDonTarget_TempLibrary

Write your TEST functions

RUN your tests

Insert run_all_tests() in the main

See failing tests in PuTTY

36

Workshop TDD4ES

28/06/2010

EP 19

Step 3

03_TDDonTarget_MockTempSensor

37

Step 3


1. Keep hardware independent code in TempSensor

2. Make the hardware dependent methods virtual

3. Move the hardware implementation to TempSensorReal

4. Implement stubs in TempSensorMock

5. Add helper functions in TempSensorMock

38

Workshop TDD4ES

28/06/2010

EP 20

Step 3


39

Step 4

04_TDDonHost_MockedHardware

Add preprocessor directives to extract the target code from the host code.

Minunit.h

TestTempSensor.h

Main.cpp 40

Workshop TDD4ES

28/06/2010

EP 21

Step 5

05_TDDWrapper

Implementing the testwrapper test

1. Implement Tempsensor_getTempDouble

Hint: Look at Tempsensor_getTemp

2. Add command to the testwrapper runner

Hint: double2uart

3. Test in putty

Hint: SI_050_000_000; //result: 25

41

Step 5

05_TDDWrapper

TestWrapper Application (proof of concept)

42

Workshop TDD4ES

28/06/2010

EP 22

TDD4ES Workshop

Remarks ?

43

Literature

• K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2003.

• K. Beck and C. Andres. Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley Professional, 2004.

• B.W. Boehm. Software Engineering Economics (Prentice-Hall Advances in Computing Science & Technology Series). Prentice Hall PTR, October 1981.

• J. Boydens, P. Cordemans, and E. Steegmans. Test-driven development of embedded software. In European Conference on the Use of Modern Information and Communication Technologies, 2010.

• E. Dijkstra. The humble programmer. Commun. ACM, 15(10):859866, 1972.

• M. Feathers. Working Effectively with Legacy Code. Prentice Hall PTR, 2005.

• J. Grenning. Progress before hardware. Agile Alliance Newsletter, 4:74_79, 2004.

• M. Karlesky, W. Bereza, and C. Erickson. Effective test driven development for embedded software. In IEEE 2006 Electro/Information Technology Conference, 2006.

• M. Müller and F. Padberg. About the return on investment of test-driven development. In International Workshop on Economics-Driven Software Engineering Research EDSER-4, 2003.

44

TDD4ES · « By writing a test before implementing the item under test, attention is focussed on...

Documents

Transcript of TDD4ES · « By writing a test before implementing the item under test, attention is focussed on...