1-4244-1468-7/07/$25.00 ©2007 IEEE 486

Mechatronic Software Testing Olivera Iskrenović-Momčilović1, Aca Micić2

Abstract - The paper describes mechtronic software testing

techniques. Testing is different from common testing and includes special features of mechtronic systems. It may be put into effect with the aim of improving quality, assessing reliability, checking and conforming correctness. Various adapted techniques may be employed for the purpose, such as, for example, the white box technique or the black box technique when correctness testing, enduarance and stress testing in relability testing or the usage of ready programs for performance testing.

Keywords – software testing, mechatronic system, real-time software

I.INTRODUCTION

Software testing is the process of executing a program or system with the intent of finding errors [1]. Or, it involves any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. Software is not unlike other physical processes where inputs are received and outputs are produced. Where software differs is in the manner in which it fails. Most physical systems fail in a fixed (and reasonably small) set of ways. By contrast, software can fail in many bizarre ways. Detecting all of the different failure modes for software is generally infeasible.

Testing is an integral part in software development. It is broadly deployed in every phase in the software development cycle. Typically, more than 50% percent of the development time is spent in testing. Testing is usually performed for the following purposes [2]: - To improve quality, - For Verification & Validation (V&V).

We can not test quality directly, but we can test related factors to make quality visible. Quality has three sets of factors -- functionality, engineering, and adaptability. These three sets of factors can be thought of as dimensions in the software quality space. Each dimension may be broken down into its component factors and considerations at successively lower levels of detail. Table I illustrates some of the most frequently cited quality considerations.

1dr Olivera I. Iskrenović–Momčilović is with the Research and Development Institute Irin, Bulevar Cara Konstantina 82-86, 18000 Niš, Serbia, Email: oljaisk@yahoo.com

2dr Aca Micić is with the Mechanical Faculty, University of Niš, Aleksandra Medvedeva 14, 18000 Niš, Serbia, Email: acamicic@masfak.ni.ac.yu

TABELE I FACTORS OF SOFTWARE QUALITY

Functionality (exterior quality)

Engineering (interior quality)

Adaptability (future quality)

Corectness Efficiency Flexibility Reliability Testability Reusability Usability Documentation Maintainability Integrity Structure

Good testing provides measures for all relevant factors. The importance of any particular factor varies from application to application. Any system where human lives are at stake must place extreme emphasis on reliability and integrity. In the typical business system usability and maintainability are the key factors, while for a one-time scientific program neither may be significant. Our testing, to be fully effective, must be geared to measuring each relevant factor and thus forcing quality to become tangible and visible.

II. TECNIQUES OF SOFTWARE TESTING

The design of software testing can be a challenging process. However software engineers often see testing as an after thought, producing test cases that feel right but have little assurance that they are complete. The objective of testing is to have the highest likelihood of finding the most errors with a minimum amount of timing and effort. A large number of test case design techniques have been developed that offer the developer with a systematic approach to testing. Techniques offer an approach that can ensure the complteness of tests and offer thi highest likelihood for uncoveribg errors in software.

Any engineering product can be tested in two ways [1]: • Blak box testing – knowing the specified functions that

the product has been designed to perform, tests can be performed that show that each function is fully operation.

• White box testing – knowing the internal workings of a product, test can be performed to see if they jell.

487

Fig. 1. Black box testing

Black box testing (Fig. 1) relates to the tests that are performed at the software interface. Although they are designed identify errors, black box tests are used to demonstrate taht software functionals are operational. That inputs are sorrectly accepted and the output is correctly produced. A black box test considers elements of the system with little interest in the international logical arrangement of the software.

Fig. 2. White box testing

White box testing (Fig. 2) involves a closer examination of procedural detail. Logical paths tgrough the software are considered by probiding test cases that exercise particular sets of conditions and/or loops. The status of the system can be identified at diverse points to establish if the expected status matches the actual status.

Using black box testing, testers examine the high-level design and the customer requirements specification to plan the test cases to ensure the code does what it is intended to do. Functional testing involves ensuring that the functionality specified in the requirement specification works. System testing involves putting the new program in many different environments to ensure the program works in typical customer environments with various versions and types of operating systems and/or applications. System testing is testing conducted on a complete, integrated system to evaluate the system compliance with its specified requirements [3]. Because system test is done with a full system implementation and environment, several classes of testing can be done that can examine non-functional properties of the system. It is best

when function and system testing is done by an unbiased, independent perspective (e.g. not the programmer) [4]. • Stress testing

Stress testing conducted to evaluate a system or component at or beyond the limits of its specification or requirement [3]. • Performance testing

Performance testing conducted to evaluate the compliance of a system or component with specified performance requirements [3]. • Usability testing

Usability testing conducted to evaluate the extent to which a user can learn to operate, prepare inputs for, and interpret outputs of a system or component [3]. While stress and usability testing can be and is often automated, usability testing is done by human-computer interaction specialists that observe humans interacting with the system.

III. MECHATRONIC REAL-TIME SOFTWARE

The dominant implementation medium as well as primary medium of expression for contemporary mechatronic systems is real-time software. Real-time software is a subdomain of software generally with distinct requirements and characteristics. While implementation media are subject to change, software is so dominant now that any design methodologies that are used must allow for the effective implementation of control systems using real-time software.

Real-time software differs from conventional software in that its results must not only be numerically and logically correct, they must also be delivered at the correct time. A design corollary following from this definition is that real-time software must embody the concept of duration, which is not part of conventional software. Duration is such an important concept that it will be the focus of the documentation technique used as the heart of a design methodology for producing reliable control system software.

The real-time software used in most mechanical system control is also safety critical. Software malfunctions can result in serious injury and/or significant property damage. Basic set of software testing design principles is: • Documentation should not be an afterthought. • Software quality assurance practices and standards

should be established. • Test designs should be kept simple. • Ways to get information about error should be designed

into the software from the beginning. • The software should be subjected to extensive testing

and formal analysis at the module and software level; system testing alone is not adequate.

In particular, it was determined that a number of these problems were associated with asynchronous operations which, while uncommon in conventional software, are the heart and soul of real-time software. Asynchronous operations arise from the interaction of the software with the physical system under control. In most cases, the physical system is made up of many components, all operating simultaneously.

Because of asynchronous operation, it becomes impossible to predict when groups of program statements will execute

Inputs causing anomalous behavior

System

Input test data

Output test results

Outputs which reveal the presence of defects

O

I

Test data

Component code

Test outputs

Tests

488

relative to other groups, as it is in synchronous software. Errors that depend on a particular execution order will only show up on a statistical basis, not predictably. Thus, the testing technique of repeating execution until the source of an error is found, which is so effective for synchronous (conventional) software, will not work for this class of real-time error.

IV. NASTY MECHATRONIC REAL-TIME SOFTWARE PROPERTIES

Software complexity tends to increase exponentially with size. What starts out as a small, manageable project can expand into something which literally never gets finished. To compound this problem, the economics of software-based products are such that over the lifetime of the product, maintenance costs dominate the total expense.

Both the complexity and the asynchronous nature of real-time software lead to situations in which errors can remain dormant for long time periods, waiting to do their dirty work at the most inopportune time.The antidotes to these problems flow from adopting an engineering perspective such that code writing (programming) should not be viewed as a creative activity!

On the engineering side, a method must be found to describe in detail how the control software should behave (a specification). This enables the needed engineering review function and enables the act of code production to become much more routine since the creative engineering part has been done at the descriptive level.

The second way of avoiding at least some of the complexity problems is to modularize the software and its production process. In this way, specific pieces of software are directly connected to certain machine functionality. System malfunctions can thus be traced to the relevant software more quickly, system changes can be made with reasonable assurances that the changes will not adversely affect some unexpected part of the machine, and individual software elements can be tested more easily. Modularization is also important in achieving reasonable delivery times for large projects. The modules can be worked on simultaneously by different people (perhaps in different parts of the world) and integrated into the final control system as they are produced.

V. SOFTWARE IN THE LOOP

We support setup, integration, application and maintenance of software in the loop (SiL) environment for the development of control software for mechatronic products.

Control software governs the behavior of the mechatronic hardware (mechanics, hydraulics, electrics, ...) of mechatronic systems. Experience shows, a tight integration of development and validation is crucial for the efficient development of high-quality control software.

To provide the control software engineers with the means to easily run versions of the control software on their laptops or PCs using simulated mechatronic hardware.

Benefit for the design process are:

• Early detection of errors and design flaws, especially in the control software

• Dimensioning, application, and optimization of control software parameters. As a result, more mature designs are transferred to later design and test stages, e.g. hardware in the loop (HiL), test rig and on-road trials.

Compared to tests using HiL, SiL tests have the following advantages: • Agility: HiL facilities are typically limited and

expensive resources. Moreover, the process chain from the developer's laptop to the mechatronic hardware in the HiL is relatively long. This results in unwanted delay (up to several days) between development of control software and HiL test. In a SiL environment, there are virtually no delays.

• No real-time requirements for the simulation code: In contrast to HiL, SiL-based testing does not require that the plant is simulated in real-time. This simplifies the modeling task for SiL considerably. Moreover, when choosing the level of detail for a SiL model, real time is not an issue. On the one hand, a SiL model may be much more complex, which enables new applications such as simulation-based dimensioning, optimization, and application of a mechatronic system, including failure simulation and validation of function quality based on one and the same model. On the other hand, simple models may produce simulations running many times faster than real-time, allowing for comprehensive logic tests with fast turn-around.

Despite these advantages, SiL-based testing cannot replace the HiL test completely. Low-level processes are not modeled in detail in a typical SiL environment, and are therefore beyond the scope of SiL-based testing.

VI. ONE SIMPLE EXAMPLE OF SIL

For instance, take a resistor and an LED in series, hook one end to 5V and one end to ground and see if that works.

We have tested both our hardware and software common. If the LED still doesn't light, attach an oscilloscope probe to the digital output where the LED circuit is connected. We may see that you are driving that pin alternately high and low like we wanted to but the LED still doesn't light. Well, maybe our circuit and software is working all right after all. Maybe it's our eyes that are fooling you - literally. If you blink an LED real fast you can't see it. You need to write a little software routine that creates a delay. void main () { LED_on(); Delay(); LED_off(); Delay(); LED_on(); Delay(); LED_off(); Delay(); }

489

Whenever we go to sit down and work on our project, or next year if we go to use another processor, always do this blinking LED test just to make sure things are working at some level. We might add this blinking LED permanently to our project just to check our sanity every time you power up. If the blinking LED isn't working, go back to something that does work. Separate the hardware from the software and test each piece independently. Go back through all the steps here and get to some point we know works before we try to debug the part that doesn't work.

VII. CONCLUSIONS

The paper describes mechtronic software testing techniques. Testing is different from common testing and includes special features of mechtronic systems. It may be put into effect with the aim of improving quality, assessing reliability, checking and conforming correctness. Various adapted techniques may be employed for the purpose, such as, for example, the white box technique or the black box

technique when correctness testing, enduarance and stress testing in relability testing or the usage of ready programs for performance testing.

REFERENCES

[1] G. J. Myers, The Art of Software Testing, John Wiley, New York, 1979.

[2] C. Kaner, J. Bach, Lessons Learned in Software Testing, John Wiley and Sons, 2002.

[3] IEEE Standard 610.12 – 1990, IEEE Standard Glossary of Software Engineering Terminology, 1990.

[4] A. Bertolino, Software Testing, IEEE SWEBOK Trial Version 1.0, 2001.

[5] D. M. Auslander, J. R. Ridgely, J. D. Ringgenberg, Introduction to Mechatronics: Using Software to Operate Mechanical Systems, Prentice Hall, 2002.

[6] D.M. Auslander, J.R. Ridgely, J.D. Ringgenberg, Control Software for Mechanical Systems: Object-Oriented Design in a Real-Time World, Prentice Hall, 2002