1. Developement of Automated Tests for native SyncML clients January 27 th , 2009 Andrea Scova

2. Agenda

Introduction

3. Building an Automated Test for native SyncML clients 4. Automated Tests Maintenance 5. Conclusions 6. Questions & Answers 7. What do Automated Tests check? The software is not breaking:

between a release and another

8. after a commit of a single piece of code 9. (e.g. a fix for a synclet) Introduction 10. Why Automated Tests are useful? They help DEV & QA to discover in a QUICK way errors in the code (tool for regression testing) BUT You cannot get rid of QA

(skipping manual regression testing is NOT an option)

Introduction 11. Introduction Why Automated Tests for native SyncML clients are important?

number of devices is huge

12. costs (devices, QA time) 13. simulation of native SyncML client based on server's log 14. How Automated Tests work (THEORY)

simulate a Native SyncML client

15. compare expected behaviour of the server against its actual behaviour simulated environment Building an Automated Test for native SyncML clients = 16. Building an Automated Test for native SyncML clients Automated Tests for SyncML clients logic (THEORY) Structured in 2 sessions:

the client sends its capabilities and all items to the server (the server has no items)

17. the server sends back the same items to the client (after updating the status) 18. NOTE: one syncsource per time 19. Building an Automated Test for native SyncML clients Setting up the environment (PRACTICE)

install thetest-ds.s4jmodule

20. catch the full logs 21. create the items (single syncsource) on the real device 22. all available fields on the device should be filled 23. include symbols and language specific chars (e.g. &, @, $, %, , , , , , ) 24. SYNC 25. Building an Automated Test for native SyncML clients Extracting the log's content (PRACTICE)

isolate the SyncML messages before the SYNCLET processing phase

26. create single .xml files 27. createheader.properties 28. from HTTP header (e.g. user-agent: Nokia SyncML HTTP Client) 29. Building an Automated Test for native SyncML clients Editing the SyncML messages replace:

deviceID ->syncml-phone

30. server-URI ->http://localhost:8080/funambol/ds 31. sessionID 32. Building an Automated Test for native SyncML clients Editing the SyncML messages 33. Building an Automated Test for native SyncML clients Documentation What the Automated Tests do (MANDATORY)

ReadMe.txt

Initial state: Server with no contacts Step performed: 1. (1-4) Slow sync. The mobile (a Nokia N70) sends 5 contacts to the server. Device capabilities are sent as well in this first step. Input synclets are tested in this step: . NokiaXin.bsha. Processes the incoming vcard items and adds the missing tokens. b. Handles large objects. c. Replace the property X-EPOCSECONDNAME with NICKNAME 2. (5-8) Fast sync: contacts status and last_timestamp are updated. The server sends all the contacts back to the device. Output synclets are tested in this step: . NokiaS60out.bsha. Moves some TEL properties to the end of the VCARD in a special orderbecausethe devices don't behave properly at update time, otherwise. b. Removes TYPE information from PHOTO token because the phones of the series 60 are not able to understand this information rightly. c. Replaces the TEL;CAR;VOICE token into TEL;VOICE;CAR. d. Replacethe property NICKNAME with X-EPOCSECONDNAME. 34. Directory structure naming convention needed to make Automated Tests work [ Manufacturer ]/ [ Model_Syncsource ]/ error / reference / response / Building an Automated Test for native SyncML clients 35. Automated Tests Maintenance Automated Tests need to be updated after any server modification that involves:

synclets

36. add/modification/deletion of a property (e.g. add a new field in the contacts) If NOT maintained the Automated Tests will systematically fail 37. Conclusion Automated Tests are a Quality Challenge:

help improve quality in our software

There's still a lot to do:

many manufacturers/devices

38. for every device 3/4 syncsources to be automatized 39. maintenance 40. Questions & Answers ...