Capture & Replay

22
TTCN-3 Test Case Generation from arbitrary traces Capture & Replay Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies)

description

Capture & Replay. TTCN-3 Test Case Generation from arbitrary traces. Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies). Agenda. The Motivation The Technical Challenges Capture & Replay for TTworkbench Demo Q & A. Motivation(s). TTCN-3 Users. - PowerPoint PPT Presentation

Transcript of Capture & Replay

Page 1: Capture & Replay

TTCN-3 Test Case Generation from arbitrary traces

Capture & Replay

Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies)

Page 2: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Agenda

The MotivationThe Technical ChallengesCapture & Replay for TTworkbench DemoQ & A

2

Page 3: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Motivation(s)

TTCN-3 UsersWe have a TTCN-3

framework, where do the test cases come from?

IssuesManual test case creation tedious taskError proneDomain Knowledge required

Non-TTCN-3 UsersWe see a scenario and we would like to replay it later

IssuesWhat means replay w.r.t. testing?What is the test system? What is the system under test?

3

The desire to capture observed communication in test scripts suitable for test automation

Page 4: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

The Idea

4

Automatically generate

test case out of traces

Reproduce & Analyse Situation

from the Field (w/o SUT)

Automate existing

manual tests

Fast IOP Trace Analysis

(IMS/TETRA/ITS)

Codec tests

Page 5: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

What is a Trace?

5

Page 6: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

What is a Trace?

The observed communication is captured (stored) in a trace (file)Trace contains

Communication between two or more entitiesMessage exchangeOne or more protocols

6

Page 7: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

A Generic Communication Setup

7

A1

A2

B1

B2

.

.

.

.

.

.

Page 8: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

The SetupBut Who is Who?

A1 B1

A1 SUT

SUT B1

IOP1 IOP2

SIM1 SIM2

Page 9: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

The Setup From Message to Templates

A1 B1

INVITE sip:[email protected] SIP/2.0Call-ID: [email protected]: <sip:[email protected]:5061;transport=UDP>Content-Disposition: sessionContent-Length: 150Content-Type: application/sdpCSeq: 2 INVITEFrom: "TESTER1" <sip:[email protected]>;tag=1169Max-Forwards: 70To: <sip:[email protected]>Via: SIP/2.0/UDP 127.0.0.1:5061;branch=z9hG4bK5704

...

template INVITE_Request T1 := { requestLine := { method := “INVITE”, requestUri := “sip:[email protected]”, sipVersion := “SIP/2.0”}, msgHeader := { callId := “[email protected]”, contact := …, cSeq := “2 INVITE”, fromField := "TESTER1 …" , toField := <sip:[email protected]>, via := …., maxForwards := { fieldName := MAX_FORWARDS_E,

forwards := 70 }, .... allow := omit, ...

Page 10: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

The Technical Challenge

It is somehow easy to generate TTCN-3 templates from each message, butthe challenge is to obtain usable and maintainable test data and test scripts

Variable informationFrom run to run, perhaps unpredictableFrom message to message, perhaps constant perhaps dependend

Variable but constant informationShould be configurable before execution

White noiseLot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution

10

Page 11: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

The Technical Challenge

It is somehow easy to generate TTCN-3 templates from each message, butthe challenge is to obtain usable and maintainable test data and test scripts

Variable informationFrom run to run, perhaps unpredictableFrom message to message, perhaps constant perhaps dependend

Variable but constant informationShould be configurable before execution

White noiseLot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution

11

Protocol and/or domain knowledgehas to be included!

Page 12: Capture & Replay

The SolutionAutomated TTCN-3 Test Generation and Execution

12

Page 13: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Example Scenario

13

SIP

User A calls User B

User B accept call

Page 14: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Example Scenario

14

SIP

Use a common Monitor Tool to capture messages e.g. Wireshark

Trace File

Page 15: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Example Scenario

15

Trace File TTworkbench Execution

Put the Trace File into TTworkbench ….

Generate the captured Test Scenarios…

Replay your previously captured Test Scenarios full automatically

Page 16: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Example Scenario

16

Trace File TTworkbench Execution

Put the Trace File into TTworkbench ….

Generate the captured Test Scenarios…

Replay your previously captured Test Scenarios full automatically

Page 17: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Know-How Integration

Manual definitions in a configuration file, toExtract existing data and use the same values in the message flow (sequence numbers, call IDs, …)Define uninteresting fields for ignoring at matching time

Automatically extract similar data from templates to create “better” templates and structuresReuse existing

Type systems and codecsTest adapters and component definitionsTest functions and Alt-Steps

Page 18: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

DEMO!

18

Page 19: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Key Benefits

Speed up functional and interoperability testingBy reducing manual task

Automate testingGenerate your test, instead of implementing it manuallyExecute them full automatically Get your detailed test reports

Quality Optimization Test case analyzer reduces errors in test cases Fast reusability and easy extension of already defined tests

19

Page 20: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Supported Capturing Sources & Requirements

Wireshark The most popular network analysis and capturing tool

Jpcap Service The remote capturing service provides capturing on remote machines

Pcap Files Trace files created beforehand

TTworkbench logs (TLZ)Analyze the logs without an SUT

Capturing Console Input and Output

Capturing of data via serial connections (RS232, USB) SSH, TELNET Local Console

WindowsWinPCAP

Linux/UnixLibpcap at least version 1.0

Recommendation: Install Wireshark

Includes the installation of WinPCAP and LibPCAP

20

Page 21: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Further information

Testing Technologiestestingtech.comfacebook.com -> Testing-Technologies

Capture & Replaytestingtech.com/products/ttworkbench.phptestingtech.com/download/datasheets/CaptureAndReplay.pdf

21

Page 22: Capture & Replay

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Contact

22

Bogdan Stanca-Kaposta

Testing [email protected]

om

Theo VassiliouTesting Technologies

[email protected]