HUAWEI TECHNOLOGIES CO., LTD. :32-35pt : R153 G0 B0 : FrutigerNext LT Medium : Arial :30-32pt : R153...

HUAWEI TECHNOLOGIES CO., LTD.

英文标题 :32-35pt

颜色 : R153 G0 B0

内部使用字体 :

FrutigerNext LT Medium

外部使用字体 : Arial

中文标题 :30-32pt

颜色 : R153 G0 B0

字体 : 黑体

英文正文 :20-22pt

子目录 (2-5 级 ) :18pt

颜色 : 黑色内部使用字体 :

FrutigerNext LT Regular

外部使用字体 : Arial

中文正文 :18-20pt

子目录 (2-5 级 ):18pt

颜色 : 黑色字体 : 细黑体

Employing TTCN-3 in a structured way for large-scale testing of telecom protocol components

Balakrishna S PolPrasanna Venkatesh

Huawei Technologies India Private Ltd

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential

Employing TTCN-3 in a structured way for large scale Protocol testing

Number of Protocols Supported

Lines of Code Maintained

Network Elements (NEs) deployment

Of history with feature growth of

almost 15% per Year


If your protocol testing business is this large then

Else you will experienceElse you will experience Escalating Cost, Escalating Cost, Resource difficulty, Quality issues, T2M issues, Resource difficulty, Quality issues, T2M issues, Maintenance issues …Maintenance issues …


Challenges in Large scale protocol testing

Maintainability, re-usability and scalability of huge number

of test suites

Maintainability, re-usability and scalability of huge number

of test suites

Various types of testing to ensure the coverage and test

quality

Various types of testing to ensure the coverage and test

quality

Significant growth in terms of number, type & features of

protocols

Significant growth in terms of number, type & features of

protocols


Challenge 1: Evolution of protocols

There are different kind of communication protocols being (by their representation and encoding) developed.

Typically there seems to be 3 kinds: Text, XML based, Binary (ASN representation or vanilla TLV representation)

For e.g. there are XML based telephony protocols like VXML/CCXML

Different Kinds of protocols

Varied Requirements Huge number and variety of

NE deployment with protocols leading to very different kind of requirements

For e.g. SIP is used in terminals, core telecom network, enterprise UC applications

SIP RFCs per year Increase in protocol features Protocol Standardization has

been continuously on the increase

For e.g. SIP which is a decade old protocol, had 20 new standards added in 2011

So we need a test methodology and framework which can scale for different representations, different requirements from the network and different standardization requirements


Approach: Test System for Protocol Testing

Standalone protocol layer testing

and not at product/solution level to

helpverifying the protocols across

devices irrespective of their nature

Embedded –portable test stub for

seamless communication with the

protocol layer to avoid device

related issues

Generic framework to cater to

different types of protocols. TTCN-3

can seamlessly take care of ASN.1

and Text. For XML based protocols a

codec plug-in is used

Systematic design of TTCN-3 test

suite to accommodate varied

requirement from customers and

enhancement in standardsSeparating out the protocol layer testing, a systematic-reusable TTCN-3 suite and separating out of codecs to handle different encodings will help to meet the protocol evolution challenge


Case Study: VoiceXML (VXML) testing

TTCN3 test case handling multiple responses from the VXML interpreter.

1 2

3

VXML test system where VXML component interprets the vxml file and interacts with other systems through different interfaces asynchronously

VXML is the W3C's standard XML format for specifying interactive voice dialogues between a human and a computer. This enables user to interact with voice applications over the telephone.

Do not try to use too many languages or frameworks. For all practical purpose TTCN-3 is good enough for any protocol testing. But the framework should see if it can handle the complexity of the protocols


Challenge 2: Coverage by testing types

Conformance testing of standards

API/interface testing Product scenario testing

Functional Testing Verification of thread model

by concurrently executing different/same test scenarios in multiple threads

Concurrency Testing Distributed architecture

testing of protocol stacks across physical nodes

1+1 or 1+N backup functionality of protocol stack

Distributed & Backup Validation of performance

requirements & benchmarking

System failure and long run testing to verify robustness

Performance & Stress

So we need a test methodology and framework which enables us to carry out different types of testing


Approach: Test Configurations for Protocol Testing

1 2

Multiple parallel test components (PTC) created under one main test component with required number of ports.

And each PTC will be used to executed one test scenario so the concurrency test can be achieved

Standalone protocol testing involve both message communication and API invocation

Every SUT instance communicate with TTCN-3 suite with bidirectional message ports for both upper & lower level. Separate ports are created for invoking APIs of the SUT

Such multiple SUT instances created to achieve the distributed and backup testing

1

2


Case Study: Concurrency testing

1

2

Test code at SUT side which creates required multithreading framework by creating thread pools and queues.

Concurrent test frameworks built using the Parallel Test Components (PTC) architecture of the TTCN3

Each PTC runs under MTC and have message and API invocation ports

Multiple TCs runs in parallel using multiple PTCs achieving concurrency testing

Sample Test Case

1

2

Through an adaptive usage of dynamic test configurations (PTC/MTC), TTCN-3 can be used for multiple types of testing, typically functional, distribution, reliability and concurrency testing.


Challenge 3 : Long Time maintenance

Test cases growth is a fact of life. We have experienced about 10% Y-o-Y growth in our test suites. If we did not have a maintainable, reusable and scalable test automation framework, the cost

would have been severe and TCs would have grown much more steeply

60,000+TCs

68,000+TCs

1,16,000+TCs

1,09,000+ TCs

93,000+ TCs84,000+

TCs76,000+TCs

1,01,000+ TCs


Approach: TTNC-3 Test Suite Design

The bottom up approach (from

Level-0 to Level-3) is used to

design the components of the test

suite

Each layer focuses on the specific

objective like Layer 0 for message

structure definition, Layer 1 & 2 for

common functions and Layer 3 for

test scripts

The design allows smooth addition

or modification of test scripts in

current & future releases and to

reuse test suite seamlessly. Finally

to achieve maintainability,

reusability and scalability aspects.


Case Study : Diameter Test Suite DesignASN.1 Definitions for Diameter messages, Diameter API parameters etc.

ASN.1 Definitions for Diameter messages, Diameter API parameters etc.

TTCN-3 templates for Diameter message, Diameter API parameters etc.

TTCN-3 templates for Diameter message, Diameter API parameters etc.

TTCN-3 Functions for SUT APIs, Sending/Receiving of Protocol message and common utilities

TTCN-3 Functions for SUT APIs, Sending/Receiving of Protocol message and common utilities

TTCN-3 Functions for test scenario Pre-conditions

TTCN-3 Functions for test scenario Pre-conditions

TTCN-3 test cases for Diameter functionality for all kinds of test scenarios

TTCN-3 test cases for Diameter functionality for all kinds of test scenarios

The scalable, reusable and maintainable framework is required to handle future needs. This requires the systematic approach toward the design of test automation framework.


Consider the longevity of the test suite/framework before designing the

framework. The longer the test suite is going to be maintained the more

compact the test suites will become in the long run.

Large scale protocol testing has the following attributes:Varied protocols

Same Protocols used for a variety of purposes and devices

Growth of protocol in one format or other over a very long time frame

TTCN-3 architecture enables different types of protocol testing. There is

almost no limit to the kind of testing you can do. Using different TTCN-3

concepts we can adapt to different circumstances

Finally, if the scale of testing is large and long, initial investment

into framework will have significant dividends.

Take Away

Thank youwww.huawei.com

HUAWEI TECHNOLOGIES CO., LTD. :32-35pt : R153 G0 B0 : FrutigerNext LT Medium : Arial :30-32pt : R153...

Documents

Transcript of HUAWEI TECHNOLOGIES CO., LTD. :32-35pt : R153 G0 B0 : FrutigerNext LT Medium : Arial :30-32pt : R153...