Definition of Moby Dick Trial Scenarios Annexpacyna/deliverables/MobyDick/D0501-Annex.pdf ·...

WP5-D0501-Annex_new.doc Version 1.0 30.09.2002

Moby Dick WP5 1 / 60

IST-2000-25394 Project Moby Dick

D0501 Annex

Definition of Moby Dick Trial Scenarios Annex

Contractual Date of Delivery to the CEC: 30 September 2002 Actual Date of Delivery to the CEC: 30 September 2002 Author(s): partners in WP5 (cf. page 3) Participant(s): partners in WP5 Workpackage: WP5 Security: Public Nature: Report Version: 1.0 Total number of pages: 60 Abstract: This document provides the definitions of tests to be carried in the field trial in the framework of the Moby Dick project. Keyword list: Evaluation, Moby Dick platform, Application, Tests, Trial, User evaluation, Expert evaluation



Authors

Partner Name Phone / Fax / E-mail

UC3M José Ignacio Moreno Phone: +34-916249183 Fax: +34-916248749 E-mail: [email protected] Pablo Serrano Phone: +34-916249756 Fax: +34-916248749 E-mail: [email protected] Antonio Cuevas Phone: +34-916249785 Fax: +34-916248749 E-mail: [email protected] Ignacio Soto Phone: +34-916245974 Fax: +34-916248749 E-mail: [email protected]

USTUTT Jürgen Jähnert Phone: +49-711-685-4273 Fax: +49-711-678-8363 E-mail: [email protected] Jie Zou Phone: +49-711-685-5734 Fax: +49-711-678-8363 E-mail: [email protected]

FhG Davinder Pal Singh Phone: +49-30-3463-7175 Fax: +49-30-3463-8175 E-mail: [email protected] UKR Piotr Pacyna Phone: +48-12-6174040 Fax: +48-12-6342372 E-mail: [email protected] Janusz Gozdecki Phone: +48-12-6173599 Fax: +48-12-6342372 E-mail: [email protected] EURECOM Michelle Wetterwald Phone: +33-4-93002631 Fax: +33-4-93002627 E-mail: [email protected] T-Nova Serge Tessier Phone: +49 (0)30 – 3497 -3114 Fax: +49 (0)30 – 3497 -3199 E-mail: [email protected] Moritz Kulbach Phone: +49 (0)30 – 3497 -2042 Fax: +49 (0)30 – 3497 -2043



E-mail: [email protected] NEC Marco Liebsch Phone: +49-6221-13 70 8-11 Fax: +49-6221-13 70 8-28 E-mail: [email protected] Telemaco Melia Phone: +49-6221-13 70 8-17 Fax: +49-6221-13708-28 E-mai: [email protected] Amardeo Sarma Phone: +49-6221-13 70 8-19 Fax: +49-6221-90511-55 E-mail: [email protected] Ralf Schmitz Phone: +49-6221-13 70 8-12 Fax: +49-6221-13 70 8-28 E-mail: [email protected] ETHZ Hasan Phone: +41-1-632 70 12 Fax: +41-1-632 10 35 E-mail: [email protected] Pascal Kurtansky Phone: +41-1-632 70 13 Fax: +41-1-632 10 35 E-mail: [email protected] CRM Nesrine Chaher Phone: +33-1-69354812 Fax: +33-1-69352501 E-mail: [email protected]



Table of Contents

1. INTRODUCTION ....................................................................................... 7

2. MOBY DICK ARCHITECTURE................................................................. 8

2.1 AAAC specific scenario ................................................................................................................. 8

2.2 Mobility-specific scenarios for Handover ...................................................................................10 2.2.1 Intra-domain handover............................................................................................................10 2.2.2 Inter-domain handover............................................................................................................11

2.3 Mobility-specific scenarios for Paging ........................................................................................11 2.3.1 Dormant registration and paging process................................................................................11 2.3.2 Paging area update ..................................................................................................................12

2.4 QoS specific scenarios...................................................................................................................12

3. EXPERT EVALUATION TEST DESCRIPTION....................................... 15

3.1 Initial Registration ........................................................................................................................15 3.1.1 Initial Registration in a Foreign Domain (Procedures) ...........................................................15 3.1.2 Initial Registration in Foreign Domain (AR/AAAC Client behavior) ....................................15 3.1.3 Profile Transfer .......................................................................................................................15

3.2 Mobility..........................................................................................................................................15 3.2.1 Intra and inter domain mobility ..............................................................................................15

3.3 Fast Handover ...............................................................................................................................15 3.3.1 Basic Fast Handover ...............................................................................................................15 3.3.2 Real time audio stream Fast Handover ...................................................................................16

3.4 IP paging........................................................................................................................................16 3.4.1 Dormant roaming ....................................................................................................................16 3.4.2 Mobile initiated dormant de-registration ................................................................................16 3.4.3 Paging initiated dormant de-registration .................................................................................16

3.5 QoS.................................................................................................................................................17 3.5.1 Reservation and release of network resources ........................................................................17 3.5.2 User traffic conformance testing.............................................................................................17 3.5.3 QoS measures .........................................................................................................................17

3.6 Accounting.....................................................................................................................................17 3.6.1 Accounting at AAAC Client ...................................................................................................17 3.6.2 Accounting at AAAC.f ...........................................................................................................18 3.6.3 Accounting at AAAC.h...........................................................................................................18

3.7 Charging ........................................................................................................................................18

3.8 Auditing .........................................................................................................................................19 3.8.1 Logging Test ...........................................................................................................................19 3.8.2 Auditing Test ..........................................................................................................................19

4. USER EVALUATION TEST DESCRIPTION........................................... 20

4.1 Introduction...................................................................................................................................20



4.1.1 Types of applications ..............................................................................................................20 4.1.2 Test-scenario overview ...........................................................................................................21

4.2 System friendliness tests ...............................................................................................................21 4.2.1 Call set-up latency...................................................................................................................21 4.2.2 Usability..................................................................................................................................21 4.2.3 Coverage test...........................................................................................................................22

4.3 Basic tests.......................................................................................................................................22 4.3.1 Basic stream_app ....................................................................................................................22 4.3.2 Basic VoIP_app – first test......................................................................................................22 4.3.3 Basic VoIP_app – second test .................................................................................................23 4.3.4 Basic game_app – first test .....................................................................................................23 4.3.5 Basic gamme_app – second ....................................................................................................23 4.3.6 Basic downl_app.....................................................................................................................23

4.4 Complex tests.................................................................................................................................23 4.4.1 Stream_app with background traffic.......................................................................................23 4.4.2 VoIP_app and game_app – first test .......................................................................................23 4.4.3 VoIP_app and game_app – second test...................................................................................24 4.4.4 VoIP_app, stream_app and game_app....................................................................................24

4.5 Final tests .......................................................................................................................................24 4.5.1 VoIP_app and stream_app, moving – first test .......................................................................24 4.5.2 VoIP_app and stream_app, moving – second test...................................................................24

5. TEST DEFINITIONS................................................................................ 25

5.1 Initial Registration in foreign domain.........................................................................................25

5.2 Initial Registration in foreign domain (AR+AAAC Client behavior) ......................................26

5.3 Profile transfer ..............................................................................................................................27

5.4 Intra and inter domain mobility ..................................................................................................28

5.5 Basic Fast Handover .....................................................................................................................30

5.6 Real time audio stream Fast Handover.......................................................................................31

5.7 Dormant roaming .........................................................................................................................34

5.8 Mobile initiated dormant de-registration ...................................................................................35

5.9 Paging initiated dormant de-registration....................................................................................36

5.10 Reservation and a release of network resources ........................................................................37

5.11 User traffic conformance testing .................................................................................................38

5.12 QoS measures ................................................................................................................................39

5.13 Accounting at AAAC client..........................................................................................................40

5.14 Accounting at AAAC.f..................................................................................................................41

5.15 Accounting at AAAC.h.................................................................................................................42

5.16 Charging at AAAC.h ....................................................................................................................43



5.17 Logging Requirements..................................................................................................................44

5.18 Auditing Requirements ................................................................................................................45

5.19 Basic stream_app test ...................................................................................................................46

5.20 Basic VoIP_app – first test ...........................................................................................................47

5.21 Basic VoIP_app – second test.......................................................................................................48

5.22 Basic game_app – first..................................................................................................................49

5.23 Basic game_app – second .............................................................................................................50

5.24 Basic downl_app ...........................................................................................................................51

5.25 Stream_app with background traffic..........................................................................................52

5.26 VoIP_app and game_app – first test ...........................................................................................53

5.27 VoIP_app and game_app – second test .......................................................................................54

5.28 VoIP_app, stream_app and game_app .......................................................................................55

5.29 VoIP_app and stream_app, moving – first test ..........................................................................56

5.30 VoIP_app and stream_app, moving – second test......................................................................57

5.31 Call set-up latency.........................................................................................................................58

5.32 Usability .........................................................................................................................................59

5.33 Coverage test .................................................................................................................................60



1. Introduction This Annex to D0501 defines tests to be used during Phase 2 and Phase 3 of the Moby Dick project. However, it does not define the tests according to these phases, but separates them into expert and user evaluation tests. The main reason to use type of evaluator instead of phase as classifier is the following: tests during the second phase are intended to provide feedback to WPs, while the tests during the third phase will provide evaluation of an overall network behavior; therefore, tests used during the second phase may also be employed during Phase 3. Notwithstanding the foregoing, different kinds of tests involve different objectives pursued. Thus, expert evaluation trials aims not only to test the correct network behavior, but also to provide diagnosis of misbehavior situations; also, this kind of users must know a priori what results are expected, so technical description of the test and situation evaluated is desirable. On the other hand, user evaluation is mostly oriented to third phase of the project, and assumes user unawareness of network internal behavior and architecture. Therefore, this kind of tests requires a less technical description, focusing on actual usage of services provided by network. Nonetheless, a technical motivation seems to be desiderable while describing the tests, to explain the rationale behind them. Both kinds of tests are defined using templates designed for that purpose. However, before that formal definition, tests are introduced by text description, allowing both figures and prose explanations, and also including detailed description of issues not to be included in templates (because they are not excessively relevant). This document focuses on test definitions. Test results, along with the different tools employed to perform the tests here described, will be reported in both D0503 and D0504 documents. The structure of this document is as follows: first, a short overview on the Moby Dick architecture is given, in order to establish the framework for all tests, and to allow a more detailed description of the relevant items; then the tests are described, grouped by kind of evaluator with internal grouping within each kind. Finally, formal descriptions of tests with templates are given.



2. Moby Dick Architecture IP technology has proved its capability to support different kind of services from voice or video services to file transfer. Future networks will support any kind of services over an integrated network based on a combination of different access technologies, a common IPv6 core network with administrative support based on an enhanced Authentication, Authorisation and Accounting Architecture (AAA Arch.) and a flexible Quality of Service (QoS) support. The Moby Dick Architecture is composed of different access networks based on Ethernet, wireless LAN (WLAN) and Wideband Code Division Multiple Access (WCDMA) technologies, and a Core Network based on IPv6. The architecture provides user mobility (horizontal and vertical handover) based on Mobile IP procedures, QoS capabilities based on DiffServ, and Authentication, Authorization, Accounting and Charging (AAAC) based on AAA procedures. This architecture allows to distinguish scenarios within administrative domains and between administrative domains. We consider that this new architecture will be the future architecture for beyond 3rd Generation Networks in which users, while using the same terminal, will change one access network to another depending on their mobility, the user preferences, price to be paid for network access, availability or performance, in a transparent way. Over this framework any kind of services will be provided by using a common infrastructure. Moby Dick network architecture elements relevant for this discussion and shown in Figure 1 are: • Mobile Terminals (MT) with multiple access technologies (WLAN, Ethernet, WCDMA and other). •

Access Routers (AR), which provide an interface between wireless wired access networks, and Core Network. From the point of view of IP subnetworks, each AR, and the terminals it interconnects, will be part of the same IP subnet.

• AAAC Server, which controls an administrative domain. • QoS Broker, which controls network resources of access networks. • Home Agent and Paging Agent are in charge of mobility issues.

Moby Dick IPv6 Backbone

802.11b AccessNetwork

WCDMA AccessNetwork

EthernetAccessNetwork

IP PublicBackbone

Moby Dick Access Network Moby Dick Core Network

AR

AR

AR

QoSBroker

AAACServer

Home Agent

Paging Agent

Moby Dick IPv6 Backbone

802.11b AccessNetwork

WCDMA AccessNetwork

EthernetAccessNetwork

IP PublicBackbone

Moby Dick Access Network Moby Dick Core Network

AR

AR

AR

QoSBroker

AAACServer

Home Agent

Paging Agent

Figure 1. Moby Dick Network Architecture

As each Access Router (AR) and terminals connected to it constitute an IP subnet, ARs are the entities controlling the network access to and from the different access networks according to user specific agreements with the network operator, and operator’s policies.

2.1 AAAC specific scenario The following Figure 2. shows a scenario focused on AAAC. From a provider (administrative domain) point of view there are two types of users: home users and foreign users. In this respect an AAAC Server will play two different roles: home and foreign AAAC server.



AAAC.f1

AAAC.h

AAAC.f2

Foreign Domain 1 Foreign Domain 2

Home Domain

AR AR ARAR

MT

AAAC Interaction

MT Movement

QoSB.f1

QoSB.f2

QoSB.h

QoSB Interaction

Figure 2. AAAC specific scenario

Each AAAC server is connected to a database, which contains a profile for each user of this domain. This database will be consulted by the home AAAC server during user registration. If the user is currently in a foreign domain, a subset of the user profile will be distributed to the respective foreign AAAC server after a successful registration. A sub-subset of this profile will further be delivered by the foreign AAAC server to the AR. The foreign AAAC will also deliver another sub-set of the profile to the foreign QoS Broker, defined as NVUP (Network View of User profile). The foreign QoS Broker, in a posterior phase (flow authorization), will provide the AR with this NVUP. The NVUP will contain technical parameters for the service provisioning. It must be noted that the AR receives profile sub-sets via the AAAC foreign and via the foreign QoS Broker. Consequently a user profile can be separated in 5 sections regarding entities interested in the information, if we deal solely with AAAC we would consider only 3 sections, which is the case in Figure 3. The user profile contains following parameters (new values can be added later):

• UserID • Password • Home_address • User_class • Total_credit

User Profile (AAAH) Personal User data, Roaming permissions

User Profile Subset 1 (AAAF) Remaining Budget, Service specific permissions

User Profile Subset 2

(AR) IPsec Key…

NVUP (QoS B) DSCP, Port, Ipaddr …

Figure 3. User Profile



In the registration a user has to perform a login procedure. Assuming the user in the foreign domain, the AAAC Client located in the AR of the current IP subnet forwards the registration message to the AAAC.f and the AAAC.f forwards this message to the AAAC.h. AAAC.h compares the password/userID and returns the corresponding profile subset to the AAAC.f and via AAAC.f to the AAAC Client (see Figure 4). The AAAC.f also provides the foreign QoSBroker part of this profile subset, that is the NVUP.

AAAF AAAH

AR

User profile DB

User profile

User profile subset

User profile subsubset QoS Broker

NVUP

Figure 4. Profile transfer

With the given user profile sub-set, the AAAC module in the AR issues router configuration commands. In a further phase (flow authorization), a QoS daemon also located in the AR, interprets QoS specific parameters of the profile –provided by the foreign QoS Broker- and transfers these parameters into router configuration commands.

2.2 Mobility-specific scenarios for Handover For handover, there are four different possibilities, each considering that the mobile terminal has already registered itself (see previous section) and that basic Mobile IPv6 procedures for registration of the Care-of-address at the Home Agent have been completed. The four possibilities are:

• Intra-domain, intra-technology handover, • Intra-domain, inter-technology handover, • Inter-domain, intra-technology handover, • Inter-domain, inter-technology handover.

It is assumed that Access Routers have security associations with neighbouring access routers of the same domain, but that this is not true for Access Routers in different domains. As a result, an encrypted profile may be transferred as part of the handover messages between the old and new access routers (HI / HAck) provided these access routers are in the same domain. The focus of Moby Dick fast handover scenario aims on the provisioning of fast handover for intra-domain handover, since inter-domain handover requires additional AAA message exchange with the home domain, which causes handover latency beyond any notion of fast or seamless handover.

2.2.1 Intra-domain handover When a mobile terminal is registered, it will monitor the surroundings for an alternative wireless and wireline access. As soon as a “better” choice is available – this could mean quality or cost – a fast handover will be initiated. The Moby Dick approach implements Mobile Terminal initiated handover and therefore the Mobile Terminal will handover to the preferred Access Router. Whether the handover is to the same technology or not is of secondaryimportance. The handover will invoke the fast handover procedure. This handover shall therefore be seamless, and communication between the old Access Router and the new Access Router prepares the handover by carrying authentication and QoS information in the handover messages.



Domain A

HADomain H

MT

oAR

nAR

Domain A

HADomain HDomain H

MT

oAR

nAR

Figure 5. Intra-domain handover

The decision to deploy fast handover or re-registration only depends on the intra- (see Figure 5) or inter-domain handover scenario (Figure 6)and is not dependent on whether or not the handover is between Access Routers supporting the same or different technologies, but the lower-layer capabilities may influence handover latency and packet loss.

2.2.2 Inter-domain handover When a Mobile Terminal needs to handover between different domains, the Fast Handover technique cannot be invoked as there is no security association between the Access Routers of different domains. Therefore, the full registration process is restarted to authenticate the Mobile Terminal in the new domain.

Domain A

HADomain H

MT

oAR

nAR Domain B

Domain ADomain A

HADomain HDomain H

MT

oAR

nAR Domain BDomain B

Figure 6. Inter-domain handover

Again, the decision to deploy fast handover or re-registration only depends on the intra- or inter-domain handover scenario and is not dependent on whether or not the handover is between Access Routers supporting the same or different technologies. In this case, the influence of lower layer characteristics is likely to be negligible.

2.3 Mobility-specific scenarios for Paging For dormant mode and IP paging, basically there are 2 scenarios to be shown. One scenario is related to a mobile terminal’s dormant mode registration with a domain’s Paging Agent (PA). Further, roaming in the registered IP paging area without sending Mobile IPv6 location updates as well as initiating the paging process in case of incoming user data packets, which are to be buffered and forwarded to a dormant mobile terminal, is part of this scenario. A second scenario comprises roaming across the borders of the registered paging area and updating the new paging area with the Paging Agent.

2.3.1 Dormant registration and paging process When a mobile terminal has no communication open and decides for entering a dormant mode in order to avoid sending superfluous location updates to the network, it discovers a responsible Paging Agent and registers with it through its current Access Router. From now the PA knows about the mobile terminal’s paging area, which comprises multiple Access Routers. The PA address is notified to the mobile terminal’s Home Agent (HA) as node where incoming traffic is to be forwarded to. In dormant mode, the mobile terminal is allowed to roam within the registered paging area without the need to send detailed location updates to its HA. In case of incoming traffic, initial user-data packets are sent to the PA and buffered for being forwarded after the paging process. The PA initiates paging the mobile node through the paging area’s ARs, aiming at re-activation of the mobile terminal and solicitation of re-establishment of routing information with the network. As part of the re-activation phase, the HA as well as the PA are notified of the mobile terminal’s exact location to allow forwarding of buffered and further incoming user-data packets. With regard to the described scenario, Figure 7 illustrates the involved architecture components.



Domain A

HADomain H

MT AR1

Domain A

HADomain HDomain H

MT

AR2

AR3

IP paging area

PA

Figure 7. Dormant registration and paging process

2.3.2 Paging area update After a mobile terminal has entered the dormant mode, it is allowed to roam within the registered paging area without the need to send location updates to its HA. To track coarse location information of a dormant mobile terminal, the paging area is registered with the PA assigned to the mobile terminal during the dormant registration process. In case of being dormant and entering a paging area, which is different to the currently registered one, the new paging area is to be registered with the PA. After the paging area update procedure, the mobile node is allowed to roam within the new paging area. This paging area update procedure does not affect the mobile terminal’s HA. This scenario is illustrated in Figure 8.

Domain A

HA Domain H

Domain A

HA Domain HDomain H

MT

IP paging area 1PA

IP paging area 2

Figure 8. Paging area update procedure

2.4 QoS specific scenarios QoS specific scenarios are based on AAAC-QoS interworking model presented in Figures 10 and 11. The positioning of the QoS Attendant (implemented as a QoS Manager software module) and AAAC Attendant in the AR-RG tandem is shown in figure 9.



AR

UMTS

802.11b

RG

Ethernet

AR

AAA

C At

tend

ant

WLANAR

QoS

Man

ager

Core network

Access Network

MT

MT

MT

Figure 9. Radio Gateway – Access Router pair

Access Router

MobyDick IPv6 Backbone

IP PublicBackbone

QoS Broker

AAAC server

AAACattendant

QoSattendant

Diameter message

COPS message

User registration

DSCPs

DSC

Ps

DSCPs

Figure 10. Interoperation of AAAC and QoS Broker during the Registration phase

Figure 10 shows the sequence of actions that are taken by the Acess Router in order to register a new Mobile Terminal. The communications between the AAAC attendant part residing in the AR and the AAAC server is done over the Diameter protocol. The AAAC server consults the QoS broker with the COPS message. This scheme sets the basis for QoS testing. The QoS specific tests are more related to the resource authorization that is performed after a mobile terminal starts to submit traffic to the AR. The first data packet triggers QoS attendant – QoS Broker signaling messages exchange over the COPS message (Figure 11).



Access Router

MobyDick IPv6 Backbone

IP PublicBackbone

QoS Broker

AAAC server

AAACattendant

QoSattendant

COPS message

Data flow

Res

ourc

e-re

ques

t

Data packets

Reso ur ce -a ut ho ri se

Figure 11. Resource authorisation by a QoS Broker



3. Expert evaluation test description

3.1 Initial Registration This section describes the test for initial registration and profile transfer.

3.1.1 Initial Registration in a Foreign Domain (Procedures) When entering a new domain, the MT must first discover the AR and know that it has to register at home. This is done due to the fact that a router advertisement message is received which has a different network prefix than the one configured at the home address. MT sends URP messages that must be translated into Diameter Messages by the AAAC attendant in the AR. Diameter messages must be exchanged between AAAC attendant AAAC.f, AAAC.h and HA to allow MT registration. Upon this process an IPsec AH tunnel between MT and AR is established. The URP message contains a user identification part which allows an AAA.f server to find out to which AAA.h server the registration message mus be forwarded.

3.1.2 Initial Registration in Foreign Domain (AR/AAAC Client behavior) Before a user has registered himself the AR must not forward/route any packet coming on any of its interfaces from this source (MT). Here are in a real network some exceptions required e.g. the fact that emergency calls must be possible without any AAA specific preconditions. But the AR must accept and process URP messages containing an AA_Request_Message. Each AR belongs to an administrative domain and, statically configured, each AAA.f server is responsible for a certain amount of ARs. So by default each AR forwards registration messages for this statically configured AAA.f server. When the user has correctly registered, the AR sets up its network filters accordingly that is granting access to the network to the MT. If the registration process was unsuccessful, the AR must inform the MT.

3.1.3 Profile Transfer The test to be conducted comprises according to the above scenario the correct transfer of the profile including user mobility functionality. So a user might use the terminal and the correct transfer of the user profile enables the required actions to provide this specific user with the network resources according to his personal SLA with the provider. If another user might use the same terminal, after registration, a new user with its network preferences stored in the new users profile gets network access according to the new parameters. This test verifies that the correct user profile is transferred to the required entities via the AAAC architecture.

3.2 Mobility

3.2.1 Intra and inter domain mobility After successful registration the MT starts an ftp session connecting to a host in its Home Network. The MT moves (ftp session running) into another WLAN cell belonging to the same administrative domain. The MT moves into another WLAN cell in another administrative domain. The basic goal of this test is to give a quantitative answer about the packet loss in such cases. Dues to the Moby Dick specific architecture it is expected that the case where two administrative domains are involved a longer disruption time is expected.

3.3 Fast Handover This section describes the expert tests needed to evaluate the intra-domain fast handover execution. The fast handover metrics to be analyzed consists mainly of the handover latency (i.e., handover related disruptiontime, in which the mobile node is not able to send or receive data) and packet loss. The latency will be measured via the network analyzer Ethereal, while the packet loss measure will use a pre-defined number of UDP packets sent, consecutively numbered.

3.3.1 Basic Fast Handover This basic fast handover test should just prove that the mechanism itself works, but does not yet analyse the handover parameters. In this test, a simple ping6 is sent from a correspondent node to the mobile node and vice versa. After a fast handover, the reachability should be confirmed. Since a ping6 request message is sent every second, this is not a performance test, but just shows that the fast handover maintains ongoing sessions.



3.3.2 Real time audio stream Fast Handover This scenario addresses and tests the real-time capability of the system. A correspondent node sends a real-time audio stream to the mobile node. This is realized via the real-time audio tool RAT, which offers the possibility to reduce all buffering to zero to simulate real-time VoIP traffic, as is the case for IP telephony. The receiving mobile terminal should continuously receive the audio stream during a handover without noticing any interruption or degradation. Furthermore, the same test should be performed with a video stream, which places even more restrictions on the system, since this will exploit the saturation of the wireless channel.

3.4 IP paging This section describes the tests to be performed for proving the IP paging concept. It covers the dormant registration and tests to prove that functional requirements are met. The checking of control messages for refreshing registrations when a Home Agent or Paging Agent registration lifetime is about to expire is not covered in the tests. Dormant transition of a mobile terminal is mobile terminal initiated. When an appropriate trigger is given to the paging module, the mobile terminal should send an ICMPv6 DormantRequest message to its current Access Router. The paging attendant function of the Access Routers relays the request to its pertinent Paging Agent (PA). The Paging Agent sends an ICMPv6 DormantReply message back to the mobile terminal through the Access Router. In the reply message, paging process related Purpose Built Keys must be conveyed from the PA to the mobile terminal. Further parameters are the CommonPagingID for the registering mobile terminal as well as the PA’s address. On reception of the reply message, the mobile terminal registers the PA’s address as alternate CoA with its Home Agent by means of sending a Binding Update message carrying the alternateCoA sub-option. After getting the BindingAcknowledge message from the HA, the mobile terminal should enter the dormant mode. Entering the final state or intermediate states could be indicated as a debug message in /var/log/messages. Also appropriate indications on the screen could be provided for the tests.

3.4.1 Dormant roaming After the mobile terminal has entered the dormant mode, it should be shown that the mobile terminal is allowed to roam between Access Routers assigned to the same paging area without sending any registration to the network. While roaming, the mobile terminal must retrieve paging area identifier information from Router Advertisements. When entering a different paging area, as indicated by reception of an appropriate paging area identifier, the mobile terminal should register the new paging area with its PA, sending an updating ICMPv6 DormantRequest message. The mobile terminal should now be allowed to roam in the new paging area without sending mobility management or authentication related control messages to the network. BU messagee with alternate CoA should be sent to HA after ICMPv6 reply. Appropriate indications of performed or suppressed actions could be provided on the screen or written in /var/log/messages. In additions, a network packet analyzer could be used to check respective signaling packets and their parameters.

3.4.2 Mobile initiated dormant de-registration In this test case, the mobile terminal has to send user-data packets. When the application sends packets, the mobile terminal should first initiate standard attach mechanisms (CoA acquisition and AA procedure), then register its new CoA with the Home Agent (BU message) and de-register with the Paging Agent (DormantRequest, lifetime = 0). Packets should be allowed to be sent and to be received after the standard network attach procedure. Appropriate indications of performed or suppressed actions could be provided on the screen or written in /var/log/messages. In additions, a network packet analyzer could be used to check respective signaling packets and their parameters.

3.4.3 Paging initiated dormant de-registration In this case, a Correspondent Node sends packets to the dormant mobile terminal. The mobile’s HA intercepts initial packets and forwards them to the registered PA. The PA buffers initial user-data packets while taking care of the paging process. To re-activate the mobile terminal and to initiate re-establishment of routing information with respective network components, the PA sends an ICMPv6 PagingRequest



message to the paging area’s Access Routers. The mobile terminal receives the PagingRequest message, as generated at the AR according to the procedure as described in [D0301], either through a wired Ethernet connection, a WLAN connection or through the W-CDMA RG it has attached to while being dormant. On reception of the paging request control message, the mobile terminal initiates the standard network attach procedure. The CoA is to be registered with the HA and the PA. After this active registration process, the mobile terminal should receive initial user-data packets, forwarded from the PA buffering function, as well as further packets sent through the HA or from the Correspondent Node by means of route optimization. Appropriate indications of performed or suppressed actions could be provided on the screen or written in /var/log/messages. In additions, a network packet analyzer could be used to check respective signaling packets and their parameters.

3.5 QoS

3.5.1 Reservation and release of network resources This scenario aims to test the resource management that is carried on by the QoS Broker. The specific focus is on the reservation and release of network resources by the QoS Broker in response to request (which is implicit) from the MT. When there are insufficient resources in the network the terminal must not be granted network service. When the resources are available they should be granted provided that the user is authorised to use the resources. The objective of this test is to verify that there is no resource leak and that QoSBroker can handle numerous requests that are issued in sequence and in parallel. The QoSManager in the ARs will use TC API to get its queues state. This is reported to the QoSBroker. QoSBroker’s reactions to different network load levels should be tested. The consistency of the configuration messages issued by the QoSBroker to the ARs it controls will be a valuable method to check the correct behaviour of the QoSBroker. The test checking correctness of the reservation and release of resources should be repeated for different classes of service. A simple way to check that network resources are reserved/released in the routers is to examine the content of the messages reporting AR’s queues state to the QoSBroker.

3.5.2 User traffic conformance testing This scenario aims to test the correct behaviour of resource allocation and traffic control procedures. Traffic control must conform to the SLS of a user that is reflected in the user profile. For this experiment we propose a netperf application which allows to control the amount of data that is injected into the network. A data stream according to a traffic contract will be generated and packet losses are evaluated. The experiment will be repeated for different user profiles and for different service classes. Repeat tests with rate set to higher values than the values in the user profile in order to simulate violation of the profile by a misbehaving user.

3.5.3 QoS measures In this test, a user moves around the test trial site so as to be covered by all the three access technologies and measures the traffic data he obtains in each location (packet loss rate, mean throughput). We propose mgen application which allows to control the amount of data injected into the network and to measure the bandwidth, the delay and packet loss rate between the receiver and the sender endpoints. Mgen can be used to simulate both user’s traffic and network’s background traffic. Measures should be done for different network loads. Besides mgen, other applications will be used to generate user’s traffic. But this time “authentic” user traffic (e.g. VoIP applications). QoS measures will be based on human perception of the service and not in parameters like bandwidth.

3.6 Accounting

3.6.1 Accounting at AAAC Client The AAAC Client receives answers from AAAC.f for issued AA requests. In case of a positive answer the AAAC Client configures its accounting functionality according to the accounting policies contained in the AA answer. If no such policies are present AAAC Client configures the meter according to a default configuration. Accounting is configured before the answer is forwarded to the MT.).



After forwarding the AA answer the AAAC client immediately generates an accounting start request and sends it to the AAAC.f. During a session runtime the client generates interim accounting messages and sends them to AAAC.f. If the client receives a session stop indication it generates an accounting stop request and sends the accounting data to the AAAC.f. For fault resilience the client should keep accounting records as long as it has not receives a positive accounting acknowledgment. The AAAC Client has an interface to the metering functionality on the access router. It sends a METER_START message containing a filter spec and all other configuration parameters when accounting is started. It may receive a handle from the metering function. It sends a METER_STOP message containing a filter spec or handle previously received to stop metering for that particular session. The following information is needed as filter: o mobile node IP address (CoA), o mobile nodes ports (in case of specific QoS sessions which are flow based) (known through specific

AA request from MT), o DSCP (either 0 for registration or specific value for QoS session; note that signaling traffic between

MT and AR is supposed to have DSCP!=0 so this traffic is not accounted which is deliberate). The AAAC client must have a function to retrieve meter data for a particular session. A GET_METER_DATA function must be supported by the metering which accepts CoA or handle and return all current accounting information for a particular MT. The test comprised the test of the correct meter configuration after the METER_START command. For this reason user 1 is attached to a profile representing a flat rate service, user 2 must pay per volume. Using the same terminal, the user 1 sends traffic and in the Accounting database the traffic patterns are stored. In this case no data is stored in the database since the user has a flat rate service. After a while the user 2 registers on the same terminal and, since this user has no flat rate service, after the METER_START message the data of this user must be stored in the database.

3.6.2 Accounting at AAAC.f Upon receiving a positive AA answer the AAAC.f may add additional local accounting policies to it. The AAAC.f must not override any policy already given by AAAC.h. The AAAC.f receives accounting requests from its AAAC clients. It inspects the user-name in the accounting request and routes the request to AAAC.h. If no user-name is specified in the accounting request it uses the session ID to get the corresponding session information which include the user-name. The AAAC.f forwards accounting answers from AAAC.h to the client.

3.6.3 Accounting at AAAC.h AAAC.h processes accounting requests and replies with accounting answers to the sender. In case the accounting request could be correctly decoded and contains all necessary fields it replies with a positive reply; otherwise with a negative. The AAAC.h stores the accounting data in its database.

3.7 Charging Charging calculates the price for a given service consumption based on session and accounting information. It maps technical values into monetary units, and therefore applies a given contractual agreed upon tariff. Tariff applied might depend on QoS parameters as well as time dependent parameters, e.g., time of the day, day of the week a service is invoked. The tariff for a particular session is assumed to be static during that session. If the user stays in a foreign domain (i.e. roaming) then the accounting data could basically be stored in an accounting database located in AAAC.f or AAAC.h. Furthermore the charging could be done either in AAAC.f or AAAC.h. Within MobyDick, all accounting data is stored in an accounting database located at the AAAC.h only. This implies that the charging component is located in the AAAC.h, too. So in the case of roaming, accounting data is flowing from the foreign to the home domain. Of course, the accounting data needs to contain information about roaming, in order to allow the charging component selecting different tariffs (i.e. roaming tariffs). Basically there exists postpaid and prepaid business models. At a first step only postpaid will be implemented, since prepaid is much more complex to handle and will need further investigations. In the postpaid scenario, the AAAC.h stores the accounting data (coming either from AAAC.h itself or AAAC.f) in the accounting database. The detailed structure of that MySQL database is described in a separate document. This structure was fixed and may not be changed any more, otherwise the charge component cannot run error free. Accounting data is added to the database after starting a session, during



a session and after the session was closed. The charging component will then periodically contact the accounting database and extract new records that have not been yet used for charge calculation purposes. For the postpaid scenario it is sufficient that the charging component will contact the database only once a day, typically during a period of low accounting activities. The charging component uses another database—the charging database— to manage the charges and tariffing schemes of the users. On the basis of the accounting data and the entries in the charging database, the appropriate tariff will be applied. After the charge calculation is finished the result (i.e. the charge) is written back in the charging database. In the accounting database all entries that have been used for the current charge calculation, will be marked accordingly or copied to another database. There won’t be any charge presentation to the user after a session nor billing be implemented, since this is out of scope of MobyDick.

3.8 Auditing The purpose of auditing in Moby Dick is to detect violations to the contract between user and provider at the provider side. Contract violations can happen during the following phases: registration, service request, and service usage. In order to be able to perform auditing, the respective entities need to provide audit trail containing the necessary log on actions and events. The following table shows circumstances where a contract is supposed to be violated.

When Circumstance

Registration Frequent failed registration of a user with a valid NAI and credentials (above the contracted value of service availability)

Service Request Frequent rejection of service request (after successful registration) which conforms to contract (e.g., within the bandwidth limit as specified in the contract)

Service Usage Users get less bandwidth than specified in the contract due to packet dropping or putting packets to lower priority queues and not due to the fact that users’ traffic is really of lower bandwidth

3.8.1 Logging Test The AR (including AAAC Client), AAAC server, QoS Broker and HA need to log certain actions that they carried out and events that they experienced. Which actions and events to be logged depend on the logging requirements, which are derived from the contract violation conditions. Logging has to be done for each registration/deregistration, service request, and service usage. The audit trail contains the needed log to allow for auditing. The test covers the correct logging mechanisms and the correct entries in the audit trail.

3.8.2 Auditing Test An auditor is responsible for examining the audit trail to prove compliance to the established contracts. The auditor will not directly accessed the contract information, but instead, a set of contract violation conditions, which are specified based on the contract information. The auditing requirements, derived from the contract violation conditions, define the tasks the auditor should perform. The test shows whether the auditor reports correctly for both cases: no contract violation and a contract violation exists.



4. User evaluation test description

4.1 Introduction Instead of writing down a list of trials, we will first give a classification of them, so as to distinguish between a few of groups sorted by complexity. “Complexity” is related to the number of applications running simultaneously at a given time, so we will have:

1. Basic tests: there is no interaction (in terms of traffic) with other applications, so individual services offered by network are evaluated. However, changes in the network state will be provoked (handover, traffic load…), so as to test its behaviour in dynamic circumstances.

2. Complex tests: here different applications are running on the same mobile terminal(s), so the behaviour of different services can be tested at the same time, with interactions possible between their traffic generated. Therefore, traffic patterns and priorities should be taken into account while evaluating the network, in order to validate system performance (by analysing the results).

3. Final tests: these are tests a little bit more complicated than the complex ones, which should evaluate the network performance in all aspects (mobility, accounting…) with normal conditions of operation.

Also, another group will be considered: system friendliness tests, considering network behaviour but, instead of related to its performance, focused on information provided, usage simplicity. So users may evaluate if they feel to be lost in some circumstances.

4.1.1 Types of applications Deliverable D0102 defines the applications that will be used during the trials. Four paradigms of traffic patterns are considered:

• Streaming: playback of a multimedia file from a remote server (called stream_app);

• VoIP: with proper servers and clients (SIP or H.323) (VoIP_app);

• Interactive playing game: using any existing and possibly free-source code application, migrated to IPv6 (game_app);

• File download: performing a file download by the deployment of a standard download application, in order to test a connection oriented TCP application (downl_app).

From a “static” point of view, tiny objections may be risen against the previous list, because the chosen paradigms are representative for most of the existing interactive applications. On the other hand, from a “dynamic” point of view (i.e.: dealing with terminal mobility), some criticism may arise indeed, so lets justify why testing the mobility is important with all the three kinds of application.

First, mobility when dealing with voice is a must, because it is the most important service (in terms of quantity and earnings) offered by operators, so if an architecture like Moby Dick aims to provide a 4G infrastructure must take into account VoIP applications.

Second, Uusage of streaming applications while moving should also be considered, because it

will provide a framework for future development of IP-based radio terminals. Moby Dick users shall be able to move while listening to their favourite music, without the need for a huge hard drive capacity (i.e.: downloading the music in real time). Of course, adding more bandwidth enables video, with new possibilities.

Third, file downloading applications must also be used, not only because of its own significance,

but also because the may introduce background traffic. Further, for reasons of fairness



between TCP and UDP applications this kind of traffic will compete with UDP traffic and the impact of this two kind of traffic in a heterogeneous mobility scenario will be quantified.

Finally, interactive game applications must also be tested with mobility. The reason might not be

seen at a first look (e.g.: it is hard to imagine oneself playing Quake 2 while moving). But if we change the application from Quake 2 to Super Mario Kart, or similar –i.e.: any actual game for Gameboy or other console- then it is easy to think in a future 4G network, which provides the capability of network playing (with proper terminals) while moving in the bus, or metro; or maybe for the kids in the back seats of the car. An also, it may useful to change one’s terminal position while playing, without the need of first disconnecting the game, and then restart the session (because points and/or classification is lost).

4.1.2 Test-scenario overview The following summary enlists the test-scenarios: Applications Reg / Dereg Intra-HO Inter-HO Paging QoS Phase stream_app X X X X VoIP_app X X X X game_app X X X X downl_app X X X X

Expert evaluation

tests

stream_app X X VoIP_app X X game_app X X downl_app X X

User evaluation

tests Basic

stream_app X VoIP_app X game_app X downl_app X

User evaluation

tests Complex

4.2 System friendliness tests

4.2.1 Call set-up latency Network service set-up latency is an important factor in evaluating the overall performance of the system. It gives indication on the robustness of the interactions between AR, BB and AAAC Server. In a QoS enabled system the network service set-up latency is an important contributor to the overall call-set up latency. By call setup latency we mean the time elapse from the moment the user requests the service to the moment he can start to use the service. Therefore the call setup latency can be used for evaluation of the network service-set up latency, provided that the system is over-dimensioned (the processing power of the physical elements does not impact the result of the measurement). The objective of this test is to evaluate the request-response time with a qualitative measures, and check it against expectations of an average user. Here, the call setup latency can be quantified as imperceptible, perceptible, acceptable or unacceptable.

4.2.2 Usability One of the key factors affecting the level of satisfaction of a user with the services offered by a provider, and overall perception of a system is the status (or state) notification. The notifications must be clear and intuitive so that the user knows the status of the network or result of his actions and choices. Notifications may also guide a user, prompt for actions or suggest steps to be taken to resolve a situation. In this test a user evaluates network status notifications in different situations (normal operation, lack of network resources, when user is not permitted to access network, when handover is not possible, etc. …) and comments on the departures of the actual behaviour of the system from his expectations. The objective of this test is to check the system (user-friendliness) and determine those phases of user interaction with the network that need improvement (e.g. registration, service request, de-registration etc.).



4.2.3 Coverage test The goal of this test is to check the coverage area of the radio technologies and the ability of the mobile terminal to move out and back in this area.

4.3 Basic tests

4.3.1 Basic stream_app The user starts his Moby Dick terminal by turning it on. It is connected over the Ethernet. After successful registration, the playback of a mp3 from a remote server is started. Then, the evaluator will disconnect the Ethernet, take a walk on the building (Figure 12) while listening carefully to the music, in order to check for sustained playback or discontinuities (writing down when and how did they appear). The walk will conclude wiring again the terminal to the network. This test is to provide some first indication about what level of QoS is required at all in the Moby Dick specific architecture for voice services. Of course, results of this test depend significantly on the higher layer mechanisms of the applications e.g. adaptive coding, but it should give a first indication about what kind of QoS is satisfactory fora user.

Figure 12. Tour for the trials (Subnet 1 and Subnet 2 belong to the same domain)

Technical notes: this tests objective is to evaluate network performance in a handover situation, not only inter technology (from wired to wireless domain) but also intra technology (because the area covered by the user will be served by two or more Access Routers). The application for streaming should be properly configured, not only its QoS parameters (if needed) but also its playing buffer size.

4.3.2 Basic VoIP_app – first test Like in the previous test, the user starts VoIP_app while on the Ethernet, initiating a conversation with another user from its terminal. Once established the flow of voice, he will walk the same path shown in the figure 12., again with a reasonable speed. Anew, he should write down when and how misbehaviour of the service appear; thus, a reasonable voice activity index must be maintained. Technical notes: this test covers the same aspects as before (mobility) -with an application with less bandwidth constrains but more interactivity- but also, because of the relative value of the application, QoS mechanisms should provide the needed resources, while AAAC system must be fully abreast of events.

Wireless Subnet 1

Access Network 1

Access Network 2

Access Network 3

Wireless Subnet

2

Wireless Subnet 3

A

BC

D

E



4.3.3 Basic VoIP_app – second test Again, the user should start his terminal in the wired Ethernet, but now he should remain inactive. Then he will take the same walk as in the previous tests (Figure 12), but once completed half the walk, he should receive a call from another user. Technical notes: here we are interested in testing paging functionality, in three steps: entering the dormant mode, sending paging area updates while inactive, and being paged when receiving a call.

4.3.4 Basic game_app – first test Here, the user starts its terminal, launching the network game and joining a previously configured match, all from a wired interface. Then he disconnects himself from the wired network, acting as an spectator and, of course, maintaining his score, while going away but taking his terminal with him. During all this time the wireless interface provides the connectivity to the network. Then he returns to a wired. Technical notes: this first test with the game application is a very simple mobility case.

4.3.5 Basic gamme_app – second This test is, again, very similar to the ones discussed before (stream_app and VoIP_app). However, due to the nature of the application considered (a Quake-like game), with high constrains on users ability (both hands are needed), a vehicle is needed. If such kind of vehicle is unavailable, it should be considered joining the game as a passive player (spectator). Technical notes: again, this test does not involve complex mechanisms inside the network, due to the relatively low resources needed by game_app.

4.3.6 Basic downl_app The users performs the previously defined walk once more, while downloading a file (e.g. mp3). As a result, the user needs to note the required download time and data rates. Technical notes: This test intends to show the benefit of fast handover procedure beyond real time applications; by the deployment of fast handover mechanism, the performance of this TCP connection should not be significantly reduced, because TCP is not forced to enter the slow-start mechanism, but can re-transmit potentially lost packets via the fast recovery procedure. In order to proof this, the same file could be downloaded without mobility and at fixed, known bandwidth.

4.4 Complex tests

4.4.1 Stream_app with background traffic Once the user is properly registered to the network, he starts downloading a very large file, via the ftp tool. Then, he launches the stream application, playing the remote file while the downloading continues. It should be checked what happens to the former (supposed the most priority traffic), while the latter descends its rate of transmission –in order to keep the total bandwidth used to a reasonable level-. Considering the previous results of the basic download test, the impact on the TCP transmission can be verified, when the user again notes time and data-rates of the download. Technical notes: this test aims to evaluate provisioning QoS mechanisms, not only the Bandwidth Broker ones, but also the ones related to the Access Router and the Core Network (DiffServ). Depending on the existing background load, it may be necessary to introduce more traffic to the network.

4.4.2 VoIP_app and game_app – first test Several users should launch the game application, in order to provide a significant background traffic –while evaluating the interactivity-. Once all joined into the same match, they (or others users) should start launching VoIP_app, and making calls between them. Due to the fact that the former traffic has more priority than the latter, users should evaluate voice quality –most important issue- and game interactivity –secondary objective- (because VoIP calls are not considered related to the game). Technical notes: due to the fact that game_app does not generate larges amounts of traffic (see D0102), it might be needed to launch other applications, in order to test network behaviour while in congestion. With this tests proper traffic distinction is evaluated.



4.4.3 VoIP_app and game_app – second test The test is almost identical to the previous, but in this case game_app traffic is the most priority one. Here we are modelling the situation of coordinate-playing: people in the same team might communicate, but the game itself is more important than the voice. Technical notes: same as beforein the previous section.

4.4.4 VoIP_app, stream_app and game_app Now, not only game_app traffic is present before launching VoIP_app, but also 50% of users will launch the remote reproduction of an audio file –their favourite soundtrack for the game-. Then, the test will continue as stated previously (users keep on playing while listening to the music and making calls), keeping an eye on traffic performance (making both internal and external calls –because, again, we are considering the calls not related to the game). Technical notes: here we evaluate, again, network behaviour for different classes of traffic, under several load circumstance. It is also important to check for proper AAAC system behaviour, because the framework is well suited (a considerable number of sessions at the same time). The percentage of users that will launch the stream_app might need for redefinition, in order to put the network into a high load situation (but avoiding congestion). Priorities should be established in the following order: VoIP_app, game_app, stream_app (from most to least priority traffic – we are considering stream_app related to the match).

4.5 Final tests

4.5.1 VoIP_app and stream_app, moving – first test This test is the merge of those defined in the Basic tests section. Therefore, the user should start its terminal in a wired interface, launching the stream_app application. Then, while moving, he should make a phone call using the VoIP_app, keeping an ear on the music quality, and the other on voice quality. His tour, again, will be the one shown in Figure 12, but maybe in the reverse direction. Technical notes: not only handover issues are evaluated here, but also QoS provision and classification, and accounting mechanisms. However, depending on the implementation of VoIP_app and stream_app, it may be possible that this test becomes unrealisable, because of the impossibility of using the sound card by two applications at the same time.

4.5.2 VoIP_app and stream_app, moving – second test This test is almost identical to the previous one, but with a difference: now two users will do the tour, starting from the same point but in different directions (so they will pass each other by). Technical notes: the issues evaluated are the same, but network operation conditions are also changed in real time but other user. Therefore, mobility is emphasized here, due to the fact that in some situations both user might be changing their point of attachment to the network at the same time.



5. Test definitions

5.1 Initial Registration in foreign domain

Test definition for Expert Evaluation Name: Initial registration in foreign domain Test number: E1 Objectives: To test the registration process. Description: The goal of this test is to have all components successfully installed which are needed as base for the AAAC/MIPv6 and AAAC policy engine implementation. The tests therefore mainly comprise of tests regarding the base components/packages used as well as some first preliminary tests of developed software which are made during the development process. Only test applications are used during this test. Functionalities to be tested:

Traffic marking Flow authorization Traffic shaping Resource reservation

WP2

Resource allocation Registration in HA Paging Area update while being dormant Intra tech. FHO Basic mobility Inter tech. FHO

WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration URP to Diameter mapping at AR Metering

WP4

Session charging Procedure: Starting conditions: MT is switched on in a foreign domain Action Observation 1 Start registration MT discovers AR in the foreign network,

knowing it has to register at home AA_Request_Message is sent to AR Test messages: AAAC.F � AAAC.H, AAAC.H � HA, HA � AAAC.H Test AAAC.H � AAAC.F (Diameter message AA_Response_Message), AAAC.F � AR AR sends AA_Response_Message MT uses AR as default router MT has an IPsec AH tunnel with AR If MIPv6 BU was not piggybacked, MT will start MIPv6 registration procedure

Additional remarks:



5.2 Initial Registration in foreign domain (AR+AAAC Client behavior)

Test definition for Expert Evaluation Name: AR+AAAC Client behaviour Test number: E2 Objectives: Test the AR+AAAC Client behaviour Description: The basic goal of this test is to verify that the correct messages are forwarded to the AAAC.f server, whereas user data packets are blocked. Functionalities to be tested:


WP2


WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration Metering

WP4

Session charging Procedure: Starting conditions:

... Action Observation 1 Start sending application traffic AR does not forward/route any packet coming on

any of its interfaces from an unauthenticated or unauthorized source (MT)

2 Start registration

AR accepts and processes URP messages containing an AA_Request_Message. AR constructs (from the AA_Request_Message received from the MT) and sends DIAMETER AA reg. request to the AAAC.f When the AR receives an AA_response_message containing no error codes sends an AA response message to the MT and sets up its routing tables/network filters accordingly that is granting access to the network to the MT.

Additional remarks:



5.3 Profile transfer

Test definition for Expert Evaluation Name: Profile transfer Test number: E3 Objectives: Since Moby Dick supports user mobility, via the correct profile distribution the users are decoupled from a terminal which gets from the network layer an IP address assigned. So the correct profile is transferred via the whole AAAC infrastructure to both, the QoS Broker and the AAAC Attendant. Here first the correct profile should be transferred and second, the corresponding correct subset of the profile must be forwarded. Description: A user registers using URP. The correct credential provides the correct profile and the corresponding subsets are forwarded. Functionalities to be tested:


WP2


WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration AAAC.f communication with BB Metering

WP4


... Action Observation 1 User logs in (User ID and incorrect

password) Message is forwarded from AR to AAAC.f Any other message is blocked at AR AAAC.f forwards message to AAAC.h Error message is returned

1 User logs in (Correct password) Message is forwarded from AR to AAAC.f Any other message is blocked at AR AAAC.f forwards message to AAAC.h Profile is transmitted to AAA Attendant at AR Router is configured properly

2 MT generates traffic exceeding QoS profile parameters (UDP)

3 TCP stream is used to verify profile context 4 A new user logs in A new profile is transferred and same procedure

is performed, but with profile context of user 2 Additional remarks:



5.4 Intra and inter domain mobility



Test definition for Expert Evaluation Name: Intra and inter domain mobility Test number: E4 Objectives: Test MIP functions like use of home address option and routing option, test fast handover procedure in the same AAA domain and IPsec, test inter-domain handover Description: The user starts and ftp application, moves from one WLAN cell to another and then moves to another trusted AAAC domain. Functionalities to be tested:


WP2


FORMCHECKBOX

WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration AAAC interdomain HA Metering

WP4


The MT is already registered Action Observation 1 Start an ftp session to a host in its Home

Network Packets are sent using Home Address option and Routing Option Tunnel HA � MT is established, if no routing optimization is used because of security concerns

2 The MT moves (ftp seesion running) into another wlan cell belonging to the same AAAC domain.

Watch the FHandover procedure (the seq. of the packets sent by the MT) Test MT has an IPsec AH tunnel with the AR Test that the ftp session is preserved oAR has to include an [AAA option] into the HI message nAR processes the [AAA option] sets up its routing tables/network filters accordingly, that is granting access to the network to the MT; it also issues an HACK to the oAR.

3 The MT moves into another wlan cell in another AAAC domain

Test the Intradomain registration (using packet sniffers at FA, AAAC.f, AAAC.h, HA) Test that the ftp session is preserved after a successful re-registration

Additional remarks:



5.5 Basic Fast Handover

Test definition for Expert Evaluation Name: Basic Fast Handover Test number: E5 Objectives: Test Fast Handover mechanism, focusing mainly on behaviour, rather than performance (latency, loss). Description: A “ping6 dialogue” is initiated from both the Correspondent Node and the Mobile Terminal, sending at one message per second –therefore, proving only that fast handover maintains ongoing sessions-. Functionalities to be tested:


WP2


WP3


WP4


The MT is already registered Action Observation 1 Start ping6 on MT and CN No packets are lost 2 Force the Fast Handover Test if session is kept alive Additional remarks:



5.6 Real time audio stream Fast Handover



Test definition for Expert Evaluation Name: Real time audio stream Fast Handover Test number: E6 Objectives: Test the real time capabilities of the system during a Fast Handover Description: First, a multimedia session (audio or video) is established between the CN and the MT; then a FHO is initiated, but no interruption nor degradation should be noticed Functionalities to be tested:


FORMCHECKBOX

WP2

Resource allocation FORMCHECKBOX

Registration in HA Paging Area update while being dormant Intra tech. FHO Basic mobility Inter tech. FHO

WP3


WP4


The MT is already registered Action Observation 1 Start the multimedia session (audio or video)

between the MT and CN No packets are lost, and quality is high

2 Force the Fast Handover Look for interruptions or degradations during the handover

Additional remarks:



5.7 Dormant roaming

Test definition for Expert Evaluation Name: Dormant roaming Test number: E7 Objectives: Test paging area updates Description: The mobile terminal will be moved between Access Routers of the registered paging area. Since in general sending of MIPv6 location update signaling messages is transparent to the user anyway, he won’t realize suppressed MIPv6 signaling in dormant mode. Only when entering a new paging area, release of paging area update signaling could be indicated in a GUI or in /var/log/messages on the mobile terminal’s screen and on the Paging Agent. There is no need to deploy any application for this test Functionalities to be tested:


WP2


WP3


WP4


The MT is in dormant mode Action Observation 1 Change paging area The area is updated Additional remarks:



5.8 Mobile initiated dormant de-registration

Test definition for Expert Evaluation Name: Mobile initiated dormant de-registration Test number: E8 Objectives: Test the mobile’s capability to enter active mode by itself Description: When the user starts an application that requires packets to be sent through the network, mobile terminal initiated active state transition is transparent to the user. Only an increased delay could be realized before the first application specific packet can be sent. If required, appropriate indications could be provided through a GUI or in /var/log/messages. Any application referred to in D0102 should be fine for this test Functionalities to be tested:


WP2


WP3


WP4


The MT is in dormant mode Action Observation 1 Start and application from MT No misbehaviour nor excessive delay is

noticeable Additional remarks:



5.9 Paging initiated dormant de-registration

Test definition for Expert Evaluation Name: Paging initiated dormant de-registration Test number: E9 Objectives: Test the network (PA) capability to force the MN to enter active state Description: When a packet is to be delivered to a dormant mobile terminal, the PA buffers this packet and pages the mobile terminal. The dormant mode and paging process is transparent to the user. He realizes only the reception of the first packet after re-activation of the mobile terminal’s capabilities. If required, appropriate indications could be provided through a GUI or in /var/log/messages. For this test, the VoIP application is the most appropriate Functionalities to be tested:


WP2


WP3


WP4


The MT is in dormant mode Action Observation 1 Start a VoIP application, and lauch a call to

the MT in dormant mode Everything works as usual (despite a little delay)

Additional remarks:



5.10 Reservation and a release of network resources

Test definition for Expert Evaluation Name: Reservation and a release of network resources Test number: E10 Objectives: To test the behaviour of network resources reservation and release procedures in the Access Router. Description: When there is no resources in a network a new registered terminal will not be granted network service. When the resources are released the new terminal can start to use a network. The procedure checking correct reservation and release of resources should be repeated for different class of service several times. Functionalities to be tested:


WP2

Resource allocation Registration in HA Loc. Update while in paging Intra tech. FHO Basic mobility Inter tech. FHO

WP3


WP4

Session charging Procedure: Starting conditions: Action Observation 1 Start an ftp application on terminals until the

network resources for a given class are exhausted

Compare the actual resource reservation on the access router against the sum of the resources in the user profiles of the involved users.

2 Close and restart applications at different terminals at random giving sufficient time for the network service to be released.

Verify that there is no resource leak.

Additional remarks: Repeat tests by involving handover rather than starting and stopping the applications.



5.11 User traffic conformance testing

Test definition for Expert Evaluation Name: User traffic conformance testing. Test number: E11 Objectives: To test the correct behaviour of traffic conditioning and allocation procedures. Description: Start a netperf application to generate a data stream according to a traffic contract and evaluate packet losses. Repeat for different user profiles and for different service classes. Repeat test for users violating their profiles Functionalities to be tested:

Traffic marking Flow authorization Traffic shaping Traffic conditioning Resource reservation

WP2

Resource allocation Registration in HA Loc. Update while in paging Intra tech. FHO Basic mobility Inter tech. FHO

WP3


WP4

Session charging Procedure: Starting conditions: Several terminals registered, network node configured to

MT is registered. Action Observation 1 Start netperf application configured to

generate traffic that fits in the user profile.

Check that the traffic gets through the access link and the access router.

2 Start netperf application configured to generate traffic that exceeds the user profile.

Observe the level of packet drops and compare goodput to the ratio of sending rate to user profile configured rate.

Additional remarks:



5.12 QoS measures

Test definition for Expert Evaluation Name: QoS measures Test number: E12 Objectives: Measure the transmission features (bit error rate, actual throughput, delay) available according to the QoS class (application) and the access technology Description: For each of the available applications, the user will make measurements of traffic data in each of the available technology. He will take a tour of the site and compare his results according to what was expected. The user will also try to go back to a previous point of attachment and check if the measured values are the same as before. Functionalities to be tested:


WP2


WP3


WP4

Session charging Procedure: Starting conditions: Action Observation 1 Generate traffic from an Ethernet interface Measure QoS parameters provided 2 Generate traffic from a WLAN interface Measure QoS parameters provided 3 Generate traffic from a WCDMA interface Measure QoS parameters provided Additional remarks:



5.13 Accounting at AAAC client

Test definition for Expert Evaluation Name: Accounting at AAAC client Test number: E13 Objectives: Test meter data retrieval and accounting request generation. Description: Upon receiving a positive AA the AAAC client starts a new session for the user. The AAAC client sends an accounting start request to the AAAC.f. During the session it sends accounting interim messages (according to accounting policies) to the AAAC.f. Finally when a session is terminated it sends a accounting stop request to the AAAC.f. Before sending an accounting request the current byte/packet counters must be read from the meter (except for the accounting start). For each accounting request the AAAC client expects to receive and accounting answer from AAAC.f. In case no such answer is received after a timeout the AAAC client will resend the accounting request. Functionalities to be tested:


WP2


WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration Accounting Message Forwarding Metering Accounting Data Storage

WP4


MT is registered Action Observation 1 Session start AAAC client informs meter about session start

and sends accounting start to AAAC.f Wait for answer from AAAC.f

2 During session AAAC client polls meter data and send interim accounting request Wait for answer from AAAC.f

3 Session end AAAC client polls meter data and send accounting stop request Wait for answer from AAAC.f

Additional remarks:



5.14 Accounting at AAAC.f

Test definition for Expert Evaluation Name: Accounting at AAAC.f Test number: E14 Objectives: Test the correct behavior of AAAC.f when adding local accounting policies and the correct forwarding of accounting requests and answers to AAAC.h and AAAC clients respectively Description: Upon receiving a positive AA answer the AAAC.f may add additional local accounting policies to it. The AAAC.f must not override any policy already given by AAAC.h. The AAAC.f forwards the answer to the AAAC client. The AAAC.f receives accounting requests from its AAAC clients. It inspects the user-name in the accounting request and routes the request to AAAC.h. If no user-name is specified in the accounting request it uses the session ID to get the corresponding session information which include the user-name. The AAAC.f forwards accounting answers from AAAC.h to the client. Functionalities to be tested:


WP2


WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration Accounting Message Forwarding Metering

WP4


MT is registered Action Observation 1 Session start AAAC.f optionally adds additional local

accounting policies to AA answer. It does not override any policy given by AAAC.h

2 Accounting start AAAC.f receives accounting request from clients, and routes them to AAAC.h User-name or Session ID is used to get information AAAC.f forwards accounting answers from AAAC.h to the client.

Additional remarks:



5.15 Accounting at AAAC.h

Test definition for Expert Evaluation Name: Accounting at AAAC.h Test number: E15 Objectives: Test the correct processing of accounting requests and the correctness of accounting answers. Test accounting data storage in a Data Base. Description: Processes accounting requests and replies with accounting answers to the sender. In case the accounting request could be correctly decoded and contains all necessary fields it replies with a positive reply; otherwise with a negative. The AAAC.h stores the accounting data in its database. Functionalities to be tested:


WP2


WP3


WP4


Session has been properly started Action Observation 1 Accounting request is sent by AAAC client AAAC.h receives the request, and processes it.

AAAC.h replies appropriately Additional remarks:



5.16 Charging at AAAC.h

Test definition for Expert Evaluation Name: Charging at AAAC.h Test number: E16 Objectives: Test charge calculation. Description: Charging calculates the price for a given service consumption based on session and accounting information. It maps technical values into monetary units, and therefore applies a given contractual agreed upon tariff. Tariff applied might depend on QoS parameters as well as time dependent parameters, e.g., time of the day, day of the week a service is invoked. Functionalities to be tested:


WP2


WP3


WP4


Accounting data was stored properly in the MySQL database. Charging database (MySQL) running.

Action Observation 1 Access accounting database Indication of successful connection 2 Get new accounting data Execution of MySQL database query 3 Access charging database Indication of successful connection 4 Select Tariff Find tariff ID of tariff to be applied for current

session and user 5 Charge calculation Mapping of technical accounting values into

monetary units Execution of tariff algorithm

6 Save charge in charging database Execution of MySQL database query 7 Mark accounting data in the accounting

database as used Execution of MySQL database query

Additional remarks:



5.17 Logging Requirements

Test definition for Expert Evaluation Name: Logging Requirements Test number: E17 Objectives: Verify that the logging mechanisms are functional and the logging entries are correct Description: For each phase in the service life cycle from registration, through service request, service usage to deregistration or service termination, the following entities: AR, AAAC Server, QoSB, and HA must log specific actions and events according to the logging requirements, derived from contract violation conditions Functionalities to be tested:


WP2


WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration Logging in AR, AAAC Server, QoSB, and

HA Metering

WP4

Session charging Procedure: Starting conditions: Action Observation 1 Registration in home domain Check that AR, AAAC.h, and HA create the

required logs in the Audit Trail 2 Registration in foreign domain Check that AR, AAAC.f, AAAC.h, and HA

create the required logs in the Audit Trail 3 MT runs application with bandwidth

requirements within the contracted limit Check that AR and QoSB create the required logs in the Audit Trail

4 MT runs application with bandwidth requirements above the contracted limit

Check that AR and QoSB create the required logs in the Audit Trail

5 Deregistration Check that AR, AAAC, QoSB and HA create the required logs in the Audit Trail

Additional remarks:



5.18 Auditing Requirements

Test definition for Expert Evaluation Name: Auditing Requirements Test number: E18 Objectives: Verify that the logging and auditing requirements are sufficiently defined to allow for detection of contract violations. Also verify that the auditor is capable of contract violation detection. Description: Based on the auditing requirements the auditor examines the audit trail to detect any violation to contract between user and provider. Functionalities to be tested:


WP2


WP3

Basic paging functionality AAAC.h registration MT-AR IPsec tunnel established AAAC.f registration Logging in AR, AAAC Server, QoSB, and

HA Metering Auditing

WP4

Session charging Procedure: Starting conditions: audit trail and accounting data are available Action Observation 1 Runs the auditor with audit trail containing

log of actions and events where contract violations have occured

Check whether the auditor can detect the violation

2 Runs the auditor with audit trail containing log of actions and events where no contract violation has occured

Check that the auditor reports correctly

Additional remarks:



5.19 Basic stream_app test

Test definition for User Trials Name: Transmission of stream traffic with mobility Test number: U1 Objectives: Test quality of streaming sound while moving , testing fast handover reliability for this kind of traffic. Description: Once registered to the network –via wired interface-, the user will start the reproduction of a mp3 file from a remote server. Then he will take a tour (see figure), evaluating quality of sound. At last, he will re-wire his terminal to the network. Functionalities to be tested:


WP2

Resource allocation Registration in HA Paging Area update while being dormant Intra tech. FHO Basic Mobility Inter tech. FHO

WP3


WP4


Action Done 1 Launch the stream_app application 2 Walk through section A, B, C, D and E while listening to the sound 3 Return to the first point of attachment Evaluation: Mark

(1-5) Issue: Quality of sound in A,B 1 Comments:

Issue: Quality of sound in C 2 Comments:

Issue: Quality of sound in D, E 3 Comments:

Additional comments:

General perception of the service:



5.20 Basic VoIP_app – first test

Test definition for User Trials Name: Transmission and reception of voice traffic with mobility Test number: U2 Objectives: Test quality of a conversation, while moving, testing fast handover reliability for this kind of traffic. Description: Once registered to the network via Ethernet interface, the user will start a conversation with another user. Then he will take a tour (see Figure 12) evaluating quality of sound. At last, he will re-connect his terminal to the network via Ethernet again. Functionalities to be tested:


WP2


WP3


WP4


Action Done 1 Launch the VoIP_app application 2 Walk through section A, B, C, D and E while listening to the sound and talking 3 Return to the first point of attachment Evaluation: Mark

(1-5) Issue: Quality of sound in A,B 1 Comments:



Additional comments: General perception of the service:



5.21 Basic VoIP_app – second test

Test definition for User Trials Name: Paging functionality and voice application Test number: U3 Objectives: Test paging functionality (both entering and exiting dormant mode, and paging area updates) using VoIP_app Description: Once registered to the network –via wired interface-, the user will remain inactive. Then he will take the tour shown in the figure, receiving a call from another user when done the first half of the walk. Functionalities to be tested:


WP2


WP3


WP4


Action Done 1 Remain Inactive for 2 minutes 2 Walk through section A, B, C, and D 3 Receive a call and then return to the first point of attachment Evaluation: Mark

(1-5) Issue: Quality of sound 1 Comments:




5.22 Basic game_app – first

Test definition for User Trials Name: Interactive gaming with basic mobility Test number: U4 Objectives: Test basic mobility for interactive gaming application, by changing the point of attachment to the network for a while. Description: The user joins a network game, playing for a while, and then he moves to another location (using wireless interface). After an amount of time, he returns back to a wired interface. No fast handover mechanisms is needed here. Functionalities to be tested:


WP2


WP3


WP4


Action Done 1 Start playing 2 Go to lunch, and keep watching the game 3 Return to the first point of attachment Evaluation: Mark

(1-5) Issue: Game performance while lunching 1 Comments:




5.23 Basic game_app – second

Test definition for User Trials Name: Interactive gaming with mobility (fast handovers) Test number: U5 Objectives: Test performance of game_app while moving through different interfaces, and keeping playing in the game –so fast handovers are evaluated. Description: Registered to the network via Ethernet the user will start playing an interactive game. Then he will take a tour (see Figure 12), evaluating game behaviour (i.e.: if sometimes the screen blocks, …). At last, he will re-connect his terminal to the wired network. Functionalities to be tested:


WP2


WP3


WP4


Action Done 1 Start playing 2 Move through section A, B, C, D and E 3 Return to the first point of attachment Evaluation: Mark

(1-5) Issue: Quality of game playing in A,B 1

Comments:

Issue: Quality of game playing in C 2 Comments:

Issue: Quality of game playing in D, E 3 Comments:




5.24 Basic downl_app

Test definition for User Trials Name: File download with mobility Test number: U6 Objectives: Test performance (bit rate, download time) of downl_app while moving through different interfaces so fast handovers are evaluated. Description: Registered to the network via Ethernet the user will start downloading a file. Then he will take a tour (see Figure 12), evaluating bit rate and download speed. At last, he will re-connect his terminal to the wired network. Functionalities to be tested:


WP2


WP3


WP4


... Action Done 1 Start downloading the file 2 Move through section A, B, C, D and E 3 Return to the first point of attachment Evaluation: Mark

(1~5) Issue: Bit Rate and download speed

1

Comments:




5.25 Stream_app with background traffic

Test definition for User Trials Name: Stream transmission of music while downloading a file Test number: U7 Objectives: Test resource reservation mechanisms with streaming and background traffic. The former should receive higher priority (i.e.: no degradation of performance) than the latter. Description: First, downloading of a file will be launched –via ftp-. Then, streaming from a server will be started, evaluating network performance with reservation and shaping mechanisms. Streaming traffic should not suffer any significant misbehaviour. Functionalities to be tested:


WP2


WP3


WP4

Session charging Procedure: Starting conditions: stream_app must mark its packets with an higher codepoint than ftp, and

the user must have a proper (i.e.: high quality) contract with the network. Probably, background traffic should be generated, so as to put the network near congestion. The terminal is connected via wired interface.

Action Done 1 Start downloading with ftp (or similar) 2 Launch stream_app, and play a music file Evaluation: Mark

(1-5) Issue: Rate of download at the beginning 1 Comments:

Issue: Rate of download once streaming started 2 Comments:

Issue: Quality of sound 3 Comments:




5.26 VoIP_app and game_app – first test

Test definition for User Trials Name: External conversations while playing game_app Test number: U8 Objectives: Test resource reservation mechanisms for voice, considering it outside of the game (so it should maintain proper behaviour, despite the needs of game_app) Description: At the beginning several users should be playing the game; then, they should make calls between themselves and to users outside the subnet, evaluating the quality of voice and the interactivity of the game. Functionalities to be tested:


WP2


WP3


WP4

Session charging Procedure: Starting conditions: Here the voice traffic is considered not related to the match, so VoIP_app

should mark its packets with higher priority than game_app. The terminal is connected via wired interface.

Action Done 1 Join the game 2 Make a user one call 3 Make another call to other user (outside the subnetwork) 4 Another call to a user inside the subnet 5 The last call, to an external user Evaluation: Mark

(1-5) Issue: Game performance 1 Comments:

Issue: Quality of voice for internal calls 2 Comments:

Issue: Quality of voice for external calls 3 Comments:




5.27 VoIP_app and game_app – second test

Test definition for User Trials Name: Game with voice, the latter considered part of the former Test number: U9 Objectives: Test resource reservation mechanisms for game, using voice as “background traffic”, so game traffic has higher priority –no losses may appear, no delays…-. Description: Users should be playing game_app, and making use of VoIP_app to coordinate their behaviour inside the game. However, in this case is more important game_app traffic than VoIP_app Functionalities to be tested:


WP2


WP3


WP4

Session charging Procedure: Starting conditions: Here the voice traffic is considered related to the match, so VoIP_app

should mark its packets with lower priority than game_app. The terminal is connected via wired interface.

Action Done 1 Join the game 2 Make a user one call 3 Another call to another user Evaluation: Mark






5.28 VoIP_app, stream_app and game_app

Test definition for User Trials Name: Quality of voice while playing and streaming, and session charging

Test number: U10

Objectives: Test resource reservation mechanisms for game_app, stream_app and VoIP_app, and then proper behaviour of metering and charging mechanisms Description: Users should be playing game_app and using stream_app to give a soundtrack for the latter. Later, they should make internal calls (to people inside the subnet) and external calls (to people outside). At last, they should receive a bill with their traffic consumption. Functionalities to be tested:


WP2


WP3


WP4

Session charging Procedure: Starting conditions: From most priority to least priority, traffic should be marked in the

following order: VoIP_app, game_app, stream_app. The terminal is connected via wired interface.

Action Done 1 Join the game 2 Make a user one call 3 Make another call to other user (outside the subnetwork) 4 Another call to a user inside the subnet 5 The last call, to an external user Evaluation: Mark



Issue: Quality of voice for external calls 3 Comments:

Issue: Quality of sound Comments:




5.29 VoIP_app and stream_app, moving – first test

Test definition for User Trials Name: Conversation with background streaming traffic while moving Test number:U11 Objectives: Test quality of voice and streaming while moving through different subnetworks (doing intra and inter technology handovers, involving fast handover mechanisms) Description: Once registered to the network –via wired interface-, and having started the streaming application (playing a mp3 from a remote server), the user will start a conversation with another user. Then he will take the tour (see Figure 12), evaluating quality of sound. At last, he will re-connect his terminal to the wired Ethernet network. Functionalities to be tested:


WP2


WP3


WP4

Session charging Procedure: Starting conditions: VoIP_app has higher priority than stream_app. Action Done 1 Start playing the mp3 2 Launch the VoIP_app application 3 Walk through section A, B, C, D and E while listening to voice and music 4 Return to the first point of attachment Evaluation: Mark

(1-5) Issue: Quality of voice in A,B 1 Comments:

Issue: Quality of voice in C 2 Comments:

Issue: Quality of voice in D, E 3 Comments:

Issue: Quality of sound in A,B 4 Comments:






5.30 VoIP_app and stream_app, moving – second test

Test definition for User Trials Name: Conversation with background streaming traffic while moving, with two users concurrently in the same subnetwork

Test number: U12

Objectives: Test quality of voice and streaming while moving, for two mobile users, who may change their point of attachment at the same time (so fast handovers are tested with more complexity) Description: Once registered to the network over Ethernet and having started the streaming application (playing a mp3 from a remote server), the user will start a conversation with another user. Then he will take a tour (see Figure 12), evaluating quality of sound. At last, he will re-connect his terminal to the network. Another user will do the same steps, but walking in the opposite direction. Functionalities to be tested:


WP2


WP3


WP4

Session charging Procedure: Starting conditions: VoIP_app has higher priority than stream_app. Action Done 1 Start playing the mp3 2 Launch the VoIP_app application 3 Walk through section A, B, C, D and E while listening to voice and music 4 Return to the first point of attachment Evaluation: Mark

(1-5) Issue: Quality of voice in A,B 1 Comments:

Issue: Quality of voice in C 2 Comments:

Issue: Quality of voice in D, E 3 Comments:

Issue: Quality of sound in A,B 4 Comments:






5.31 Call set-up latency

Test definition for User Trials Name: Call set-up latency Test number: U13 Objectives: To evaluate a network service set-up latency as perceived by user. Description: User evaluates request-response time when using client-server applications Functionalities to be tested:


WP2


WP3


WP4


1. Application server available in fixed network. 2. The network is unloaded.

Action Done 1 Action 1. Start the client application. 2 Action 2. Send a request to the server. Evaluation: Mark

(1-5) Issue: Evaluate the request-response time as imperceptible, perceptible, acceptable or unacceptable

1

Comments:

Issue:

2

Comments:




5.32 Usability

Test definition for User Trials Name: Usability test Test number: U14 Objectives: To evaluate status notification services Description: One of the key factors affecting the level of user satisfaction with the services offered by a provider and overall perception of a system is the status (or state) notification service. This service must be clear and intuitive so that the user knows the status or result of his actions and choices. Notifications may also guide a user, prompt for actions or suggest steps to be taken to resolve a situation. In this test a user evaluates network status notifications in different situations (normal operation, lack of network resources, when user is not permitted to access network, when handover is not possible, etc. …) Functionalities to be tested:

Traffic marking Flow authorisation Traffic shaping Resource reservation

WP2


WP3


WP4

Session charging Procedure: Starting conditions: depend on a kind of a test.

... Action Done 1 Action 1. User starts a terminal and receives a notification of the registration status. 2 Action 2. User requests a service by submitting traffic and receives status notifications

for unsuccessful requests.

Action 3. User receives a notification giving the cause for unsuccessful handover attempt.

Evaluation: Mark

Issue: Notification messages messages fit the users’ expectations.

1

Comments:

Issue:

2

Comments:




5.33 Coverage test

Test definition for User Trials Name: Moving out of the coverage area and back in Test number: U15 Objectives: Test the MN reactivity to a complete loss of contact with all the access routers available in all access technologies and how it re-starts when back in the coverage area. Description: Once registered to the network, the user will start the reproduction of a mp3 file from a remote server. Then he will take a tour and unplug or move out of the area covered by the AR/RG. When the loss of all network connections has been detected, he will move back in to check the re-start of the application. Functionalities to be tested:


WP2


WP3


WP4


Action Done 1 Launch the stream_app application 2 Walk out of all covered sections until loss of the sound 3 Return to the initial point of attachment Evaluation: Mark

(1-5) Issue: From A or B, stop and re-start of the application 1 Comments:

Issue: From C, stop and re-start of the application 2 Comments:

Issue: From D or E, stop and re-start of the application 3 Comments:

Additional comments:

General perception of the service:

Definition of Moby Dick Trial Scenarios Annexpacyna/deliverables/MobyDick/D0501-Annex.pdf ·...

Documents

Transcript of Definition of Moby Dick Trial Scenarios Annexpacyna/deliverables/MobyDick/D0501-Annex.pdf ·...