Yardstick Proposal

• Goal– Verify that the infrastructure is able to handle the

NFV application requirements• Challenges– NFV applications are very different– Complex to configure/handle– Faults within NFV application are very complex to

debug/understand

Motivation

• Simple test programs which breaks down NFV requirements into several simple test cases

• Flexible test framework to introduce new test cases when needed/new problem discovered

• Good fault isolation to be able to identify/detect faults early.

• Test framework based upon open API (OpenStack/Heat).

Solution

• Long term stability test• Make sure that NFV application survives infrastructure

upgrade• Fault injection (Alarm correlation between application and

infrastructure)• Hot migration• Infrastructure HA• ........

Use cases

Test Framework

• First version of a Test Framework– Written in Python– Uses OpenStack – WebDAV file system– Supports affinity/anti-affinity– Orchestration using Heat templates– Logging using the facility provided with Python– Documentation generated using Sphinx

• Implemented test cases– Ping – TIPC

What is done

GAT

Orchestrator Basecamp#1

Backpack Backpack Backpack

ExplorerExplorerExplorer

1…m instances

1…n instances

1

1

1

1

1

1

Abstraction API Start, stop, eval functions

webDAV

HeatNova

I want to run my test

Test Host

Basecamp#2 Basecamp#N

Cloud VM VM

• Orchestrator - Calls Heat/Nova using templates to deploy the guest application.

• Explorer - Defines an application and its functionality that executes a test (e.g. ping) to all the VMs in the scope.

• Backpack - Configures the measurements that will be taken and the way to retrieve the results.

• Basecamp - Where all the Explorers meet, get their Backpacks and are deployed.

Main components

• VM - Describes a cluster member belonging to an application.

• WebDAV File System - Provides a interface to open files via a WebDAV URL.

• GAT (Guest Application Test framework): start and stops all configured guest applications and explorers, and runs evaluation function continuously every interval for total time.

Main components

• Test cases examples: Explorers, Backpacks and Basecamps for Ping and TIPC

Implemented

• Result presentation• Swift implementation• Simplified API• Additional Networking topologies• Additional test cases and examples• …

To be developed