Introduction to Quality Assurance for Service Provider ... · 9/8/2015 · Introduction to Quality...
Transcript of Introduction to Quality Assurance for Service Provider ... · 9/8/2015 · Introduction to Quality...
Introduction to Quality Assurance for
Service Provider Network Functions Virtualization
Carsten Rossenhoevel, Managing Director European Advanced Networking Test Center
2015-09-08
Company Introduction
Vendor-neutral test lab based in Berlin Manufacturer Testing and Certification: Performance,
Scalability, Availability, Interoperability Proof of Concept testing and Audits for service providers,
large enterprises and government organizations Focusing on innovative technologies on layers 2 to 7
Unique, state of the art testing expertise for MPLS, SDN, NFV, 4G/LTE/VoLTE
Emulating fully realistic scenarios representative for today’s production networks
EANTC is 100% independent and vendor-neutral
20-year testing experience matches highest quality assurance standards
Industry Differentiation
EANTC is the most innovative independent test lab publicly active in NFV Testing
We are experienced in testing NFV use case applications and use case-agnostic NFV infrastructure
The EANTC team masters latest technologies – on par with vendor experts
We create independent reports and technical white papers with unique, truly validated content
We organize turn-key live showcases, including all the logistics
Future NFV Testing Directions
Time
Read
ines
s for
SP
Depl
oym
ent
2013 2014 2018 2017 2015
Prototype
Field Trial
Deployment
Legacy
ETSI NFV PoCs
Open Source Baseline Testing
2019 2020
EANTC’s Role in the NFV Ecosystem
• Active member • Defining testing standards for performance and
interoperability ETSI NFV ISG
• Following OPNFV, OpenDaylight, OpenStack • Will contribute to OPNFV testing projects
Open Source Projects
• Created New IP Agency Test Program with Light Reading • Intel Network Builders Partner for Testing
Industry Initiatives
• Service Provider Proof of Concept Tests • Verification of NFV functions, performance, reliability • Integration tests/audits for trials and early deployments
Private testing
First Public NFV World Showcase October 2013
EANTC staged the showcase at Layer123’s SDN & OpenFlow World Congress
Focus on instantiation & provisioning, portability and elasticity: Virtualized CPE Environment Virtualized BNG Virtualized Middleware (CDNs,
Cache Servers, Load Balancers, Application accelerators, firewalls)
Virtualized Carrier-Grade Network Address Translation (CG-NAT)
EANTC is a participating member of ETSI's NFV Industry Specification Group (ISG) which defined the use cases chosen for the demonstration
Participants in 2013:
Procera Virtual PacketLogic Software Performance Test Published October 2014
Performance Results Highlights: NFV implementation demonstrates bare metal performance Layer 7 full DPI throughput of 40Gbit/s Average TCP Response Time under 1ms
Verifying ETSI NFV Proof of Concept (PoC) #21 October 2014
PoC members showed the benefits of hardware acceleration in NFV environments for functions that are compute or network intensive
Participants provided various solutions to the gateway function
Tested at EANTC‘s lab in Berlin Results were published and presented live at the SDN &
OpenFlow World Congress 2014
PoC Participants:
NFV Industry Schedule (as disclosed)
Q4/2015 Q4/2016 Q1 Q2 Q3
ETSI NFV ISG Phase 2 Define functional blocks, ref points; focus on interop
OPNFV Brahmaputra
OPNFV C Release
Openstack Liberty
OpenDaylight Beryllium
Standards and open source progress swiftly and in parallel
Many upstream projects are getting aligned
Some vendors in turn base development on OPNFV
ETSI NFV Release 2015
NFV Projects And Ecosystems Intersect
ETSI NFV ISG
OPNFV
OpenDaylight
OpenStack There are more projects. And I am certainly wrong about the exact overlap.
NFV Commercial Solution Environment
ETSI NFV ISG
OPNFV
OpenDaylight
OpenStack
Commercial NFV Infrastructure
Based on OPNFV Commercial NFVI
Based on Proprietary Solutions
Commercial OSS
Solutions
Commercial Orchestration
Solutions
Landscape of NFV Testing
Each open source project does code/system testing OPNFV has more testing projects beyond ETSI NFV ISG
creates testing guidelines
IETF Benchmarking WG creates or coordinates performance testing standards
https://wiki.opnfv.org/testing, as of Sept 8, 2015
Most Important OPNFV Testing Projects
Functest: Basic OPNFV release testing for functionality
Yardstick: Infrastructure verification from the point of view of the VNF
Qtip: Performance benchmark suite for OPNFV platform, bottom-up approach (computing / networking / storage)
VSPERF: Virtual switch testing framework and test cases
Pharos: Distributed lab infrastructure
ETSI NFV Testing Projects
NFV001: Pre-deployment Testing Methodology NFV002: Interoperability Testing Methodology NFV003: Open source components for NFV
(report and awareness creation/human interface) Other ETSI NFV working groups tap into Vswitch performance testing Acceleration hardware implications Public information: Work program – https://portal.etsi.org/tb.aspx?tbid=789&SubTB=789,832,831,801,798,799,802,828 Public drafts - http://docbox.etsi.org/isg/nfv/open/Drafts/
Testing in the ETSI NFV Reference Model
[1] Based on GS NFV 002Document
(Multiple Vendors) Server Hardware optionally with acceleration such as DPDK
VIM: vanilla or enhanced
OPNFV, or proprietary
VNFM(s) generic or
aligned with VNFs
Orchestrator of SP’s Choice
vCPE vFW vDPI vAAA vEPC
OPNFV or other NFVI
Vswitch & network access
Storage
Potentially with individual EMs
ETSI NFV TST001
ETSI NFV TST002
We are here Future
Interoperability Reference Architectures ETSI NFV-TST002
Interoperability Features ETSI NFV-TST002
Identify topics („features“) of interoperability
Define which functional blocks are involved
Summarize all relevant interfaces
Next step: Test suite diving into these topics
Test Methodologies and Coverage
Functionality / Compliance
Interoper-ability
Performance
VNF Lifecycle Management VNF Package / Software Image Management VNF Fault and Performance Management Network Service Lifecycle Management Network Service Fault and Performance Management Virtualized Resource Management NFVI Networking Management (vSwitch) () Infrastructure Resource Management NFV Resilience () NFV Policy Administration NFV Acceleration Management ()
NFV Use Cases
Switching/ Routing
Elements
Broadband Network Gateway
Carrier-Grade NAT
Provider Edge, CPE Routers
Home Routers, Set-top Boxes
Mobile Network Nodes
Packet and Serving
Gateways
HLR/HSS
IMS
Security/ Filtering
IPsec Gateways
Deep Packet Inspection
Firewalls IPv4/ IPv6
IPS/ IDS
Virus Scanners/ Spam Filters
Policy Management
Video, Voice QoE
Session Border Controllers
AAA Servers
Mobile Policy Mgmt (PCRF)
Content Management
Cache Servers
Application-layer Load Balancers
Don’t Forget About NFV Application Testing
NFV Service Integration Validate multi-layer
independence or interoperability or performance
Verify ETSI defined requirements being met
vEPC, vIMS vGI-LAN vCPE vRouter,
vBNG
vFirewall, vContent
Filter
NFV
Inte
grat
ion
Use Case Integration
Orchestration
Management
VNF
Hypervisor
Host Hardware
NFV Use Case Validation Test multiple virtualized
solutions on the same platform(s) across all layers
Verify ETSI defined requirements being met
New IP Agency Interoperability Testing Initiative
Ensuring virtualized network function solutions work together industry-wide Infrastructure (NFVI) VNF Lifecycle Forwarding Graphs Orchestration OSS First Phase starting in Q4/2015
Phase 1 Testing
• VNF to NFVI Interoperability (vertical interop) • Service chains interoperability (horizontal interop) • VNFM to VNF interoperability (if exist) • VIM to NFVI interoperability (if exist)
VNF …N VNF 2
VIM…N VIM B
NFVI …N NFVI B
NFVI A
VNF 1
VIM A
VNFM…N VNFM 2
VNFM 1
Phase 2 Testing Orchestrator focus
Orchestrator to VNFM interoperability
VNF …N VNF 2
VIM…N VIM B
NFVI …N NFVI B
NFVI A
VNF 1
VIM A
VNFM…N VNFM 2
VNFM 1
Orch…n Orch b
Orch a
Thank you for your interest!
For further information, please contact us: EANTC AG Salzufer 14 D-10587 Berlin Germany Phone: +49.30.318 05 95-0 Fax: +49.30.318 05 95-10 E-mail: [email protected] www.eantc.de Follow us