TDC Telecom’s experience with HP Network Node Manager (NNMi)

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Discover 2013 Barcelona | December 10-12, 2013

Network Node Manager i (NNMi)success story at TDC Sweden Lars Niklasson

Tyler Peterson

Tapan Shah

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2

Who are we?

Lars Niklasson • Senior IT Consultant, TDC

• Working with HP NNM and other network management tools for over 12 years

• Beta tester of NNMi + iSPIs

• Practicing Aikido for over 20 years

• Living together with Karin, five kids and three king pythons

Tyler Peterson • Senior Product Manager, Network Management Center (NMC), Hewlett-Packard Software

• Based in Fort Collins, Colorado, worked with HP and NNM / NNMi for 14 years

Tapan Shah • Expert Product Consultant, Hewlett-Packard (based in Bangalore, India)

• Working with HP and NNM product for almost 11 years now in different capacities and roles

• Taking care of customer deployments in EMEA and APJ regions for NNMi and iSPIs


Agenda

TDC background

Network topology

HP solution Integrations Tips and

tricks


TDC background


Company background

TDC in Nordics

• Parent company in Denmark

• Approx. 9,300 employees

• Turnover: 26.1 billion DKK (2012)

• A Nordic network

• Coordinated Nordic SLA

• Coordinated Nordic support

TDC in Sweden

• Offices nationwide

• 830 employees

• Turnover: 3.1 billion SEK (2012)

• 100% owned by TDC Group (Denmark)

• The Nordic Region's most modern IP network

TDC background

Network topology


tricks


TDC's product and service portfolio

Networking

• IP VPN

• Ethernet VPN

• Internet access

• Security services

• Local networks

• Data Centre

• Support for IPv4/IPv6

• Mobile data access

Unified Communications

• Telephony access

• Number services

• Meeting services

• Company switchboards

• Videoconferencing

• Contact Centre

• Telephony applications

• Mobile applications

• Mobile telephony access

TDC

background Network topology


tricks


TDC's product and service portfolio

Operating model • Public cloud

• Private cloud

• Hybrid

• Customer managed

• Hosting (data centre)

Service management • Mobile integration

• Consultancy and integration

• Service concept

• Operating services

• Installation services

Integrated communication services • We can help you take further steps in your development by taking on overall responsibility for your

communication and helping you with other aspects such as financing.

Service Management • TDC Service Online (TSO)

• Device management

• Product support

• Energy efficiency

TDC background

Network topology


tricks


Company background

Products for medium-sized enterprises • TDC Scale – a modern IP Centrex service located in the cloud which supports both fixed and mobile

connections.

• TDC Mobile – a comprehensive mobile service with subscription packages for every type of corporate user.

• TDC Internet Access – tailored to both the small and medium-sized enterprise.

• IP VPN – Connects your business, scalable, high availability, short wait speeds.

• TDC Service Online – our customer portal for optimal overview and administration of your services.

• Direct sales products

TDC background

Network topology


tricks


Customers

TDC background

Network topology


tricks

http://www.svenskmjolk.se/index.asp

http://www.volvo.com/

http://www.scandic-hotels.se/SiteHomePage

http://www.anticimex.se/default.asp

http://taz.vv.sebank.se/cgi-bin/pts3/pow/wcp/index.asp


Partners

TDC background

Network topology


tricks

http://www.extremenetworks.com/index.aspx

http://www.polycom.com/index.html?showme=y


Example on SOC/NOC services

• IT Management Network - NNMi

• IT Management Server – NNMi, Symantec

• IT Management Telephony – NNMi + IPT iSPI

• IT Report – NNMi and iSPIs

• TDC KST Services (KST stands for Communication As Service)

– KST Telco and KST LAN/WAN

TDC background

Network topology


tricks

http://hpsw.co/f3WRe68


Network topology


TDC Operator Nordic—network topology

TDC background

Network topology


tricks

TDC Operator • Monitor TDC’s own devices and LAN As A Services

• 60 000 network devices in the Nordic countries

• Performance metrics, IP SLA and NetFlow in CPEs

• Auto-creation of trouble-tickets

• One of the largest telecom networks in the Nordic countries

• Customer support centres in all four Nordic countries

• The only major operator with a fully integrated pan-Nordic IP network


TDC Operator Nordic—network topology

TDC background

Network topology


tricks


TDC Integrator SE - Customer Access Network

TDC Integrator SE • Monitor customer devices and

servers/network equpiment used for deliver Integrator services

• 4 000 network devices

• 1 500 telco switches

• 500 servers which delivers TDC services to customer

• Auto-creation of trouble-tickets TDC core

MPLS-VPN

Customer 2.

Help desk Technicians, Specialist, Tekniker

Firewall

Firewall

Router/Firewall

Authentication

Logging

Dial Manager

The Function House

NOC

Customer 1.

Firewall

Router/Firewall

ISDN

PSTN Internet

Router/Firewall VPN

ISDN dial out

IDS

Back end servers, Terminal servers

TDC background

Network topology


tricks


HP solution


Network Management Center

What is the solution?

HP applications used by TDC

• NNMi

• iSPI Performance for Metrics

• iSPI Performance for Traffic

• iSPI Performance for Quality Assurance

• iSPI IP Telephony

• ArcSight Logger/Express

Network Operation Center

• Fault availability performance change • Configuration compliance diagnostics • Advanced network services trending reporting

Automated

Secure Efficient

TDC background

Network topology


tricks

HP ArcSight Tickets


Current solution deployment – Nordic (SE/NO/FI)

TDC background

Network topology


tricks


Current solution deployment – Nordic (SE/NO/FI)

TDC Integrator Swedish

• NNMi Unlimited


• iSPI Traffic

• iSPI Quality Assurance

• iSPI IP Telephony

• Global Network Manager (RMs are SE Operator and one customer NNMi)

TDC Operator Swedish

• NNMi Unlimited


• iSPI Traffic


• Regional Manager (GM is Swedish Integrator NNM)

TDC Operator Norwegian

• NNMi Unlimited


• iSPI Traffic


• Standalone

TDC Operator Finnish

• NNMi Unlimited


• iSPI Traffic


• Standalone

TDC background

Network topology


tricks


Current solution deployment

Hardware sizing TDC Norway setup (the biggest)

• NNMi + iSPI QA, Metrics, Traffic Ext

– Physical HW 16 cores, 48 GB RAM, 500 GB DISK

• NPS

– Physical HW 64 cores, 96 GB RAM, 7 TB DISK

• Traffic Master + Leaf

– Physical HW 12 cores, 42 GB RAM, 100 GB DISK

TDC background

Network topology


tricks


Objects

TDC Operator Norway

• Nodes 18274

• SNMP Agents 17637

• Interfaces 1011753

• IP Addresses 116250

• L2Connections 19966

• Incidents 24926

TDC Operator Norway

• IP Subnets 54180

• VLAN5 36523

• Node Components 186009

• Cards 32401

• Ports 143458

• Router Redundancy Groups 1421

TDC background

Network topology


tricks


Polls

TDC Operator Norway

State Poller Health

• Status: Normal Collections requested in last 5 minutes: 71778 Collections completed in last 5 minutes: 71759 Collections in process: 123 Time to execute skips in last 5 minutes: O Stale collections in last 5 minutes: O Poller result queue length 5 min. avg.: 0.0 State mapper queue duration 5 min. avg.: 0 State updater queue duration 5 min. avg.: O State updater exceptions in last 5 minutes: O

TDC background

Network topology


tricks


Relations between systems—TDC Integrator SE

TDC background

Network topology


tricks


Other applications used

• EasySoft EasyCall - Proxy for email and SMS

• CheckPoint Mobile Access - Remote access for customers

• Op5 Monitor - Monitoring the NOC environment and TDC internal IT environment

• HP ArcSight Logger/Express

• Cisco JouleX Energy Manager

• Cisco Prime Infrastructure

• Symantec - Antivirus, backup, server monitoring

TDC background

Network topology


tricks


Services addressed by this setup

Performance analysis

• Configure NNMi to monitor performance metrics

• Monitoring and/or schedule reports weekly and monthly

• Remote access via CheckPoint Mobile Access, now working on TSO - NNMi integration for reports

Different needs – different reports

• Troubleshooting – utilization, errors, discards, CPU loads etc

• Planning – utilization, forecasts

• Burst control TDC



tricks



Traffic analysis

• Configure customers’ devices to send NetFlow to our Traffic Leaf (either placed at customer or at TDC). Depends on DNS lookup or not


• Remote access via CheckPoint Mobile Access

QA analysis

• Configure customers devices with IP SLA. Either via NNMi or manually


• Remote access via CheckPoint Mobile Access

New feature in latest NNMi

• iResponse Agent–end-to-end and capability to offer QA services to non-Cisco customers

TDC



tricks



What to think about

Pricing is relatively easy for Performance and QA but not for Traffic

• Traffic needs more analysis for correct pricing or start with POC

Remote access to the customer

• Shorter contracts use VPN over Internet

• More long term contracts use IPVPN

Security

TDC background

Network topology


tricks


Integrations and other solutions


NNMi customization / automation utilities

• Homemade auto-discovery of monitoring • SNMP polling and setting CNA which Node groups definition uses and Custom poll uses

• End customers access to NNMi • CheckPoint Mobile Access to secure the access, NNMi security groups and tenant divide the environment

• Automated NNMi tenant and security modeling • Scripting the security configuration of NNMi, eliminating human errors

• Making automatic actions flexible • Using external script accessing NNMi DB collecting more data and also do different actions based on

source node

TDC background

Network topology


tricks


NNMi customization / automation utilities

• Trap and incident management • Regular trimming based on top contributors of traps

• Reporting on incidents in iSPIs using some db queries

• Script check top 20 contributors to traps and incidents

• Incident reports without iSPIs • Script creating and emailing report of RC incidents including duration time

• Nodes Out-of-service / Not Managed report • Script check nodes reachability with ICMP on Out-of-service/Not-Managed nodes

TDC background

Network topology


tricks


Tips and tricks


QA iSPI tuning

• Beware of bugs in RTT-MIB and CBQoS in Cisco IOS when enabling QA SPI. Some IOSs reboot when SNMP is polled in these MIBs

• Disable CBQoS and create filters for IP SLA probes if you know you have lots of implementations of these configurations. Otherwise it can create a big load on the system

• Use CNA. Create node groups base on CNA. We integrate with other system to set CNA SLA for different devices automatically. It helps the NOC to prioritize issues if lots of problems happen at the same time

• We also integrate with other systems to add CNAs to show which services this node delivers

• Or use SNMP location for grouping

TDC background

Network topology


tricks


Traffic SPI tuning

• Figure the flow records/minute received by each of the leaf collectors

• Recommended: One logical leaf instance per physical leaf collector

• Maximum of two logical leaf instances per physical collector

• Based on the flow records/minute, find out uniqueness of records

• Tune the flow record object pools based on the unique records/minute

• Tune the Java Heap accordingly

• Disable auto-tuner in a high-scale traffic environment

• Adjust the traffic flow batch processing size for high-scale traffic deployment

• Talk to us in case of any further details on “how-to”

TDC background

Network topology


tricks


NPS tuning

• Sybase main and temporary cache tuning

• Dealing with dbsize.ovpl

• ETL tuning

• Importance of I/O

• Follow NPS performance tuning whitepaper for large scale deployments

TDC background

Network topology


tricks


Get started now ! https://hpln.hp.com/group/bsm-community-content

Check your BSM community for content!

• Free content, add value to BSM products

• Visit Expert Blogs for tips, tricks, hints, & workshops on BSM products and the solution in general

• Share your integrations, snippets, scripts, articles and everything else the community can benefit from

• Find answers, interact with community members

• Contact HP experts on BSM products and the solution in general

How can you benefit?

http://hpsw.co/Ri3s6NC


For more on NNM hp.com/go/nnm



TDC Telecom’s experience with HP Network Node Manager (NNMi)

Technology

Transcript of TDC Telecom’s experience with HP Network Node Manager (NNMi)