Data Mediator Install and Admin Guide R2 (13.1.10)

8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)

http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 1/113



Data Mediator Installation and Administration Guide

P/N D357009 R2



Important Notice

Data Mediator Installation & Administration Guide iii

Important NoticeAllot Communications Ltd. ("Allot") is not a party to the purchase agreement under which NetEnforcer was purchased, andwill not be liable for any damages of any kind whatsoever caused to the end users using this manual, regardless of the form of

action, whether in contract, tort (including negligence), strict liability or otherwise.

SPECIFICATIONS AND INFORMATION CONTAINED IN THIS MANUAL ARE FURNISHED FOR

INFORMATIONAL USE ONLY, AND ARE SUBJECT TO CHANGE AT ANY TIME WITHOUT NOTICE, ANDSHOULD NOT BE CONSTRUED AS A COMMITMENT BY ALLOT OR ANY OF ITS SUBSIDIARIES. ALLOT

ASSUMES NO RESPONSIBILITY OR LIABILITY FOR ANY ERRORS OR INACCURACIES THAT MAY APPEAR IN

THIS MANUAL, INCLUDING THE PRODUCTS AND SOFTWARE DESCRIBED IN IT.

Please read the End User License Agreement and Warranty Certificate provided with this product before using the product.

Please note that using the products indicates that you accept the terms of the End User License Agreement and WarrantyCertificate.

WITHOUT DEROGATING IN ANY WAY FROM THE AFORESAID, ALLOT WILL NOT BE LIABLE FOR ANYSPECIAL, EXEMPLARY, INDIRECT, INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND,

REGARDLESS OF THE FORM OF ACTION WHETHER IN CONTRACT, TORT (INCLUDING NEGLIGENCE),

STRICT LIABILITY OR OTHERWISE, INCLUDING, BUT NOT LIMITED TO, LOSS OF REVENUE ORANTICIPATED PROFITS, OR LOST BUSINESS, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

CopyrightCopyright © 1997-2013 Allot Communications. All rights reserved. No part of this document may be reproduced,

photocopied, stored on a retrieval system, transmitted, or translated into any other language without a written permission and

specific authorization from Allot Communications Ltd.

TrademarksProducts and corporate names appearing in this manual may or may not be registered trademarks or copyrights of their

respective companies, and are used only for identification or explanation and to the owners' benefit, without intent to infringe.

Allot and the Allot Communications logo are registered trademarks of Allot Communications Ltd.

NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 ofthe FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment

is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if notinstalled and used in accordance with the instruction manual, may cause harmful interference to radio communications.

Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be

required to correct the interference at his own expense.

Changes or modifications not expressly approved by Allot Communication Ltd. could void the user's authority to operate theequipment.



Important Notice

Data Mediator Installation & Administration Guideiv

Version History

Doc Version Product Date Updates

v2b11 MED13.1 24.07.13 List of required portsammended

v2b10 MED13.1 09.07.13 ExcludedConversations CDRsclarified.

v2b9 MED13.1 26.05.13 GA Version, RecordExport Rate

clarification

v2b8 MED13.1 05.02.13 BDR Configuration

v2b7 MED13.1 24.12.12 Enabling commands,Obfuscation, Custom

Attributes added

v2b6 MED13.1 16.12.12 DistributedCollection update,

ConversationsPriority CLI changes

v2b5 MED13.1 04.12.12 DistributedCollection and DataExport Rate

information added

v2b4 MED13.1 02.12.12 General edits, XMLinformationreorganized

v2b3 MED13.1 29.11.12 Field Descriptions

v2b2 MED13.1 25.11.12 New CDR Types

v2b1 MED13.1 21.11.12



Data Mediator Installation & Administration Guide v

Table of Contents

Important Notice .......................................................................................................................... iii

Table of Contents ........................................................................................................................... v

CHAPTER 1: OVERVIEW ................................................................................ 1-1

Data Types................................................................................................................................... 1-2

Record Export Rates ................................................................................................................. 1-3

DM Performance Rates ............................................................................................................. 1-3

Data Flow – System Level .......................................................................................................... 1-4

Data Flow – Data Mediator ....................................................................................................... 1-5

Glossary ....................................................................................................................................... 1-7

CHAPTER 2: HARDWARE .............................................................................. 2-1

Overview...................................................................................................................................... 2-1

AC Power Supply ....................................................................................................................... 2-2

AC Data Mediator (PA-MED-SRV and PA-MED-SRV-HAP) ............................................... 2-2

AC Storage Device ................................................................................................................... 2-3

DC Power Supply ....................................................................................................................... 2-4

DC Data Mediator (PA-MED-SRV-HAP-DC) ........................................................................ 2-4

DC Storage Device ................................................................................................................... 2-5

CHAPTER 3: INSTALLING DATA MEDIATOR ............................................... 3-1

Connecting to the Network ........................................................................................................ 3-1

Preparing the Network .............................................................................................................. 3-1

Connecting a Single Data Mediator .......................................................................................... 3-2

Connecting an HA Data Mediator Cluster ................................................................................ 3-2

CHAPTER 4: CONFIGURING DATA MEDIATOR ........................................... 4-1

Data Mediator Configuration .................................................................................................... 4-1

Configuring Network Parameters ............................................................................................. 4-1

Connecting to Data Mediator ................................................................................................ 4-1

Configuring Device Parameters ................................................................................................ 4-6

Usage ..................................................................................................................................... 4-6

Example ................................................................................................................................ 4-6

Configuring IMM Parameters ................................................................................................... 4-6

Rebooting the Data Mediator .................................................................................................... 4-9

Data Mediator High Availability .............................................................................................. 4-9

Pre-Installation Requirements ................................................................................................... 4-9

Installing the HA Software ..................................................................................................... 4-10

Editing the /etc/hosts file ........................................................................................................ 4-11

Verify a Successful Installation .............................................................................................. 4-12

Enabling and Configuring Data Mediation............................................................................ 4-12

Enabling Subscriber Data Export License .............................................................................. 4-12

Enabling Distributed Collection ............................................................................................. 4-13



Data Mediator Installation & Administration Guide vii

CHAPTER 10: USAGE DETAIL RECORDS (UDR) ........................................10-1

Configuring UDRs on the Data Mediator .............................................................................. 10-1

Monitored Applications .......................................................................................................... 10-3

Naming Conventions ................................................................................................................ 10-4

UDR Interface Fields ................................................................................................................ 10-5 Output Example ....................................................................................................................... 10-6

CHAPTER 11: BINARY DATA RECORDS (BDR) ..........................................11-1

Enabling and Configuring BDRs on the In-Line Platform ................................................... 11-1

Legend .................................................................................................................................... 11-2

Configuring BDR Export on the NetXplorer ......................................................................... 11-3

Naming Conventions ................................................................................................................ 11-5

CHAPTER 12: SNMP INTERFACE .................................................................12-1

Data Mediator Traps ................................................................................................................ 12-1

Data Mediator KPIs ................................................................................................................. 12-1 Incoming Records ................................................................................................................... 12-2

Processed Records .................................................................................................................. 12-2

Output Records ....................................................................................................................... 12-3

CHAPTER 13: APPENDICES .........................................................................13-1

Appendix A: Data Mediator Logs ........................................................................................... 13-1

Appendix B: Upgrade Procedure ............................................................................................ 13-1

Appendix C: Customizing Data Export ................................................................................. 13-3

DataExportConfig.xml Template ............................................................................................ 13-4

HDR XML Format.................................................................................................................. 13-5

UDR XML Format.................................................................................................................. 13-6 MOU XML Format ................................................................................................................. 13-6

VC XML Format .................................................................................................................... 13-6

Conversations XML Format ................................................................................................... 13-7

SDR XML Format .................................................................................................................. 13-7

Using Obfuscation .................................................................................................................. 13-8

Example .............................................................................................................................. 13-8



Data Mediator Installation & Administration Guide 1-1

Chapter 1: Overview

In addition to its powerful core functionality, the Allot solution elements can serve as

a comprehensive data source. Working together with the Data Mediator, data can becompiled, collected, mediated and exported to external systems for the purpose ofstorage or analysis.

Various types of statistics and detail records are generated by the Allot solution. The

data must be collected periodically from the Data Source (Allot In-Line Platforms orSubscriber Management Platforms), transformed into a suitable format and then

pushed to target devices that will process or store the data according to the needs ofeach individual customer.

The Allot Data Mediator is an infrastructure component that provides:

Single and unified infrastructure for data mediation between

producers and consumers, where consumers may be internal orexternal to the system

Shorter implementation time to support collection,

transformation and transfer of new data types

Faster compliance with customer needs by supporting pluggableadapters without modifying generic infrastructure



Data Mediator Installation & Administration Guide1-2

Data TypesAllot in-line platforms, together the Allot Data Mediator, serve as a data source for

the following data types:

This data can be used to generate many different types of statistics, including:

1. Usage – based on Conversations records extracted from in-linetraffic, Allot can provide a 360 degree view of network,subscriber, and app usage for monitoring and planning

2. Mobile – based on Mobile session records (SDR/UDR)generated by the Allot SMP (Subscriber Management Platform)from RADIUS messages it receives. Allot can provide statisticson Mobile handset usage that is useful for new handsetintroduction and monitoring as well as monitor Mobile networkload e.g. by analyzing session bitrates.

3. OTT – based on MOU records, Allot can monitor the leakage ofVoice calls to OTT VoIP.

4. Domain-specific – Allot can provide HTTP records extractedfrom HTTP traffic. The customer can use that for Web traffic

analysis and subscriber profiling and trends and for generatingnew revenue streams (Proactive Analytics).

5. Policy – based on VC stats extracted from in line classified

traffic providing usage based on the service plan and subscriber

•Bandwidth per connection, service

•Customization of preferred and excluded applicationConversation

•Bandwidth per subscribers

•Customization of filtering rules

Session(SDR/UDR)

•Web activity usage per subscriber

•Define Application to monitorHDR

•Bandwidth per policy elementVC

•VOIP usage per service plan and serviceMOU

•Binary raw dataBDR




level info, Allot can provide subscriber service plan quota usage,

for service plan tuning and capacity planning NP> rephrase. VCrecords provide usage statistics per the policy elements managed

by the system. The policy elements may represent subscribers

and service plans.

Allot data feeds are delivered with minimum impact on DPI performance. Recordsinclude a unique identifier which allows association between the different statistical

record types for creating a single enriched record.

Record Export Rates

For SG-Sigma and SG-Sigma E Service Gateways, the following values are per CoreController.

NOTE 1 Data Mediator supports up to 20 CC-300 blades.

PERINTERVAL(SECONDS)

AC1400 AC3000 AC6000 SIGMASIGMAE

UDR 300180,000 180,000 360,000 480,000 1,500,000

HDR 11,000 1,000 7,500 2,500 10,000

VC 300160,000 160,000 500,000 500,000 1,200,000

VC 30 80,000 80,000 250,000 250,000 600,000

Conversation 300600,000 600,000 1,200,000 600,000 1,500,000

Conversation 30300,000 300,000 600,000 300,000 750,000

MOU 3001,600 1,600 1,600 1,600

DM Performance Rates

5 MINS AGGREGATION 10-60 MINS AGGREGATION

UDR 360M 180M

HDR 720M 360M




5 MINS AGGREGATION 10-60 MINS AGGREGATION

VC* 288M 144M

Conversation 360M 180M

SDR 22M 11M

MOU 400K 200K * The In-line platform limits the number of exported VC’s to the # of supported VC in thedistributed collector (4M per 5 Minutes).

The Aggregation value affects all exported records. For information concerningsetting this value, see Configuring Conversations Records on the Data Mediator on

page 8-1.

Data Flow – System Level

Figure 1-1: System Level Data Flow

Step 1- The RADIUS server performs the AAA functionality forthe new subscriber session.

Step 2 – The RADIUS Server passes the RADIUS accounting

message to the SMP.

Step 3 – The SMP extracts relevant subscriber information fromthe RADIUS message and passes this on to the In-line Platform.




It also generates SDR buckets which aggregate this information

on the SMP.

Step 4a – The subscriber is granted access to the Internet and

subscriber traffic flows through the In-line platform. The In-line platform records the relevant subscriber data (in a series of

different data buckets) and…

Step 4b – …forwards the subscriber’s traffic to the internet

according.

Step 5 – The Data Mediator polls the various data sources (inthis case the In-line Platform and the SMP) for the configured

data buckets,

Step 6 – The Data Mediator formats the data and sends it on to

the Data Warehouse or third-party analytics system. Data Flow – Data Mediator

The Data Mediator performs three key functions:

Collects data from in-line platforms

Transforms the data to an output format

Pushes the data to the required targets

Figure 1-2: Data Mediator Data Flow

Data MediatorData

Source

Collector

File ListTransformed

Output (4)

Transformer(3)

Purge(6)

Push (5)

DataTarget

1

2




The Data Mediator collects data from a list of configured Allot data sources.Periodically, the Data Mediator retrieves (1) the list of potential files to collect from

each data source (in-line platform or SMP)

NOTE It is the responsibility of the Data Mediator to identify the filesthat have not been processed yet, and to avoid duplicateprocessing of files.

New files are retrieved by the Collect (2) function, and depending on the type of thecollected data, a suitable Transform (3) function is executed. The Data Mediatorsupports different transform adapters per data type to facilitate changing adapters

without making any change to the infrastructure.

Transformations are either of the following types:

Simple assignments from input to output

Data type conversions – binary to textual (e.g. date, time, IP

address)

A transformed file is stored in the Data Mediator storage (4).

The system supports a push mechanism towards the Target consumers. Therefore,files are pushed by the Push (5) function.

At the completion of the life cycle, transformed files are purgedfrom the Data Mediator storage by the Purge (6) function,

according to the retention policy for the data type.




Chapter 2: Hardware

OverviewThe Allot Data Mediator is a 1U high appliance based on an IBM M3 x3550 server. Theappliance is shipped from Allot with a Linux based CentOS operating system and Allotsoftware already installed.

The Allot Data Mediator is available in the following configurations:

PA-MED-SRV (AC standalone)

PA-MED-SRV-HAP (AC HAP)

PA-MED-SRV-HAP-DC (DC HAP)

A front and rear view of an Allot Data Mediator (PA-MED-SRV) is illustrated below:

Figure 2-1: PA-MED-SRV – Front View

Figure 2-2: PA-MED-SRV – Rear View

A Data Mediator High Availability cluster is made up of two AC or DC Data Mediators

along with a Storage device. The Storage device supports the same power supply (ACor DC) as the Data Mediators it is clustered with.




Figure 2-3: PA-MED-SRV-HAP cluster with Storage device – Front View

AC Power Supply

AC Data Mediator (PA-MED-SRV and PA-MED-SRV-HAP)

Two field replaceable and hot swappable 100 VAC to 240 VAC power supplies provideAC power to the PA-MED-SRV and to each Data Mediator Node in the PA-MED-SRV-HAP.

NOTE AC Power Supply units should be removed for maintenancepurposes only.

The server is equipped with two AC power inlets which are located on the rear side ofthe server. The AC inlet connectors are also located on the rear of the server.

Figure 2-4: AC Power Inlets




Each AC power supply is capable of supplying 835w. One power supply is fullycapable of supplying the power consumed by a fully occupied server. The second powersupply is for use in cases where redundancy is needed.

NOTE Connect each power entry to a different power source forredundancy

Further power specifications for the Allot Data Mediator (PA-MED-SRV) are detailedin the table below:

Power Consumption (max) 835 W 2,851 BTU/hour

AC Power Supply 1+1 for redundancy 100-127 VAC, 10A max

200-240 VAC, 5A max

50/60Hz

Form factor / height Rack/1U

Network Interface Integrated dual Gigabit Ethernet

Power Supply (std/max) 835W 1/2 AC standard

Hot swap components Power supply, fans and hard

disk drives

AC Storage Device

The PA-MED-SRV-HAP comes with an IBM storage device

NOTE The power control button on the device and the power switch onthe power supply do not turn off the electrical current supplied to

the device. The device also might have more than one powercord. To remove all electrical current from the device, ensure thatall power cords are disconnected from the power source.

You must follow the power sequence in the order shown in thefollowing procedure. To establish power redundancy forenclosures with two power supplies, use at least two powerdistribution units (PDUs) in the rack cabinet. Split the powerconnections from each enclosure into the separate PDUs. Then,connect the PDUs to external power receptacles that are ondifferent circuits.

To connect the AC Storage Device to the power supply, follow the procedure below:

To turn on the storage subsystem ac power, complete the following steps:




6. Connect a power cord to each power supply in the storagesubsystem.

Figure 2-5: AC Power Inlet, Storage (Right Side)

7. Connect the other end of each power cord to the power receptacle.

8. Turn on the power to the switches, if applicable.

9. Turn on both power switches on all of the attached storage

enclosures, and wait 60 seconds.

10. Turn on both power switches on the storage subsystem.

NOTE When you turn off the power to the storage subsystem, completethe preceding steps in reverse order. Turn off the power to thestorage subsystem first; then, turn off the power to the storageenclosures.

DC Power Supply

DC Data Mediator (PA-MED-SRV-HAP-DC)

Two field replaceable and hot swappable -48V DC power supplies provide DC power tothe PA-MED-SRV-HAP-DC.

NOTE DC Power Supply units should be removed for maintenancepurposes only.

The server is equipped with two -48V DC power inlets which are located on the rearside of the server. DC power cables are not provided by Allot.

Figure 2-6: DC Power Inlets




Each DC power supply is capable of supplying 675w. One power supply is fullycapable of supplying the power consumed by a fully occupied server. The second powersupply is for use in cases where redundancy is needed.

Further power specifications for the Allot Data Mediator (PA-MED-SRV-HAP-DC) aredetailed in the table below:

Power Consumption (max) 675W

DC Power Supply 1+1 for redundancy -48 V DC

Form factor / height Rack/1U

Network Interface Integrated dual Gigabit Ethernet

Power Supply (std/max) 675W 1/2 DC standard

Hot swap components Power supply, fans and hard

disk drives

DC Storage Device

The PA-MED-SRV-HAP-DC comes with an IBM storage device. To connect the DCStorage Device to the power supply, follow the procedure below:

1. Make sure that the power on-off switches of both DC powersupplies in the storage subsystem and all attached storageenclosures are in the off position.

Figure 2-7: DC Power Inlet, Storage (Right side)




2. Connect a DC jumper cable to each DC power supply in thestorage subsystem and to all attached storage enclosures:

Using the supplied strap, tie the DC power supply cable to the rail

to provide strain relief for the power cable.

Connect the DC power cable to the power supply. See below for

the DC power supply connector PIN positions.

Figure 2-8: DC Pin Positions

NUMBER FUNCTION DC POWER CABLE WIRE COLOR

1 Pin 1: - 48 V dc Brown

2 Pin 2: POS RTN Blue

3 Pin 3: GND Green/yellow

3. Connect the - 48V wire (brown) of the DC power cable to anapproved disconnect device (circuit breaker) rated at 30 A. The

disconnect device must be easily accessible from the back of theStorage unit.

NOTE The disconnect device (circuit breaker) must be rated at 30 A.

Make sure that only 10 AWG copper conductor wires are used forall of the wiring between the DC power connectors and the DCpower source.

Complete the wiring from the disconnect device to the terminalmarked -48V of the reliably grounded safety extra low voltage(SELV) DC power source. Connect the POS RTN wire (blue) andthe ground wire (green/yellow) of the dc power cable to theterminals marked POS RTN and GND on the DC power source.




Figure 2-9: DC Storage Device Wiring

NUMBER FUNCTION

1 Pin 1: - 48 V DC

2 Pin 2: POS RTN

3 Pin 3: GND

4 DC power source

5 Disconnect device (circuit breaker)

4. After connecting all of the DC jumper cables, turn on the power tothe -48 V dc SELV power source.

5. Turn on both power switches on all of the attached storage

enclosures, and wait 60 seconds.

6. Turn on both power switches on the storage subsystem.

NOTE When you turn off the power to the storage subsystem, completethe preceding steps in reverse order. Turn off the power to thestorage subsystem first; then, turn off the power to the storageenclosures. Lastly, turn off the disconnect devices installed onthe -48V SELV power source.




Chapter 3: Installing Data Mediator

Connecting to the Network

Preparing the Network

Certain ports need to be open to enable the required communication between thedifferent components of the Data Mediator solution. The ports required are presented inthe table below.

SOURCE DESTINATION APPLICATION PROTOCOL PORT COMMENTS

DM NX RMI tcp 1098

Data transformationconfiguration(DataExportConfig.xml)

DM NX RMI tcp 3873 Catalogs

DM NX RMI tcp 4446 Subscriber data

DM NX RMI tcp 1099JNP server bindaddress

NX DM SNMP udp 161 set NX ip

DM NX SNMP udp 161,162 traps

Operator DM SNMP udp 161,162

MIB retreival for

KPIs

DM NE HTTP tcp 80*Manifest and rawdata files retrieval

DM SMP HTTP tcp 80Manifest and rawdata files retrieval

DM OperatorFTPSFTPSCPFile Copy tcp

21,22(or any portrequired for

communicationwith destination

server)

Push transformedfiles to operator fileserver or DWHenvironment

Operator DM SFTP tcp

21,22

(or any portrequired forcommunicationwith destination

server) Operator pullstransformed data

DM NX NTP udp,tcp 123If NX is the source ofTime Synch




SOURCE DESTINATION APPLICATION PROTOCOL PORT COMMENTS

DMExternal TimeServer NTP udp,tcp 123

If an external Timeserver of Operator's

network is thesource of TimeSynch

* If TCP port 80 is closed between the Data Mediator and the NE/SG, the NetXplorer

will report the Data Mediator is unreachable.

That is in addition to the ports which need to be open in every NetXplorer installation(see NetXplorer Installation and Admin Guide, Appendix E)

Connecting a Single Data Mediator

To connect a single Data Mediator to the network, follow the instructions below:

Connect a straight copper cable from port 1 illustrated below on

the rear of the Data Mediator to a management switch.

Connect a straight copper cable from the IMM port illustrated

below on the rear of the Data Mediator to a laptop for directconnectivity and maintenance (in blue).

Optionally, you may also connect an additional straight coppercable from the adjacent port (port 2) to a secondary management

switch for backup purposes.

Connecting an HA Data Mediator Cluster

Data Mediator Cluster Hardware Summary

Figure 3-1: Connecting a Single Data Mediator (M3 Hardware)




To connect the HA Data Mediator Cluster (2 x Data Mediators and a RAID storagedevice with dual controllers), refer to Figure 3-2 and follow these instructions:

Figure 3-2: HA Cluster Network Connections (M3 Hardware)

In a High Availability Cluster configuration, the Data Mediator units are connected bytwo physical links (and each Data Mediator server to the RAID by dedicated SAScables) as follows:

1. Use a crossed copper cable to connect between Port 3 on one DataMediator server and Port 3 on the second Data Mediator server.(illustrated in green above)

Figure 3-3: Close up Rear View of Data Mediator node in Data Mediator-HAP

(M3)

2. Use a null modem serial cable (RS 232) to connect between theSerial COM port on one Data Mediator server and the Serial COM

port on the second Data Mediator server. (illustrated in red above)

3. Use a Serial Attached SCSI (SAS) cable (see Figure 3-4) toconnect between each Data Mediator server and the RAID storageserver (as illustrated in Figure 3-5). (These connections and theexact ports required are illustrated in orange in Figure 3-2 above)




Figure 3-4: SCSI (SAS) Cable

Figure 3-5: SCSI (SAS) Cable connections to RAID Storage Server

4. Connect each Data Mediator server to the management networkvia Port 1 (shown in blue in Figure 3-2) with an additional link viaPort 2.

5. Additional management cables should be connected to the IMMmodule on each Data Mediator server for direct connectivity andmaintenance (in blue).

6. An additional optional management cable may be connected to theRAID storage server for storage management and storage traps.

The serial (SAS) cable, the crossed copper cable and the two SAS cables are providedwith the Data Mediator High Availability kit.




Chapter 4: Configuring Data Mediator

Data Mediator ConfigurationIrrespective of whether you are working with basic Data Mediator architecture or a highavailability Data Mediator cluster, each Data Mediator server that you are working with

must be prepared for use. Four initial steps are required

1. Configure network parameters

2. Configure device parameters

3. Configure IMM parameters

4. Reboot the Mediator

Configuring Network Parameters

The following network parameters need to be assigned to each of the Data Mediatorsyou are using:

Data Mediator IP and mask

Data Mediator Gateway IP for network access

DNS Server IP

Host name

NTP Time Server IP

NOTE The default gateway must be configured. If not configured, insome circumstances, you may not be able to add the SMP to theNetXplorer.

Connecting to Data Mediator

Connect a keyboard, mouse and monitor to the front of the Data Mediator server asshown in Figure 4-1.




Figure 4-1: Connecting the Data Mediator (M3) – Front View

Log in as a root user: username: root password: bagabu

1. Configure the network settings via the CentOS UI.

2. Open a network connection by selecting Administration/Networkfrom the System menu as shown in Figure 4-2 below

Figure 4-2: Open Network Configuration

3. From the “Devices” tab, select the “eth0” interface as shown inFigure 4-3 below.




Figure 4-3: Select eth0

Double click the selected interface. The Ethernet Device dialog

appears.

Figure 4-4: Ethernet Device dialog box

4. In the General tab set the IP address, Subnet mask and Default

Gateway in the “Statically set IP addresses” section.

NOTE The Default Gateway MUST be set during the initial configuration.

5. Click OK to save and return to the Network Configuration dialog.




Figure 4-5: Network Configuration dialog box, Hosts tab

6. Open the Hosts tab and click New to create a new Host.

The Add/Edit Hosts dialog appears.

Figure 4-6: Network Configuration dialog box, Add/Edit Hosts dialog

7. Enter the IP Address, Host name and Alias for the new host andclick OK to return to the Network Configuration dialog. The new

host will appear in the Hosts tab. To edit an existing Host, clickthe Edit button.

Figure 4-7: Network Configuration dialog box, DNS tab




8. Open the DNS tab and enter the Host name and DNS IPaddresses.

NOTE The Hostname entered in the DNS tab must be the same as was

added in the Hosts tab.9. Select Save from the File menu to save all changes.

10. Restart the network process via the Devices tab by selecting eth0and then Deactivate followed by Activate.




Configuring Device Parameters

Once you have installed the MD package and configured the network parameters of theData Mediator, you need to configure the appropriate device parameters in order to

inform the Data Mediator server what functional elements should be enabled.

The dev-setup command informs the Data Mediator what its identity is. You can seethe list of available parameters by running the command “dev_setup.sh” without

parameters.

NOTE If a Data Mediator has been shipped directly from Allot, thisprocess has already been done and should be skipped.

Usage

dev_setup.sh [ -v ] [ -m MODE ] [ -f ] [ -a ]

-v View device setup properties and license details

-m <MODE> Device mode to set

-f Force change of mode and key

-a High availability mode and key

The proper MODE to use in the command to configure a Data Mediator isdatamediation. Once that is set, reboot the Data Mediator.

Once the device is set to datamediation you will need to open the NetXplorer GUI toassign the Data Mediator to that NetXplorer.

Example

dev_setup.sh -m datamediation

Setting device mode to datamediation...

Request completed successfully.

Device mode successfully changed to datamediation.

Please reboot your device.

Configuring IMM Parameters

The default details of the IMM are as follows:

Default IP: 192.168.70.125

Default User Name: USERID




Default Password: PASSW0RD (where the “0” is not “o” but “zero”)

To configure the network settings of the Integrated Management Module, follow thesteps below:

1. Connect directly from a laptop to the IMM interface on the rear of the DataMediator. The interface is labeled “SYSTEM MGMT” as shown in Figure 4-8

below

Figure 4-8: IMM "System Management" Port

2. Open a web browser. In the address field type the IP address or host name ofthe IMM to which you want to connect.

NOTE If you are logging in to the IMM for the first time after installation,the IMM defaults to DHCP. If a DHCP host is unavailable, it usesthe default static IP address 192.168.70.125. You can obtain theDHCP-assigned IP address or the static IP address from theserver BIOS or from your network administrator.

3. Enter User ID and Password

4. You will be prompted to specify an inactive session timeout value. Choose avalue from the dropdown list and click on Continue.

5. You will see the IMM User Interface, with the default “System Status” in view,as seen in below:




Figure 4-9: IMM System Status Screen

6. Select “Network Interfaces” from the system tree on the left side of the screen.

7. In the Ethernet section, make sure that “interface” is enabled, and IPv6 DHCP

is disabled. In addition, DDNS status should be set to “Disabled” and “Domain Name Used” should be set to “manual”.

8. In the IPv4 section, make sure that the DHCP field is set to: “Disabled – UseStatic IP configuration”. Assign an IP, mask and default gateway as seen above

and click Save. You can now access the IMM remotely using these networksettings.




Figure 4-10: IMM Network Interfaces

Rebooting the Data Mediator

To reboot the Data Mediator server, log in as a root user and enter: reboot

Request completed successfully

Broadcast message from root (Sun Mar 1 16:59:18 2009):

Data Mediator High Availability

The High Availability (HA) solution for Allot’s Data Mediator Platform is based onusing two installations of Data Mediator for high availability applications and a sharedstorage unit for Sybase database files.

Pre-Installation Requirements

Before starting a “high availability” installation, you will need to ensure that both the

Data Mediator servers which you will be using have the same Data Mediator softwareversion.

NOTE: Make sure that both DMs are configured with the dev-setup “-a ”flag for high availability i.e: dev-setup.sh –m datamediator –a




192.168.168.2 MED-1

x.x.x.x host.example.com host

In this example, x.x.x.x represents the management interface IP address - it should be

different in each Data Mediator node.

Verify a Successful Installation

After running the hainstall script, there are several checks that you can run to verify asuccessful installation.

Verifying the creation of a cluster using the display file system command

df-h

Verify network parameters for the cluster using ifconfig – a

Check the High Availability status by entering the crm_mon command.

Confirm storage connectivity by entering the cat/proc/partitions command.

Enabling and Configuring Data MediationThe process of configuring Subscriber Data Export consists of 4 distinct stages:

1. Enabling Subscriber Data Export license via activation key

2. Adding IP addresses to each Core Controller

3. Adding a Data Mediator

4. Configuring and enabling each type of CDR.

Each of these stages is examined in detail below.

Enabling Subscriber Data Export License

In order for the Subscriber Data Export feature to work, the operator must first have purchased the correct license for each Core Controller blade on one or more of theService Gateways in the network. Only those Service Gateways for which SubscriberData Export have been enabled will be able to collect usage data to be later exported.

There are two different licenses for subscriber data export functionality:




Statistics Export License for Conv, VC, MOU, SDR, UDRexport

HTTP CDR License for HTTP CDR and BDR export

To Verify that the Proper Subscriber Data Export License is Activated:

1. In the Navigation pane, select and right-click the NetEnforcer orService Gateway in the Navigation tree and select Configuration

from the popup menu.

OR

Select the NetEnforcer or Service Gateway in the Navigation tree and thenselect Configuration from the View menu.

ORSelect the NetEnforcer or Service Gateway in the Navigation tree and then

click the Configuration icon on the toolbar.

The Configuration window for the selected NetEnforcer or Service Gateway isdisplayed.

2. Select the Identification & Key tab.

3. In the Activation Key Details pane, scroll down to check ifStatistics Export has been activated for each Core Controller in theService Gateway as shown in Figure 4-11 below. If the correctnumber of Core Controllers do not appear in the Statistics Exportline, contact Allot Customer Support at [email protected].

Figure 4-11: Subscriber Data Export License Enabled

NOTE This may also be verified from the CLI using the followingcommand:

gcy key

Enabling Distributed Collection

In multi-blade In-line Platforms Allot has two modes of data collection:




Centralized Collection: where data is collected on each Core

Controller blade and transferred to the Host Blade (SGSV-100 on

SG-Sigma or SFB-300 in slot 7 on SG-Sigma E) where it is stored.The host blade maintains a list of each data file that is stored.

Distributed Collection: where data is collected on each Core

Controller blade and stored there. The host blade maintains a

central list of each data file, and the core controller informs thehost blade which files should be added or removed from the listand on which blade each one is stored.

Distributed collection must be used with the Data Mediator. In order to enabledistributed collection you will need to assign IP addresses from the managementnetwork to each Core Controller and SFB in the system and then enable direct access to

them. The procedures for doing this are detailed below.

To set the IPs for CC blades in slots 1-5 and 10-14 in two steps:Enter the following commands:

go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots1:5


In the commands above x.x.x.x:255.255.0.0 represents the IP:Subnet Mask whilethe y.y.y.y represents the Default Gateway. The – slots value 1:5 means that theslot number will increment up from 1 to 5, while 10:14 means the slot numberwill increment up from 10 to 14.

Using these commands will cause the IP address (x.x.x.x in the example) to

increment up as follows:

CC n will get IP address x.x.x.x

CC n+1 will get IP address x.x.x.x+1

etc.

To set the IP for each CC blade individually:

1. Enter the following commands for the relevant slots:

go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 1:1













2. To view the configuration, type the following command:sysadmin@EXC-SBH[1/6]:~$ go config view

blade_mngt_ips

To set the IP for each SFB blade individually:

1. Enter the following commands for the relevant slots:





Output Example

==== Blade Management IPs ====

|Slot |IP |Mask |Gateway

--------------------------------------------------------------

|4 |10.150.6.34 |255.255.0.0 |10.150.0.1

--------------------------------------------------------------

Request completed successfully.

To enable Distributed Collection in the NetXplorer:

1. In the Navigation pane, select and right-click the desired In-linePlatform in the Navigation tree and select Configuration from the

popup menu.

OR

Select the desired In-line Platform in the Navigation tree and then selectConfiguration from the View menu.

OR




Select the desired In-line Platform in the Navigation tree and then click the

Configuration icon on the toolbar.

The Configuration window for the selected NetEnforcer or Service Gateway isdisplayed.

2. Select the IP Properties tab.

3. In the Direct Access pane, check the Management IP per CCcheckbox.

Figure 4-12: IP Properties tab

Adding a Data Mediator

Open NetXplorer.1.

In the Navigation pane, right-click Servers in the Network pane in2.

the Navigation tree and select New Data Mediation… from the popup menu.

The Data Mediation Properties - New dialog is displayed.




Figure 4-13: Data Mediation Properties




Enter the name and IP address of the Data Mediator.3.

In the Source Units area, use the arrow keys to move In-line4.

Platforms and SMPs from the Available to the Selected lists.

Those In-line Platforms and SMPs selected will provide data tothe Data Mediator.

In the Record Types to Collect area, select those record types that5.

this Data Mediator will collect. Multiple data types may beselected, depending upon the capabilities of the platforms selected

as Source Devices as well as the current license key.

The possible data types are VC, Conversations, HTTP Detailed

Records (HDR), Minutes of Use (MOU), Binary Detail Records(BDR), Usage Detail Records (UDR) and Session Detail Records

(SDR).

NOTE SDR collection is only possible if you have included an SMP inthe Selected Sources.

Click Save to add the Data Mediator to the network.6.

Click Save. The Monitoring Collector is added to the Navigation7.

tree. The New Collector operation can take up to a couple ofminutes to complete.

Enabling and Configuring CDR Types

The way that the Data Mediator works with each of the seven types of CDRs (VCs,Conversations, HTTP Detailed Records (HDR), Minutes of Use (MOU), Binary Detail

Records (BDR), Usage Detail Records (UDR) and Session Detail Records (SDR)) must be configured from the Network Configuration dialog within the NetXplorer GUI.

In addition, HDRs, MOU and Conversation records must be enabled on each relevantIn-line Platform individually. SDRs must be enabled on each relevant SMP, which willalso enable UDRs.

For details, see the subsequent chapters outlining the requirements for each CDR type.

NOTE In some circumstances you may wish to customize the format ofthe data exported. Allot provides direct access to theconfiguration file to enable you to do this if required. For moreinformation, see Appendix C on page 13-3.




Chapter 5: HTTP Detail Records (HDR)

HTTP CDRs are exported in CSV file format. Every HTTP complete transaction isrepresented by a record in the CSV file.

The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external systemfrequently (every 1-2mins). These settings can be changed from the Data Mediation tabin the NetXplorer GUI.

The Allot system can be configured to purge the files after a pre-defined period of time(default is 24 hours).

Enabling and Configuring HDRs

Enabling HDRs per In-line Platform

1. In the Navigation pane, select and right-click the relevant In-linePlatform in the Navigation tree and select Configuration from the

popup menu.

OR

Select the In-line Platform in the Navigation tree and then selectConfiguration from the View menu.

OR

Select the In-line Platform in the Navigation tree and then click the


The Configuration window for the selected In-line Platform is displayed.

2. Select the Service Activation tab.

3. To enable HDRs, select Enable from the HDR Generation

Enablement drop down field.




Figure 5-1: HDR CDR area in the Service Activation tab

4. In the HDR CDR, the following other fields may be configured.

HDR Generation Rate Limit – This defines the maximum

number of entries generated per second. The maximum possiblevalue depends upon which In-line Platform or Core Controller

blade is being used, as follows:

CC-300 Blade – 10000

CC-200 Blade – 2500

AC-6000 – 7500

AC-1400/AC-3000 - 1000

HDR File Compression – This sets the file compression methodto be used when storing the file on the In-Line Platform. The

possible options are GZIP and no compression. GZIP is selected by default.

5. Save the configuration changes to apply the new settings.

NOTES The following environments do not support HDRs- Asymmetric Deployment- Captive Portal set to “Use Request”

A Websafe blacklist external server will not create HDRs

In Active Redundancy, the record will be generated on theNetEnforcer receiving the HTTP request

HTTP requests created by a steering service will not be includedin HDR




HTTPS traffic is not included in HDRs

Configuring HDRs on the Data Mediator

1. In the Navigation pane, select and right-click the Network in the Navigation tree and select Configuration from the popup menu.

OR

Select the Network in the Navigation tree and then select Configuration fromthe View menu.

OR

Select the Network in the Navigation tree and then click the Configuration

icon on the toolbar.

The Configuration window for the Network is displayed.

2. Select the Data Mediation tab.

3. To configure HDRs, open the Data Definition tab and select theHDR sub-tab.

Figure 5-2: HDR Data Definition Properties

4. In the HDR sub-tab, the following fields may be configured.

File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file before compression. The default is

100,000 bytes (100 MB) and the maximum possible value is1,000,000,000 bytes (1 GB).




File Closing Interval (in seconds) – This defines how long after

being created the export file is closed. The default is 300

seconds (5 minutes) and the maximum is 3600 seconds (1 hour).

File Compression method – This sets the file compressionmethod to be used when exporting the file. The possible optionsare GZIP and no compression. GZIP is selected by default.

Delete files older than (in minutes) – This defines the amount oftime the system retains files. The default is 1440 minutes (24

hours) and the maximum is 4320 minutes (72 hours).

Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis andstorage. The four push method options are SFTP, FTP, SCP and

Copy. If SFTP is selected, the Username, Password, Server and

Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the pathmust be defined. The default is Enabled.

Filtering Rules – See Below.

Filtering Rules

Using the Filtering Rules area, you may elect to Include All Data in your HDRs or toExclude All Data from your HDRs by selecting the appropriate radio button. You may

then add exceptions to the list below by clicking on the Add button.

Figure 5-3: Add Rule dialog

When the Add Rule dialog appears, you may create an exclusion rule by entering aname and clicking Add to add attributes to the rule.




Figure 5-4: Add Attribute dialog

Clicking the Add button on the Add Rule dialog allows you to select to add aContentType, Path, Method or Domain and then enter the appropriate value. Click OK

to save your selections.

KEY DESCRIPTION EXAMPLES

ContentType The MIME type ofthis content(MultipurposeInternet Mail

Extensions)

text/html

image/gif

image/jpeg

Path The URI /index.path

/news

Method The desired actionto be performed on

the resource.

GET, CONNECT, POST

Domain The domain name ofa server.

www.cnn.com

www.ynetnews.com

Naming ConventionsHTTP CDRs files are generated according to the following naming conventions:

<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Datatype>_<Version>.[<FE>]

FIELD NAME DESCRIPTION NOTES

< Source unit name > The name of the source unit thatgenerated the file

Each name must beunique

<Source file IP> The IP Address of the Core Controller.

http://www.ynetnews.com/







<Mediation IP> The IP address of the Mediation unitthat created the file

<Seq> A sequencer that is never reset.

<Date> Creation date of the file in the formyyyymmddhhmmss

<Data type> Represent the file content:

“HDR” – for files containing HTTPCDR records

<Version> Version of the entry format.

Format: V<format number>

The version is set from theDataExportConfig.xml file where therequired output structure is defined.

<FE> File extension.

The file extension is csv.gz for HTTPCDR files.

While the file is being copied a “.tmp”

suffix will be added. This suffix will be removed after the copy has been

completed successfully.

HDR Interface Fields

ATTRIBUTENAME

DESCRIPTION FORMAT

Enforcer Name In-line Platform name which generated the

CDR

String (in quotes)

StartTime Date and Time that the HTTP request

detected in the device.

By default – represented as yyyy-MM-

ddThh:mm:ss+hh:mm

This field can also be set to be represented

as a simple numeric string containing UTC

time in seconds since 1.1.1970

Date String




ATTRIBUTENAME

DESCRIPTION FORMAT

SubscriberID Unique subscriber identifier.

Zero if deployment is without SMP.

Value is external identifier (in mobile

networks) or internal identifier (in Fixed

networks).

String (Numeric String in

mobile environments)

SessionKey Unique identifier of the subscriber session.

The same SessionKey is used for the

Subscriber session in HDRs, SDRs and

UDRs.

Present in subscriber management

environment only. If no SMP is in the

Network, this value appears as 0.

Numeric String

ClientIP IP (IPv4 or IPv6) of the HTTP request

originator

Dotted Numeric String

ClientPort TCP port of the request originator Numeric String

ServerIP IP (IPv4 or IPv6) of the designated HTTP

server.


ServerPort TCP port of designated HTTP server Numeric String

Service name The Service name mapped from the DPI

application identifier

String (in quotes)

HTTPMethod HTTP method type of the request (GET,HEAD, POST, etc.)

String (in quotes)

Host Domain of the HTTP request, taken from

the Host header

String (in quotes)

URI URI field of the request String (in quotes)

UserAgent UserAgent header from the HTTP request String (in quotes)

Referer Referer header from the HTTP Request String (in quotes)

DownloadContentL

ength

The content length field from the server

response as set by the server application.

Limited to 4GB.

Numeric String




ATTRIBUTENAME

DESCRIPTION FORMAT

UploadContentLen

gth

The content length field from the request

as set by the client application. Limited to4GB.

Numeric String

ResponseCode The response code from the server

response

Numeric String

ServerInitialRespo

nseTime

Time interval in milliseconds from

identifying the request until receiving the

server response.

Numeric String

Duration Transaction duration in milliseconds (until

last packet of the response is received)

Numeric String

RequestActualByte

Count

Actual byte count of outgoing traffic as

counted by the DPI engine.

Numeric String

ResponseActualByt

eCount

Actual byte count of incoming traffic as


Numeric String

DNT DNT or x-do-not-track header from the

HTTP request

String (in quotes)

Output Example

Figure 5-5: HDR .csv file example




Chapter 6: Minutes of Use (MOU)

MOU CDRs present data for the minutes of use for different over the top VoIP

applications running through the network. MOU CDRs are exported in CSV file format.Every complete transaction is represented by a record in the CSV file.

The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external systemfrequently (every 1-2mins). These settings can be changed from the Data Mediation tab

in the NetXplorer GUI.


Enabling and Configuring MOU

Enabling MOU per In-Line Platform

1. In the Navigation pane, select and right-click the relevant In-linePlatform in the Navigation tree and select Configuration from the

popup menu.

OR

Select the In-line Platform in the Navigation tree and then selectConfiguration from the View menu.

OR

Select the In-line Platform in the Navigation tree and then click the


The Configuration window for the selected In-line Platform is displayed.

2. Select the Service Activation tab.

3. To enable MOU, select Enable from the VOIP Data Collection drop down field in the VOIP Reports area.




Figure 6-1: VOIP Reports in the Service Activation tab

4. Save the configuration changes to apply the new settings.

Configuring MOU on the Data Mediator


OR


OR





3. To configure MOUs, open the Data Definition tab and select theMOU sub-tab.




Figure 6-2: MOU Data Definition Properties

4. In the MOU sub-tab, the following fields may be configured.

File Size Limit (in bytes) – This defines the maximum possible

size of each exported file. The default is 100,000 bytes (100MB) and the maximum possible value is 1,000,000,000 bytes (1GB).




File Compression method – This sets the file compression

method to be used when exporting the file. The possible optionsare GZIP and no compression. GZIP is selected by default.

Delete files older than (in minutes) – This defines the amount of

time the system retains files. The default is 1440 minutes (24hours) and the maximum is 4320 minutes (72 hours).

Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis and

storage. The four push method options are SFTP, FTP, SCP andCopy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and

password must be defined. For FTP and Copy, only the pathmust be defined.




Naming ConventionsMOU files are generated according to the following naming conventions:





<Source file IP> The IP Address of the file origin.





“MOU” – for files containing Minutesof Use records



The version is set from theconfiguration file where the requiredoutput structure is defined.


The file extension is csv.gz for MOUfiles.

While the file is being copied a “.tmp”suffix will be added. This suffix will

be removed after the copy has beencompleted successfully.

MOU Interface Fields

ATTRIBUTENAME

DESCRIPTION FORMAT




ATTRIBUTENAME

DESCRIPTION FORMAT

Start Time Start of the time interval measured by this record.

By default this is represented as yyyy-MM-

ddThh:mm:ss+hh:mm

Can also be set to be represented as a simple

numeric string containing UTC time in seconds

since 1.1.1970

Date String

End Time End of the time interval measured by this record

By default this is represented as yyyy-MM-

ddThh:mm:ss+hh:mm

Can also be set to be represented as a simple

numeric string containing UTC time in secondssince 1.1.1970

Date String

Service Plan Name Service plan name String (in quotes)

Service Name The name of the application mapped from the DPI String (in quotes)

Total seconds VoIP usage in seconds Numeric (D)

Output Example

Figure 6-3: MOU .csv file example




Chapter 7: VC Records

NOTE VC Records are automatically enabled for all In-line Platforms.

This information concerns traffic that is classified into different rules in the Allotsystem. The data is based on the service plan and the subscriber. Depending on theimplementation, VC statistics may provide policy enforced application or application

group volumes based on the service plan definitions for traffic that needs to be shaped, blocked, steered or TOS-marked. They are extracted from the in-line traffic and arecompiled on the in-line platform (e.g: SG-Sigma)

VC CDRs are exported in CSV file format. Every complete transaction is represented by a record in the CSV file.

The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external system

frequently (every 1-2mins). These settings can be changed from the Data Mediation tabin the NetXplorer GUI.


Configuring VCs on the Data Mediator1. In the Navigation pane, select and right-click the Network in the

Navigation tree and select Configuration from the popup menu.

OR


OR





3. To configure VCs, open the Data Definition tab and select the VCsub-tab.




Figure 7-1: VC Data Definition Properties

4. In the VC sub-tab, the following fields may be configured.










Push Properties – Push properties, once enabled, define how the

export files are sent to their proper locations for analysis and

storage. The four push method options are SFTP, FTP, SCP andCopy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and





ATTRIBUTE NAME DESCRIPTION FORMAT

Enforcer Name Device name which generated the

CDR

String (in quotes)

Start Time Start of the time interval measured bythis record.

By default this is represented as yyyy-

MM-ddThh:mm:ss+hh:mm

Can also be set to be represented as asimple numeric string containing UTCtime in seconds since 1.1.1970

Date String

End Time End of the time interval measured by

this record.

By default this is represented as yyyy-MM-ddThh:mm:ss+hh:mm

Can also be set to be represented as a

simple numeric string containing UTCtime in seconds since 1.1.1970

Date String

Line Name representing the Line String (in quotes)

Pipe Name representing the Pipe String (in quotes)

VC Name representing the VirtualChannel

String (in quotes)

Instance Type See Note 1 Numeric String

Instance Key 1 See Note 1 Numeric String


Net Unit ID Cell ID Numeric String

Live connections Number of live Connection in thetime interval

Numeric String

Dropped connections Number of dropped Connection in thetime interval

Numeric (D)





3. To configure Conversations CDRs, open the Data Definition taband select the Conversations sub-tab.

Figure 8-1: Conversations Data Definition Properties

4. In the Conversations sub-tab, the following fields may beconfigured.

File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file. The default is 5 minutes and you may

opt for bytes (100 MB) and the maximum possible value is1,000,000,000 bytes (1 GB).

Aggregation Period (min) – This defines how long after beingcreated the export file is closed. The default is 5 minutes and

you may opt for 10, 15, 30 or 60 minutes from the drop downlist. This value will affect ALL record types, not justConversations.




time the system retains files. The default is 1440 minutes (24


Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis and




storage. The four push method options are SFTP, FTP, SCP and

Copy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and

password must be defined. For FTP and Copy, only the path

must be defined.

Prioritizing Conversations Statistics

It is possible for the operator to prioritize the Conversation records by application asfollows:

Golden: Conversation CDRs are exported in full detail due to highinterest.

Excluded: Conversation CDRs are exported in an abridged format

due to lack of interest (only number of connections and throughput

is saved, with no rule ID and client IP).

Normal: Conversation CDRs are exported in summary detail. This

priority level is set to applications by default.

All Conversation records are collected in detail and are summarized to reduce thestorage space required. This feature allows the operator determine which applicationsshould be kept and exported in full Conversation detail, which applications can besummarized and which are exported in a highly abridged format. Sufficient storagespace is then set based on these priorities.

The operator may further define whether the Conversation data should be exported in 5-minute or 1-hour aggregations.

All Conversation records are collected in detail and are summarized to reduce thestorage space required. This feature allows the operator determine which applications

should be kept and exported in full Conversation detail, which applications can besummarized and which need not be exported at all. Sufficient storage space is then set

based on these priorities.

Assigning Priority

Priority levels need to be configured via the CLI using the following commands:

To set the priority level:

go add app_collection [app_name] –priority golden/excluded

app_name – service catalogue name entry (excluding service groups)

To show the priority list by application:

go list app_collection

Output example




sysadmin@AC3K-5:~$ go list app_collectionGolden services:

ViberSkypeTango

LineMGCPYahoo ChatJabberGoogle PlayiTunesFacebook ChatBaidu HiQQ ChatTwitterFacebookYandexVKontakteGmailCall Of Duty

GoogleTalkWhatsAppMailRuYahoo MailDiabloIIILineage2Kakao

Excluded services:BGPDNSEGPICMPIGMPOSPF

RIPARPPPPoE-DiscoveryPPPoE-ControlDHCP-ClientDHCP-ServerNTPIS-ISLACP

Request completed successfully.sysadmin@AC3K-5:~$

To change/remove the configuration for a specific application:

go update app_collection [app name] –priority golden/excludedgo delete app_collection [app name] –priority golden/excluded

Naming Conventions

Conversations records are generated according to the following naming conventions:




ATTRIBUTENAME

DESCRIPTION FORMAT

Start Time

Start of the time interval measured bythis record.

By default this is represented as yyyy-MM-ddThh:mm:ss+hh:mm

Can also be set to be represented as asimple numeric string containing UTC

time in seconds since 1.1.1970

Date String

End Time

End of the time interval measured bythis record.

By default this is represented as yyyy-

MM-ddThh:mm:ss+hh:mm

Can also be set to be represented as a

simple numeric string containing UTCtime in seconds since 1.1.1970

Date String

Internal IP Internal IP address; both IPv4 andIPv6 are supported


External IP External IP address; both IPv4 andIPv6 are supported


Service Name The name of the application mapped

from the DPI

String (in quotes)

Line Name representing the Line Numeric String

Pipe Name representing the Pipe Numeric String

VC Name representing the Virtual Channel Numeric String

Instance Type See Note 1 Numeric String



Net Unit ID Cell ID Numeric String




ATTRIBUTENAME

DESCRIPTION FORMAT

Live connections Number of live Connection in the timeinterval Numeric String

Dropped connections Number of dropped Connection in thetime interval Numeric (D)

New connections Number of new Connection in the

time interval Numeric (D)

Octets in Byte count of incoming traffic Numeric (D)

Octets out Byte count of outgoing traffic Numeric (D)

Note 1:

INSTANCE TYPE INSTANCE KEY 1 INSTANCE KEY 2

0 Empty, no instance Empty

1 IP address (IPv4 or IPv6) asPipe instance

Prefix for IPv6

2 IP Address (IPv4 or IPv6) asVC instance

Prefix for IPv6

3 Host group name as Pipeinstance

Empty

4 Host group name as VCinstance

Empty

5 Subscriber Identifier as Pipeinstance

Session Key

6 Subscriber Identifier as VCinstance

Session Key




Output Example

Figure 8-2: CONV .csv file example




Chapter 9: Session Detail Records (SDR)

SDRs are extracted from the AAA function (e.g: RADIUS) and are compiled on theSMP Server. An SDR is created for every segment of a session (called a sub-session) on

a periodic basis (20m by default. To change the interval, contact [email protected]). Asession is broken into a sub-session whenever there is a change in the session data (e.g:switch of service plan) or by time interval.

SDRs are exported in CSV file format. Every complete transaction is represented by arecord in the CSV file.

The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external system

frequently (every 1-2mins). These settings can be changed from the Data Mediation tabin the NetXplorer GUI.


NOTE SDRs are only available from an SMP, not from an In-linePlatform.

Enabling SDRs per SMP

Session Detail Records (SDRs) must be generated on the SMP server to extract therelevant information from the RADIUS message. This is enabled via the NetXplorerGUI.

1. In the Navigation pane of the NetXplorer User interface, right-

click the Network and select Configuration from the popup menu

OR

Select Network in the navigation pane, and then chooseConfiguration from the Actions menu.

The network tabs will be displayed in the Applications pane.

2. Select the SMP tab. In the SMP preferences section, 4 additionaltabs will appear.

3. Select “Session Parameters”. From the Session Parameters tab,determine the extra RADIUS parameters which you wish to beincluded in the SDRs. Check whichever fields you wish toinclude, as shown in Figure 9-1 below.




Figure 9-1: Adding Additional RADIUS Parameters to the SDR

NOTES You can also set a default value for some fields. This defaultvalue will be included in each SDR for the particular fieldspecified, should no actual value be received.

The AVPs which you select in this dialog to be included in theSDR must have first been configured to be extracted from theRADIUS message in the radius_params.conf file. The procedurefor extracting these AVPs is detailed in Error! Reference source not

ound.

4. Select the “Policy and Charging” tab. Set the SDR CollectionStatus field to “On”

NOTE Unless SDR Collection Status is defined as On in the Policy andCharging Tab, no SDRs will be generated even if SDR Export isenabled in other areas of the GUI.




Figure 9-2: Policy and Charging Tab

5. You will be prompted to reboot the SMP server, whereupon SDRcollection will commence.

Configuring SDRs on the Data Mediator1. In the Navigation pane, select and right-click the Network in the


OR


OR





3. To configure Conversations CDRs, open the Data Definition taband select the Conversations sub-tab.




Figure 9-3: SDR Data Definition Properties

4. In the SDR sub-tab, the following fields may be configured.






File Compression method – This sets the file compressionmethod to be used when exporting the file. The possible options

are GZIP and no compression. GZIP is selected by default.


time the system retains files. The default is 1440 minutes (24hours) and the maximum is 4320 minutes (72 hours).


export files are sent to their proper locations for analysis and

storage. The three push method options are SFTP, FTP andCopy. If SFTP is selected, the Username, Password, Server and

Path must be defined. For FTP and Copy, only the path must bedefined.

Naming ConventionsSDRs files are generated according to the following naming conventions:






< Source unit name > The name of the source unit that

generated the file

Each name must be unique


<Mediation IP> The IP address of the Mediation unit that

created the file


<Date> Creation date of the file in the form

yyyymmddhhmmss


“SDR” – for files containing HTTP CDR

records



The version is set from the configuration

file where the required output structure is

defined.


The file extension is csv.gz for VC files.

While the file is being copied a “.tmp”

suffix will be added. This suffix will be

removed after the copy has been


SDR Interface Fields


Enforcer Name Device name which generated the CDR string (in quotes)

Record creation

timestamp

SDR generation timestamp Date String





Subscriber ID Unique subscriber identifier

Zero if deployment is without SMP.

Value is external identifier (in mobile networks)

or internal identifier (in Fixed networks).

Numeric String

Session Key Unique identifier of the subscriber session. The

same SessionKey is used for the Subscriber

session in HDRs, SDRs and UDRs.

Present in subscriber management environment

only. If no SMP is in the Network, this value

appears as 0.

Numeric String

Sub-Session Key Unique sub-session identifier per subscriber

session. Sub session is created if there is a

change in the service plan of the subscriber

during the session.

Present in subscriber management environment

only

Numeric String

Sub Session Start Time Start time of the sub session Date String

Sub Session End Time End time of the sub session Date String

Service Plan Service plan name String (in quotes)

Open reason Reason for opening the session. One of the

following values:

START

SERVICE_PLAN_CHANGE

INFO_CHANGE

SCHEDULE

String (in quotes)

Close reason Reason for closing the session. One of the

following values:

SERVICE_PLAN_CHANGE

INFO_CHANGESCHEDULE

CLOSE

String (in quotes)

IMEI This field contains the International Mobile

Equipment Identity (IMEI) of the equipment

served, if available (3GPP-IMEISV)

Numeric String





SGSN Initial SGSN addresses used received on session

start (3GPP-SGSN-Address)

Numeric String

IMEI-TAC The TAC part of the IMEI for device type

identification (3GPP-IMEISV)

Numeric String

RAT Radio Access Type used to start the session

(3GPP-RAT-Type)

Numeric String

Source IPv4 Source address of the session in IPv4 format Dotted Numeric

String

Source IPv6 Source address of the session in IPv6 format Dotted Numeric

String

APN Access Point through which the session started

(Called-Station-Id)

String (in quotes)

Location Initial location where the session started (3GPP-

User-Location-Info)

String (in quotes)

MCC MNC Mobile Country Code and Mobile Network

Code (MCC-MNC) of the subscriber's home

operator (3GPP-IMSI-MCC-MNC)

String (in quotes)

CustomAttr1 Attributes that can extracted on-demand by the

end-user

String (in quotes)


end-user

String (in quotes)


end-user

String (in quotes)

Custom Attributes

To define the three custom attributes (CustomAttr1, CustomAttr2 and CustomAttr3),the fields must be added to the radius-dictionary file on each SMP and then added to the

DataExportConfig.xml file.

These three custom fields will then be extracted from the radius message and saved inthe SDR.

For example:




Service Plan ID

Every service plan configured in the system has an internal ID number. To determinethe internal ID number of each service plan, run the pccAdmin.sh – p command on the

SMP. One of the logs created is the ServicePlanCatalog.log. This log will list the ID ofeach Service Plan created, mapping between ID and Service plan name as shown in thesample output below:

less ServicePlanCatalog.log

===== Service Plan Catalog =====

Service Plan Catalog Size= 26

Service Plan Id= 2 ,Service Plan Name= Block Service Plan ,Gx SessionExclude Group Id= 1025

Service Plan Id= 7 ,Service Plan Name= gold ,Gx Session Exclude GroupId= 1030

Service Plan Id= 1 ,Service Plan Name= Unlimited Service Plan ,GxSession Exclude Group Id= 1024

Service Plan Id= 9 ,Service Plan Name= Unlimited ServicePlan_1318349232984 ,Gx Session Exclude Group Id= 0

Service Plan Id= 19 ,Service Plan Name= Block Service Plan_1318349038792,Gx Session Exclude Group Id= 0

Service Plan Id= 6 ,Service Plan Name= vc_sp3 ,Gx Session Exclude GroupId= 1029

Open Reason / Close Reason

The open reason and close reason detail why a new sub-session (represented by a newline within the SDR) has been opened. A value between 1 and 6 is used to detail the

reason, in accordance with the table below:

VALUE DEFINITION

1 START Session radius start message received. The first sub-session recordof a session will always have an open reason of Start.

2 START_REPORT You will only see this value if Allot Customer Support have set the“Session Report on Start” status configuration to "enable". In thiscase, a record with open reason Start is created. The close reasonfor such records is Start_Report. Start_Report will then also be

used as the open reason of the following sub-session.

3 SP_CHANGE A new SP is assigned to the session.

4 INFO_CHANGE Any session information change received from Radius. Used as bothclose and open reason.

5 SCHEDULE You will only see this value if Allot Customer Support have




VALUE DEFINITION

changed the value of the “Sub-session lifetime” field. By default theconfiguration is 24hr. When the duration of a sub-session reaches

the configured time, the sub-session is closed by the SMP with aSchedule record type and a new sub-session is opened. This is usedas both close and open reason.

6 CLOSE Session radius stop message received.

IMEISV Number

The IMEISV number is extracted from the RADIUS message. IMEISV contains 15digits and has the following format: AA-BBBBBB-CCCCCC-D

AA: These two digits are for the Reporting Body Identifier, indicating the GSMA

approved group that allocated the TAC (Type Allocation Code).

BBBBBB: The remainder of the TAC

CCCCCC: Serial sequence of the Model D: Luhn check digit of the entire model or 0 (This is an algorithm that validates

the ID number)

Output Example

Figure 9-4: SDR .csv file example




Chapter 10: Usage Detail Records (UDR)

NOTE UDRs are automatically enabled if SDRs are enabled. If SDRs are

not being generated by one or more SMPs, then UDRs cannot begenerated.

UDRs collect usage information per session as well as usage information perapplication within a session.

UDR information is extracted from the in-line traffic and is compiled on the in-line platform (e.g: SG-Sigma). UDRs are created on a periodical basis (1hr by default. Tochange the interval, contact [email protected]) and count the sub-session uplink and

downlink volumes of traffic. When not interrupted by session changes, these records arealigned by the hour (e.g: 12.00, 13.00, 14.00 etc.)

UDRs are exported in CSV file format. Every complete transaction is represented by a

record in the CSV file.

The CSV files are pushed compressed in GZIP format from the Data Mediator to theexternal system frequently (every 1-2mins).


NOTE A UDR is generated for each monitored application configured, inaddition to a UDR with total usage for that session.

Allot’s in-line platforms use proprietary “monitoring rules” tocount the number of bytes used per subscriber for eachconfigured application and for the total bytes used by the

subscriber’s session if configured to do so. The in -line platformwill use a single monitoring rule per each application (and for thesession counted bytes if configured to do so) defined for UDRcollection.

Allot supports a total of 13 monitoring rules per subscriber. EveryUDR monitoring rule is coupled by an SDR monitoring rule.Therefore the total number of applications which can bemonitored per subscriber is 6.

Monitoring Rules are also consumed when implementing Gxbased volume reporting, an online charging policy, an offlinecharging policy and service flow detection (event based triggers.

Configuring UDRs on the Data Mediator1. In the Navigation pane, select and right-click the Network in the


OR





OR





3. To configure UDRs, open the Data Definition tab and select theUDR sub-tab.

Figure 10-1: UDR Data Definition Properties

4. In the UDR sub-tab, the following fields may be configured.

File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file. The default is 100,000 bytes (100MB) and the maximum possible value is 1,000,000,000 bytes (1GB).


being created the export file is closed. The default is 300seconds (5 minutes) and the maximum is 3600 seconds (1 hour).








Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis andstorage. The four push method options are SFTP, FTP, SCP and

Copy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and


Monitored applications – See below.

Monitored Applications

The Monitored Applications list is where applications-based UDRs are configured. Theapplications to be included in the UDR must be selected. In order for an application to

be selected, it must have already been added to the Charging Applications catalog.

You may add Monitored Applications to the list by clicking on the Edit button.

Figure 10-2: Change Monitored Applications dialog

When the Change Monitored Applications dialog appears, use the Arrow buttons tomove applications from the Available Applications list to the Selected Applications list.Only those applications that have previously been added to the Charging Applicationscatalog will appear in the Available Applications list.

Click OK to save and return to the UDR tab. Those applications you have selected nowappear in the Monitored Applications list.

Modification of the Selected Applications list while the system is running will affect

only new sessions. Statistics for sessions that started prior to the modification willcontinue reporting according to the previous Selected Applications list.

NOTE A UDR is generated for each monitored application configured, inaddition to a UDR with total usage for that session. The Sessionapplication is created automatically and cannot be removed.

For more information on Charging Applications, see the NetXplorer Operations Guide.




Naming ConventionsUDR files are generated according to the following naming conventions:










“UDR” – for files containing UDRrecords



The version is set from theconfiguration file where the requiredoutput structure is defined.


The file extension is csv.gz for UDRfiles.


be removed after the copy has beencompleted successfully.




UDR Interface Fields


Subscriber ID Unique subscriber identifier in

subscriber management

environment as received from

SMP.

Zero if deployment is without

SMP.

Value is external identifier (in

mobile networks) or internal

identifier (in Fixed networks).

Numeric String

Session Key Unique identifier of the

subscriber session. The sameSessionKey is used for the

Subscriber session in HDRs,

SDRs and UDRs.

Present in subscriber

management environment only.

If no SMP is in the Network, this

value appears as 0.

Numeric String

Sub-Session Key Unique sub-session identifier per

subscriber session. Sub session is

created if there is a change in the

service plan of the subscriberduring the session.

Present in subscriber

management environment only

Numeric String

Monitored Application Name The name of the monitored

application as defined by the

end-user in the application

groups catalogs

String (in quotes)

Start Time Start of the time interval for

which the statistics measures are

given (e.g. Octets in)

Date String

End Time End of the time interval for

which the statistics measures are

given (e.g. Octets in)

Date String

Octets in Byte count of incoming traffic


Numeric String





Octets outByte count of outgoing traffic

counted by the DPI engine. Numeric (D)

Output Example

Figure 10-3: UDR .csv file example




Chapter 11: Binary Data Records (BDR)

Enabling and Configuring BDRs on the In-LinePlatform

To enable BDR identification and data records generation, use the following procedure:

1. Connect to the in-line platform, as sysadmin

2. Create a directory somewhere called WAConfig

3. Copy the configuration files into that directory. i.e.cp /opt/allot/conf/WebAnalytics/* ./WAConfig

4. Edit your LOCAL ./WAConfig/server.cfg

5. Change the relevant parameters as in the following example:

#

# Pilot Packet configuration file

#

####################################################

# Connection Tuple

# protocol list: 0-TCP / 1-UDP / 2-ANY

# port: 0-65535 (0 for ANY)

# IP: a valid ipv6 address (::ffff:0.0.0.0 for ANY)

####################################################

intIp="::ffff:127.0.0.1"

intPort=0

extIp="::ffff:255.255.255.255"

extPort=8080

protocol=1

####################################################

# DPI Parameters

####################################################

# establish the connection: 0-Parsed / 1-Established

shouldEstablish=0

# DPI connection timeout optimization ( 0 - disabled / 1 - enabled )

special_timeout=1

# predefined servers mode ( 0 - disabled / 1 - enabled )




predefined_servers_enable=1

###################################################

# special tuple parameters

###################################################

behaviour=0

###################################################

# WebAnalytics

###################################################

hdr_enable=1

gcdr_enable=1

Legend gcdr_enable – Enable(1) or disable(0) BDRs.

IMPORTANT The default configuration is for BDR to be disabled. Forcustomers using the BDR feature, this flag HAS to be changed to1.




operation_mode – 1 is enabled, 0 is disabled. This disables theentire list of servers.

extIp / intIp – One of these fields is mandatory. The other isoptional. Therefore, there should be at least one IP address

provided here.

intPort , extPort – Optional.

protocol – Indicates whether it's TCP (0) or UDP (1) or ANY (2).

For BDRs you should use UDP, 1 , as in the example above.

shouldEstablish – Must be 0 for BDRs.

appId – Must be 407 , which is the AppID for "Other IM

Applications" which was altered on the previous instructions.

hdr_enable – Not relevant, this feature is controlled via the

NetXplorer GUI.

special_timeout – 1 = Enabled, 0 = disabled. This enables the useof HDR duration and a real byte count for timeout purposes. After

editing this parameter, restart the DataSrv (using keeperMgr -SW -

R DataSrv) or reboot the host blade.

IMPORTANT Use 0 if HDR duration or a real byte count fields are not required.

6. After changing the configuration, please apply using the followingcommand:

/opt/allot/bin/wa-update.sh <WACONFIG DIRECTORY>

Output Example

# /opt/allot/bin/wa-update.sh /home/sysadmin/WAConfig

Creating a tar file...

genericcdr.xml

webanalytics.xml

webanalytics.xsd

Updating CCs...

Done!

7. This will copy the files in the directory into/opt/allot/conf/WebAnalytics and apply the configuration.

Configuring BDR Export on the NetXplorer1. To enable BDR configuration from the NetXplorer GUI, contact

Allot Customer Support.




2. Once BDRs have been enabled in the NetXplorer GUI, select andright-click the Network in the Navigation tree and selectConfiguration from the popup menu.

ORSelect the Network in the Navigation tree and then select Configuration fromthe View menu.

OR





4. To configure BDRs, open the Data Definition tab and select theBDR sub-tab.

Figure 11-1: BDR Data Definition Properties

5. In the BDR sub-tab, the following fields may be configured.

File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file before compression. The default is

100,000 bytes (100 MB) and the maximum possible value is1,000,000,000 bytes (1 GB).

File Closing Interval (in seconds) – This defines how long after being created the export file is closed. The default is 300







Delete files older than (in minutes) – This defines the amount oftime the system retains files. The default is 1440 minutes (24hours) and the maximum is 4320 minutes (72 hours).


export files are sent to their proper locations for analysis andstorage. The four push method options are SFTP, FTP, SCP andCopy. If SFTP is selected, the Username, Password, Server and

Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the pathmust be defined. The default is Enabled.

Naming ConventionsBDR files are generated according to the following naming conventions:










“B40” – for files containing BDRrecords



The current version is V1






The file extension is bin.gz for BDRfiles.


be removed after the copy has been





Chapter 12: SNMP Interface

Data Mediator TrapsThe Data Mediator sends 5 different types of traps to the Service Provider’s NOC.

NAME DESCRIPTION OID

alDmCopyToFailure This trap is sent whenever a file push

failed. The alarm is raised provided that

a configured interval has passed since

the last time a copytofailure alarm was

raised towards the same target

1.3.6.1.4.1.2603.12.4.1

alDmRecordRateUp This trap is sent if the rate of CDRs

processed per second passes the

configured threshold for each interval.

1.3.6.1.4.1.2603.12.4.2

alDmRecordRateDown This trap is sent if the rate of CDRs processed per second returns to below

the configured threshold for each

interval.

1.3.6.1.4.1.2603.12.4.3

alDmCopytoFailureDown This trap is sent if there was a failure to

push a file to the target host.

The trap will include the failed target

host.

1.3.6.1.4.1.2603.12.4.4

alDNGetFileFailureUp This trap is sent if the “get file”

operation from the in-line platform

failed. The trap will include the relevant

in-line platform and the relevant

filename.

1.3.6.1.4.1.2603.12.4.5

Data Mediator KPIsThe Data Mediator can be polled by an external SNMP server for the purpose of healthmonitoring.

Using the standard HOST-RESOURCES MIB, you can poll each appliance for healthmonitoring and key performance indicators.

The standard LINUX Host-Resources MIB has the OID of 1.3.6.1.2.1.25. It can be polled for information about:




Output RecordsNAME DESCRIPTION OID

alDmKpiOutputFilesCreatedNum The number of files

created by the datamediator since the last

boot

1.3.6.1.4.1.2603.12.3.1.1.1.6

alDmKpiOutputFilesPushSuccess

Num

The number of filessuccessfully pushed by

the data mediator to theoperator’s SFTP serversince the last boot

1.3.6.1.4.1.2603.12.3.1.1.1.7

alDmKpiOutputFilesPushFailNum The attempts to pushfiles by the data

mediator to the

operator’s SFTP serversince the last boot whichfalied (note: this is notthe number of files

which have not been pushed)

1.3.6.1.4.1.2603.12.3.1.1.1.8

alDmKpiOutputFilesPendingPush Files to be pushed – thetotal number of files

waiting to be pushed.

1.3.6.1.4.1.2603.12.3.1.1.1.10




Chapter 13: Appendices

Appendix A: Data Mediator Logs

Datamediation.log – a log providing all system messages at twoconfigurable levels, info and debug (debug mode will printadditional information in the log).

App-datamediation.log – a user oriented log, prints messages at 3levels:

failure (logging all failures to push to target or pull from

source)

push (logging those files successfully pushed)

all (logging all files successfully pushed, pulled or createdon the Data Mediator)

An entry in the App-datamediation.log contains the followinginformation:

Name or address of the source unit

Name or address of the target unit

File name

Transfer start date and time

Transfer end date and time

File size in Mbytes

Number of records in file - for stored or pushed files

Status – success or failure

Reason – description of cause of failure

Appendix B: Upgrade Procedure

Use the following procedures to upgrade your Data Mediator devices:




6. On both nodes make sure that there are no DB processes running by using the following command:

ps ax |grep -v grep |grep dbsrv -c

The output should be 0.

7. On both nodes make sure no storage is mounted by entering thefollowing command:

mount |grep dm-1 –c

The output should be 0.

8. On DM-0 run the following command:

mount /dev/dm-1 /opt/sybase/data


rpm -Uvh <filename>.rpm

Where < filename> is the name of the new MD installation package stored in a temp folder on the Data Mediator/


rpm -Uvh <filename>.rpm

Where < filename> is the name of the new MD installation package stored in a temp folder on the Data Mediator/

11. Reboot both nodes.

Appendix C: Customizing Data ExportIn some circumstances you may wish to customize the format of the data exported.Allot provides direct access to the configuration file to enable you to do this via thefollowing process.


OR


OR

Select the Network in the Navigation tree and then click the Configuration icon on the toolbar.


2. Select the General tab.

3. The current DataExportConfig.xml file is displayed. This filecontrols what fields are included in the exported CDR for each of

the data types. If no custom file has been created, a default file isloaded and displayed here.

4. To customize the contents of the export files, you may compose anew version of the DataExportConfig.xml file following thetemplate shown below. Use the Interface fields as listed for each

CDR type in the following chapters. Click Upload and browse tothe location of the new DataExportConfig.xml to load it.

NOTE The DataExportConfig.xml cannot be edited from the NetXplorerGUI, it is only displayed for reference.

DataExportConfig.xml Template

<DataExportConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="./DataExportConfig.xsd">

<DataType name="HDR">

<FilteringRules defaultAction="include">

<Rule action="exclude">

<Condition type="ContentType">image</Condition>

<Condition type="ContentType">text</Condition>

<Condition type="Host">www.google.com</Condition>

</Rule>

<Rule action="exclude">

<Condition type="Host">www.ynet.co.il</Condition>

<Condition type="ContentType">application/x-javascript</Condition>

</Rule>

</FilteringRules>

<OutputStructure version="1">



<Field ID="2" Type="RequestHeader" Name="Host"MaxLength="70"/>

<Field ID="3" Type="StartTime"Format="Numeric/DateTime/String/IPV4/IPV6"/>

<Field ID="4" Type="SubscriberID"Obfuscate="None/Hash/Encrypt"/>

<Field ID="50" Type="ClientIP"/>

<Field ID="51" Type="ClientPort"/>

</OutputStructure>

</DataType>

<DataType name="SDR">

<FilteringRules defaultAction="include"/>

<OutputStructure version="1">

<Field ID="50" Type="ClientIP"/>

<Field ID="51" Type="ClientPort"/>

<Field ID="67" Type="CustomAttr1" RadiusAttr=”26”

RadiusVendorID=”2603” RadiusVendorAttr=”8”/>

</OutputStructure>

</DataType>

</DataExportConfig>

HDR XML Format

NOTE It is recommended that the OutputStructure version value of thesection be increased by one before saving if any changes weremade to this section.

- <DataType name="HDR "> <FilteringRules defaultAction="include" /> - <OutputStructure version="4"> <Field ID="2" Type="EnforcerID" /> <Field ID="1" Type="StartTime" /> <Field ID="3" Type="SubscriberID" /> <Field ID="4" Type="SessionKey" /> <Field ID="5" Type="ClientIP" /> <Field ID="6" Type="ClientPort" /> <Field ID="7" Type="ServerIP" /> <Field ID="8" Type="ServerPort" /> <Field ID="9" Type="ServiceID" /> <Field ID="10" Type="HTTPMethod " MaxLength="10" /> <Field ID="11" Type="RequestHeader" Name="Host" MaxLength="70" /> <Field ID="12" Type="URI" MaxLength="150" />

<Field ID="13" Type="RequestHeader" Name="User-Agent" MaxLength="150" /> <Field ID="14" Type="RequestHeader" Name="Referer" MaxLength="150" /> <Field ID="15" Type="ResponseHeader" Name="Content-Type" MaxLength="50" /> <Field ID="16" Type="DownloadContentLength" /> <Field ID="17" Type="UploadContentLength" /> <Field ID="18" Type="ResponseCode" MaxLength="20" /> <Field ID="19" Type="ServerInitialResponseTime" /> <Field ID="20" Type="Duration" /> <Field ID="21" Type="RequestActualByteCount" />

http://deer/RnD/System/System_Documents/DataExportConfig.xml




<Field ID="22" Type="ResponseActualByteCount" /> <Field ID="23" Type="RequestHeader" Name="DNT,x-do-not-track" MaxLength="1"

/> </OutputStructure> </DataType>

UDR XML Format


- <DataType name="UDR "> <FilteringRules defaultAction="include" />

- <OutputStructure version="1"> <Field ID="3" Type="SubscriberID" /> <Field ID="4" Type="SessionKey" /> <Field ID="39" Type="SubSessionKey" />

<Field ID="40" Type="ChargingApplicationID" /> <Field ID="1" Type="StartTime" /> <Field ID="59" Type="EndTime" /> <Field ID="32" Type="OctetsIn" /> <Field ID="33" Type="OctetsOut" />

</OutputStructure> </DataType>

MOU XML Format


- <DataType name=" MOU"> <FilteringRules defaultAction="include" />

- <OutputStructure version="1"> <Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="45" Type="ServicePlanID" /> <Field ID="9" Type="ServiceID" /> <Field ID="58" Type="SessionDuration" />

</OutputStructure>

VC XML Format

NOTE It is recommended that the OutputStructure version value of the

section be increased by one before saving if any changes weremade to this section.

<DataType name=" VC"> <FilteringRules defaultAction="include" />

- <OutputStructure version="1"> <Field ID="2" Type="EnforcerID" />














<Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="25" Type="LineID" /> <Field ID="26" Type="PipeID" />

<Field ID="27" Type="

VCID" /> <Field ID="28" Type="InstanceType" />

<Field ID="29" Type="InstanceKey1" /> <Field ID="30" Type="InstanceKey2" /> <Field ID="31" Type=" NetUnitID" /> <Field ID="32" Type="LiveConnections" /> <Field ID="33" Type="DroppedConnections" /> <Field ID="34" Type=" NewConnections" /> <Field ID="37" Type="OctetsIn" /> <Field ID="38" Type="OctetsOut" /> </OutputStructure> </DataType>

Conversations XML FormatNOTE It is recommended that the OutputStructure version value of the

section be increased by one before saving if any changes weremade to this section.

- <DataType name="CONV "> <FilteringRules defaultAction="include" />

- <OutputStructure version="1"> <Field ID="2" Type="EnforcerID" /> <Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="5" Type="ClientIP" /> <Field ID="7" Type="ServerIP" />

<Field ID="9" Type="ServiceID" /> <Field ID="25" Type="LineID" /> <Field ID="26" Type="PipeID" /> <Field ID="27" Type=" VCID" /> <Field ID="28" Type="InstanceType" /> <Field ID="29" Type="InstanceKey1" /> <Field ID="30" Type="InstanceKey2" /> <Field ID="31" Type=" NetUnitID" /> <Field ID="32" Type="LiveConnections" /> <Field ID="34" Type=" NewConnections" /> <Field ID="37" Type="OctetsIn" /> <Field ID="38" Type="OctetsOut" />

</OutputStructure> </DataType>

SDR XML Format


- <DataType name="SDR ">










<FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="41" Type="EnforcerDBID" /> <Field ID="42" Type="RecordCreationTimestamp" /> <Field ID="3" Type="SubscriberID" />

<Field ID="4" Type="SessionKey" /> <Field ID="39" Type="SubSessionKey" /> <Field ID="43" Type="SubSessionStartTime" /> <Field ID="44" Type="SubSessionEndTime" /> <Field ID="45" Type="ServicePlanID" /> <Field ID="46" Type="OpenReason" /> <Field ID="47" Type="CloseReason" /> <Field ID="48" Type="SessionIMEI" /> <Field ID="49" Type="SessionSGSN" /> <Field ID="50" Type="SessionTAC" /> <Field ID="51" Type="SessionRAT" /> <Field ID="52" Type="SessionIPV4" /> <Field ID="53" Type="SessionIPV6" /> <Field ID="54" Type="SessionAPN" /> <Field ID="55" Type="SessionLocation" />

<Field ID="56" Type="SessionMCC" /> </OutputStructure> </DataType>

Using Obfuscation

Obfuscation can be defined for any attribute in DataExportConfig.xml by editing and

uploading the file. The function will be executed as the last function after any othertransformation function that is defined for the attribute. As an example, Subscriber IDwould first be transformed from binary to a string, and then would be obfuscated (if sorequested).

Two obfuscation methods are supported:

Hash – irreversible, using SHA-1 algorithm

Encrypt – reversible, using AES 128 bit algorithm

Example

In this example, the HDR section of the DataExportConfig.xml file has been edited todefine with obfuscation for the SubscriberID and URI attributes, appearing in red:

- <DataType name="HDR"><FilteringRules defaultAction="include" />

- <OutputStructure version="4"><Field ID="2" Type="EnforcerID" /><Field ID="1" Type="StartTime" /><Field ID="3" Type="SubscriberID" Obfuscate="Hash" /><Field ID="4" Type="SessionKey" /><Field ID="5" Type="ClientIP" /><Field ID="6" Type="ClientPort" /><Field ID="7" Type="ServerIP" /><Field ID="8" Type="ServerPort" />



Data Mediator Install and Admin Guide R2 (13.1.10)

Documents

Transcript of Data Mediator Install and Admin Guide R2 (13.1.10)