Post on 04-Jun-2018
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 1/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 2/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 3/113
Data Mediator Installation and Administration Guide
P/N D357009 R2
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 4/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 5/113
Important Notice
Data Mediator Installation & Administration Guide iii
Important NoticeAllot Communications Ltd. ("Allot") is not a party to the purchase agreement under which NetEnforcer was purchased, andwill not be liable for any damages of any kind whatsoever caused to the end users using this manual, regardless of the form of
action, whether in contract, tort (including negligence), strict liability or otherwise.
SPECIFICATIONS AND INFORMATION CONTAINED IN THIS MANUAL ARE FURNISHED FOR
INFORMATIONAL USE ONLY, AND ARE SUBJECT TO CHANGE AT ANY TIME WITHOUT NOTICE, ANDSHOULD NOT BE CONSTRUED AS A COMMITMENT BY ALLOT OR ANY OF ITS SUBSIDIARIES. ALLOT
ASSUMES NO RESPONSIBILITY OR LIABILITY FOR ANY ERRORS OR INACCURACIES THAT MAY APPEAR IN
THIS MANUAL, INCLUDING THE PRODUCTS AND SOFTWARE DESCRIBED IN IT.
Please read the End User License Agreement and Warranty Certificate provided with this product before using the product.
Please note that using the products indicates that you accept the terms of the End User License Agreement and WarrantyCertificate.
WITHOUT DEROGATING IN ANY WAY FROM THE AFORESAID, ALLOT WILL NOT BE LIABLE FOR ANYSPECIAL, EXEMPLARY, INDIRECT, INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND,
REGARDLESS OF THE FORM OF ACTION WHETHER IN CONTRACT, TORT (INCLUDING NEGLIGENCE),
STRICT LIABILITY OR OTHERWISE, INCLUDING, BUT NOT LIMITED TO, LOSS OF REVENUE ORANTICIPATED PROFITS, OR LOST BUSINESS, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
CopyrightCopyright © 1997-2013 Allot Communications. All rights reserved. No part of this document may be reproduced,
photocopied, stored on a retrieval system, transmitted, or translated into any other language without a written permission and
specific authorization from Allot Communications Ltd.
TrademarksProducts and corporate names appearing in this manual may or may not be registered trademarks or copyrights of their
respective companies, and are used only for identification or explanation and to the owners' benefit, without intent to infringe.
Allot and the Allot Communications logo are registered trademarks of Allot Communications Ltd.
NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 ofthe FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment
is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if notinstalled and used in accordance with the instruction manual, may cause harmful interference to radio communications.
Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be
required to correct the interference at his own expense.
Changes or modifications not expressly approved by Allot Communication Ltd. could void the user's authority to operate theequipment.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 6/113
Important Notice
Data Mediator Installation & Administration Guideiv
Version History
Doc Version Product Date Updates
v2b11 MED13.1 24.07.13 List of required portsammended
v2b10 MED13.1 09.07.13 ExcludedConversations CDRsclarified.
v2b9 MED13.1 26.05.13 GA Version, RecordExport Rate
clarification
v2b8 MED13.1 05.02.13 BDR Configuration
v2b7 MED13.1 24.12.12 Enabling commands,Obfuscation, Custom
Attributes added
v2b6 MED13.1 16.12.12 DistributedCollection update,
ConversationsPriority CLI changes
v2b5 MED13.1 04.12.12 DistributedCollection and DataExport Rate
information added
v2b4 MED13.1 02.12.12 General edits, XMLinformationreorganized
v2b3 MED13.1 29.11.12 Field Descriptions
v2b2 MED13.1 25.11.12 New CDR Types
v2b1 MED13.1 21.11.12
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 7/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 8/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 9/113
Data Mediator Installation & Administration Guide v
Table of Contents
Important Notice .......................................................................................................................... iii
Table of Contents ........................................................................................................................... v
CHAPTER 1: OVERVIEW ................................................................................ 1-1
Data Types................................................................................................................................... 1-2
Record Export Rates ................................................................................................................. 1-3
DM Performance Rates ............................................................................................................. 1-3
Data Flow – System Level .......................................................................................................... 1-4
Data Flow – Data Mediator ....................................................................................................... 1-5
Glossary ....................................................................................................................................... 1-7
CHAPTER 2: HARDWARE .............................................................................. 2-1
Overview...................................................................................................................................... 2-1
AC Power Supply ....................................................................................................................... 2-2
AC Data Mediator (PA-MED-SRV and PA-MED-SRV-HAP) ............................................... 2-2
AC Storage Device ................................................................................................................... 2-3
DC Power Supply ....................................................................................................................... 2-4
DC Data Mediator (PA-MED-SRV-HAP-DC) ........................................................................ 2-4
DC Storage Device ................................................................................................................... 2-5
CHAPTER 3: INSTALLING DATA MEDIATOR ............................................... 3-1
Connecting to the Network ........................................................................................................ 3-1
Preparing the Network .............................................................................................................. 3-1
Connecting a Single Data Mediator .......................................................................................... 3-2
Connecting an HA Data Mediator Cluster ................................................................................ 3-2
CHAPTER 4: CONFIGURING DATA MEDIATOR ........................................... 4-1
Data Mediator Configuration .................................................................................................... 4-1
Configuring Network Parameters ............................................................................................. 4-1
Connecting to Data Mediator ................................................................................................ 4-1
Configuring Device Parameters ................................................................................................ 4-6
Usage ..................................................................................................................................... 4-6
Example ................................................................................................................................ 4-6
Configuring IMM Parameters ................................................................................................... 4-6
Rebooting the Data Mediator .................................................................................................... 4-9
Data Mediator High Availability .............................................................................................. 4-9
Pre-Installation Requirements ................................................................................................... 4-9
Installing the HA Software ..................................................................................................... 4-10
Editing the /etc/hosts file ........................................................................................................ 4-11
Verify a Successful Installation .............................................................................................. 4-12
Enabling and Configuring Data Mediation............................................................................ 4-12
Enabling Subscriber Data Export License .............................................................................. 4-12
Enabling Distributed Collection ............................................................................................. 4-13
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 10/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 11/113
Data Mediator Installation & Administration Guide vii
CHAPTER 10: USAGE DETAIL RECORDS (UDR) ........................................10-1
Configuring UDRs on the Data Mediator .............................................................................. 10-1
Monitored Applications .......................................................................................................... 10-3
Naming Conventions ................................................................................................................ 10-4
UDR Interface Fields ................................................................................................................ 10-5 Output Example ....................................................................................................................... 10-6
CHAPTER 11: BINARY DATA RECORDS (BDR) ..........................................11-1
Enabling and Configuring BDRs on the In-Line Platform ................................................... 11-1
Legend .................................................................................................................................... 11-2
Configuring BDR Export on the NetXplorer ......................................................................... 11-3
Naming Conventions ................................................................................................................ 11-5
CHAPTER 12: SNMP INTERFACE .................................................................12-1
Data Mediator Traps ................................................................................................................ 12-1
Data Mediator KPIs ................................................................................................................. 12-1 Incoming Records ................................................................................................................... 12-2
Processed Records .................................................................................................................. 12-2
Output Records ....................................................................................................................... 12-3
CHAPTER 13: APPENDICES .........................................................................13-1
Appendix A: Data Mediator Logs ........................................................................................... 13-1
Appendix B: Upgrade Procedure ............................................................................................ 13-1
Appendix C: Customizing Data Export ................................................................................. 13-3
DataExportConfig.xml Template ............................................................................................ 13-4
HDR XML Format.................................................................................................................. 13-5
UDR XML Format.................................................................................................................. 13-6 MOU XML Format ................................................................................................................. 13-6
VC XML Format .................................................................................................................... 13-6
Conversations XML Format ................................................................................................... 13-7
SDR XML Format .................................................................................................................. 13-7
Using Obfuscation .................................................................................................................. 13-8
Example .............................................................................................................................. 13-8
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 12/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 13/113
Data Mediator Installation & Administration Guide 1-1
Chapter 1: Overview
In addition to its powerful core functionality, the Allot solution elements can serve as
a comprehensive data source. Working together with the Data Mediator, data can becompiled, collected, mediated and exported to external systems for the purpose ofstorage or analysis.
Various types of statistics and detail records are generated by the Allot solution. The
data must be collected periodically from the Data Source (Allot In-Line Platforms orSubscriber Management Platforms), transformed into a suitable format and then
pushed to target devices that will process or store the data according to the needs ofeach individual customer.
The Allot Data Mediator is an infrastructure component that provides:
Single and unified infrastructure for data mediation between
producers and consumers, where consumers may be internal orexternal to the system
Shorter implementation time to support collection,
transformation and transfer of new data types
Faster compliance with customer needs by supporting pluggableadapters without modifying generic infrastructure
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 14/113
Data Mediator Installation & Administration Guide1-2
Data TypesAllot in-line platforms, together the Allot Data Mediator, serve as a data source for
the following data types:
This data can be used to generate many different types of statistics, including:
1. Usage – based on Conversations records extracted from in-linetraffic, Allot can provide a 360 degree view of network,subscriber, and app usage for monitoring and planning
2. Mobile – based on Mobile session records (SDR/UDR)generated by the Allot SMP (Subscriber Management Platform)from RADIUS messages it receives. Allot can provide statisticson Mobile handset usage that is useful for new handsetintroduction and monitoring as well as monitor Mobile networkload e.g. by analyzing session bitrates.
3. OTT – based on MOU records, Allot can monitor the leakage ofVoice calls to OTT VoIP.
4. Domain-specific – Allot can provide HTTP records extractedfrom HTTP traffic. The customer can use that for Web traffic
analysis and subscriber profiling and trends and for generatingnew revenue streams (Proactive Analytics).
5. Policy – based on VC stats extracted from in line classified
traffic providing usage based on the service plan and subscriber
•Bandwidth per connection, service
•Customization of preferred and excluded applicationConversation
•Bandwidth per subscribers
•Customization of filtering rules
Session(SDR/UDR)
•Web activity usage per subscriber
•Define Application to monitorHDR
•Bandwidth per policy elementVC
•VOIP usage per service plan and serviceMOU
•Binary raw dataBDR
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 15/113
Data Mediator Installation & Administration Guide 1-3
level info, Allot can provide subscriber service plan quota usage,
for service plan tuning and capacity planning NP> rephrase. VCrecords provide usage statistics per the policy elements managed
by the system. The policy elements may represent subscribers
and service plans.
Allot data feeds are delivered with minimum impact on DPI performance. Recordsinclude a unique identifier which allows association between the different statistical
record types for creating a single enriched record.
Record Export Rates
For SG-Sigma and SG-Sigma E Service Gateways, the following values are per CoreController.
NOTE 1 Data Mediator supports up to 20 CC-300 blades.
PERINTERVAL(SECONDS)
AC1400 AC3000 AC6000 SIGMASIGMAE
UDR 300180,000 180,000 360,000 480,000 1,500,000
HDR 11,000 1,000 7,500 2,500 10,000
VC 300160,000 160,000 500,000 500,000 1,200,000
VC 30 80,000 80,000 250,000 250,000 600,000
Conversation 300600,000 600,000 1,200,000 600,000 1,500,000
Conversation 30300,000 300,000 600,000 300,000 750,000
MOU 3001,600 1,600 1,600 1,600
DM Performance Rates
5 MINS AGGREGATION 10-60 MINS AGGREGATION
UDR 360M 180M
HDR 720M 360M
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 16/113
Data Mediator Installation & Administration Guide1-4
5 MINS AGGREGATION 10-60 MINS AGGREGATION
VC* 288M 144M
Conversation 360M 180M
SDR 22M 11M
MOU 400K 200K * The In-line platform limits the number of exported VC’s to the # of supported VC in thedistributed collector (4M per 5 Minutes).
The Aggregation value affects all exported records. For information concerningsetting this value, see Configuring Conversations Records on the Data Mediator on
page 8-1.
Data Flow – System Level
Figure 1-1: System Level Data Flow
Step 1- The RADIUS server performs the AAA functionality forthe new subscriber session.
Step 2 – The RADIUS Server passes the RADIUS accounting
message to the SMP.
Step 3 – The SMP extracts relevant subscriber information fromthe RADIUS message and passes this on to the In-line Platform.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 17/113
Data Mediator Installation & Administration Guide 1-5
It also generates SDR buckets which aggregate this information
on the SMP.
Step 4a – The subscriber is granted access to the Internet and
subscriber traffic flows through the In-line platform. The In-line platform records the relevant subscriber data (in a series of
different data buckets) and…
Step 4b – …forwards the subscriber’s traffic to the internet
according.
Step 5 – The Data Mediator polls the various data sources (inthis case the In-line Platform and the SMP) for the configured
data buckets,
Step 6 – The Data Mediator formats the data and sends it on to
the Data Warehouse or third-party analytics system. Data Flow – Data Mediator
The Data Mediator performs three key functions:
Collects data from in-line platforms
Transforms the data to an output format
Pushes the data to the required targets
Figure 1-2: Data Mediator Data Flow
Data MediatorData
Source
Collector
File ListTransformed
Output (4)
Transformer(3)
Purge(6)
Push (5)
DataTarget
1
2
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 18/113
Data Mediator Installation & Administration Guide1-6
The Data Mediator collects data from a list of configured Allot data sources.Periodically, the Data Mediator retrieves (1) the list of potential files to collect from
each data source (in-line platform or SMP)
NOTE It is the responsibility of the Data Mediator to identify the filesthat have not been processed yet, and to avoid duplicateprocessing of files.
New files are retrieved by the Collect (2) function, and depending on the type of thecollected data, a suitable Transform (3) function is executed. The Data Mediatorsupports different transform adapters per data type to facilitate changing adapters
without making any change to the infrastructure.
Transformations are either of the following types:
Simple assignments from input to output
Data type conversions – binary to textual (e.g. date, time, IP
address)
A transformed file is stored in the Data Mediator storage (4).
The system supports a push mechanism towards the Target consumers. Therefore,files are pushed by the Push (5) function.
At the completion of the life cycle, transformed files are purgedfrom the Data Mediator storage by the Purge (6) function,
according to the retention policy for the data type.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 19/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 20/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 21/113
Data Mediator Installation & Administration Guide 2-1
Chapter 2: Hardware
OverviewThe Allot Data Mediator is a 1U high appliance based on an IBM M3 x3550 server. Theappliance is shipped from Allot with a Linux based CentOS operating system and Allotsoftware already installed.
The Allot Data Mediator is available in the following configurations:
PA-MED-SRV (AC standalone)
PA-MED-SRV-HAP (AC HAP)
PA-MED-SRV-HAP-DC (DC HAP)
A front and rear view of an Allot Data Mediator (PA-MED-SRV) is illustrated below:
Figure 2-1: PA-MED-SRV – Front View
Figure 2-2: PA-MED-SRV – Rear View
A Data Mediator High Availability cluster is made up of two AC or DC Data Mediators
along with a Storage device. The Storage device supports the same power supply (ACor DC) as the Data Mediators it is clustered with.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 22/113
Data Mediator Installation & Administration Guide2-2
Figure 2-3: PA-MED-SRV-HAP cluster with Storage device – Front View
AC Power Supply
AC Data Mediator (PA-MED-SRV and PA-MED-SRV-HAP)
Two field replaceable and hot swappable 100 VAC to 240 VAC power supplies provideAC power to the PA-MED-SRV and to each Data Mediator Node in the PA-MED-SRV-HAP.
NOTE AC Power Supply units should be removed for maintenancepurposes only.
The server is equipped with two AC power inlets which are located on the rear side ofthe server. The AC inlet connectors are also located on the rear of the server.
Figure 2-4: AC Power Inlets
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 23/113
Data Mediator Installation & Administration Guide 2-3
Each AC power supply is capable of supplying 835w. One power supply is fullycapable of supplying the power consumed by a fully occupied server. The second powersupply is for use in cases where redundancy is needed.
NOTE Connect each power entry to a different power source forredundancy
Further power specifications for the Allot Data Mediator (PA-MED-SRV) are detailedin the table below:
Power Consumption (max) 835 W 2,851 BTU/hour
AC Power Supply 1+1 for redundancy 100-127 VAC, 10A max
200-240 VAC, 5A max
50/60Hz
Form factor / height Rack/1U
Network Interface Integrated dual Gigabit Ethernet
Power Supply (std/max) 835W 1/2 AC standard
Hot swap components Power supply, fans and hard
disk drives
AC Storage Device
The PA-MED-SRV-HAP comes with an IBM storage device
NOTE The power control button on the device and the power switch onthe power supply do not turn off the electrical current supplied to
the device. The device also might have more than one powercord. To remove all electrical current from the device, ensure thatall power cords are disconnected from the power source.
You must follow the power sequence in the order shown in thefollowing procedure. To establish power redundancy forenclosures with two power supplies, use at least two powerdistribution units (PDUs) in the rack cabinet. Split the powerconnections from each enclosure into the separate PDUs. Then,connect the PDUs to external power receptacles that are ondifferent circuits.
To connect the AC Storage Device to the power supply, follow the procedure below:
To turn on the storage subsystem ac power, complete the following steps:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 24/113
Data Mediator Installation & Administration Guide2-4
6. Connect a power cord to each power supply in the storagesubsystem.
Figure 2-5: AC Power Inlet, Storage (Right Side)
7. Connect the other end of each power cord to the power receptacle.
8. Turn on the power to the switches, if applicable.
9. Turn on both power switches on all of the attached storage
enclosures, and wait 60 seconds.
10. Turn on both power switches on the storage subsystem.
NOTE When you turn off the power to the storage subsystem, completethe preceding steps in reverse order. Turn off the power to thestorage subsystem first; then, turn off the power to the storageenclosures.
DC Power Supply
DC Data Mediator (PA-MED-SRV-HAP-DC)
Two field replaceable and hot swappable -48V DC power supplies provide DC power tothe PA-MED-SRV-HAP-DC.
NOTE DC Power Supply units should be removed for maintenancepurposes only.
The server is equipped with two -48V DC power inlets which are located on the rearside of the server. DC power cables are not provided by Allot.
Figure 2-6: DC Power Inlets
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 25/113
Data Mediator Installation & Administration Guide 2-5
Each DC power supply is capable of supplying 675w. One power supply is fullycapable of supplying the power consumed by a fully occupied server. The second powersupply is for use in cases where redundancy is needed.
Further power specifications for the Allot Data Mediator (PA-MED-SRV-HAP-DC) aredetailed in the table below:
Power Consumption (max) 675W
DC Power Supply 1+1 for redundancy -48 V DC
Form factor / height Rack/1U
Network Interface Integrated dual Gigabit Ethernet
Power Supply (std/max) 675W 1/2 DC standard
Hot swap components Power supply, fans and hard
disk drives
DC Storage Device
The PA-MED-SRV-HAP-DC comes with an IBM storage device. To connect the DCStorage Device to the power supply, follow the procedure below:
1. Make sure that the power on-off switches of both DC powersupplies in the storage subsystem and all attached storageenclosures are in the off position.
Figure 2-7: DC Power Inlet, Storage (Right side)
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 26/113
Data Mediator Installation & Administration Guide2-6
2. Connect a DC jumper cable to each DC power supply in thestorage subsystem and to all attached storage enclosures:
Using the supplied strap, tie the DC power supply cable to the rail
to provide strain relief for the power cable.
Connect the DC power cable to the power supply. See below for
the DC power supply connector PIN positions.
Figure 2-8: DC Pin Positions
NUMBER FUNCTION DC POWER CABLE WIRE COLOR
1 Pin 1: - 48 V dc Brown
2 Pin 2: POS RTN Blue
3 Pin 3: GND Green/yellow
3. Connect the - 48V wire (brown) of the DC power cable to anapproved disconnect device (circuit breaker) rated at 30 A. The
disconnect device must be easily accessible from the back of theStorage unit.
NOTE The disconnect device (circuit breaker) must be rated at 30 A.
Make sure that only 10 AWG copper conductor wires are used forall of the wiring between the DC power connectors and the DCpower source.
Complete the wiring from the disconnect device to the terminalmarked -48V of the reliably grounded safety extra low voltage(SELV) DC power source. Connect the POS RTN wire (blue) andthe ground wire (green/yellow) of the dc power cable to theterminals marked POS RTN and GND on the DC power source.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 27/113
Data Mediator Installation & Administration Guide 2-7
Figure 2-9: DC Storage Device Wiring
NUMBER FUNCTION
1 Pin 1: - 48 V DC
2 Pin 2: POS RTN
3 Pin 3: GND
4 DC power source
5 Disconnect device (circuit breaker)
4. After connecting all of the DC jumper cables, turn on the power tothe -48 V dc SELV power source.
5. Turn on both power switches on all of the attached storage
enclosures, and wait 60 seconds.
6. Turn on both power switches on the storage subsystem.
NOTE When you turn off the power to the storage subsystem, completethe preceding steps in reverse order. Turn off the power to thestorage subsystem first; then, turn off the power to the storageenclosures. Lastly, turn off the disconnect devices installed onthe -48V SELV power source.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 28/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 29/113
Data Mediator Installation & Administration Guide 3-1
Chapter 3: Installing Data Mediator
Connecting to the Network
Preparing the Network
Certain ports need to be open to enable the required communication between thedifferent components of the Data Mediator solution. The ports required are presented inthe table below.
SOURCE DESTINATION APPLICATION PROTOCOL PORT COMMENTS
DM NX RMI tcp 1098
Data transformationconfiguration(DataExportConfig.xml)
DM NX RMI tcp 3873 Catalogs
DM NX RMI tcp 4446 Subscriber data
DM NX RMI tcp 1099JNP server bindaddress
NX DM SNMP udp 161 set NX ip
DM NX SNMP udp 161,162 traps
Operator DM SNMP udp 161,162
MIB retreival for
KPIs
DM NE HTTP tcp 80*Manifest and rawdata files retrieval
DM SMP HTTP tcp 80Manifest and rawdata files retrieval
DM OperatorFTPSFTPSCPFile Copy tcp
21,22(or any portrequired for
communicationwith destination
server)
Push transformedfiles to operator fileserver or DWHenvironment
Operator DM SFTP tcp
21,22
(or any portrequired forcommunicationwith destination
server) Operator pullstransformed data
DM NX NTP udp,tcp 123If NX is the source ofTime Synch
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 30/113
Data Mediator Installation & Administration Guide3-2
SOURCE DESTINATION APPLICATION PROTOCOL PORT COMMENTS
DMExternal TimeServer NTP udp,tcp 123
If an external Timeserver of Operator's
network is thesource of TimeSynch
* If TCP port 80 is closed between the Data Mediator and the NE/SG, the NetXplorer
will report the Data Mediator is unreachable.
That is in addition to the ports which need to be open in every NetXplorer installation(see NetXplorer Installation and Admin Guide, Appendix E)
Connecting a Single Data Mediator
To connect a single Data Mediator to the network, follow the instructions below:
Connect a straight copper cable from port 1 illustrated below on
the rear of the Data Mediator to a management switch.
Connect a straight copper cable from the IMM port illustrated
below on the rear of the Data Mediator to a laptop for directconnectivity and maintenance (in blue).
Optionally, you may also connect an additional straight coppercable from the adjacent port (port 2) to a secondary management
switch for backup purposes.
Connecting an HA Data Mediator Cluster
Data Mediator Cluster Hardware Summary
Figure 3-1: Connecting a Single Data Mediator (M3 Hardware)
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 31/113
Data Mediator Installation & Administration Guide 3-3
To connect the HA Data Mediator Cluster (2 x Data Mediators and a RAID storagedevice with dual controllers), refer to Figure 3-2 and follow these instructions:
Figure 3-2: HA Cluster Network Connections (M3 Hardware)
In a High Availability Cluster configuration, the Data Mediator units are connected bytwo physical links (and each Data Mediator server to the RAID by dedicated SAScables) as follows:
1. Use a crossed copper cable to connect between Port 3 on one DataMediator server and Port 3 on the second Data Mediator server.(illustrated in green above)
Figure 3-3: Close up Rear View of Data Mediator node in Data Mediator-HAP
(M3)
2. Use a null modem serial cable (RS 232) to connect between theSerial COM port on one Data Mediator server and the Serial COM
port on the second Data Mediator server. (illustrated in red above)
3. Use a Serial Attached SCSI (SAS) cable (see Figure 3-4) toconnect between each Data Mediator server and the RAID storageserver (as illustrated in Figure 3-5). (These connections and theexact ports required are illustrated in orange in Figure 3-2 above)
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 32/113
Data Mediator Installation & Administration Guide3-4
Figure 3-4: SCSI (SAS) Cable
Figure 3-5: SCSI (SAS) Cable connections to RAID Storage Server
4. Connect each Data Mediator server to the management networkvia Port 1 (shown in blue in Figure 3-2) with an additional link viaPort 2.
5. Additional management cables should be connected to the IMMmodule on each Data Mediator server for direct connectivity andmaintenance (in blue).
6. An additional optional management cable may be connected to theRAID storage server for storage management and storage traps.
The serial (SAS) cable, the crossed copper cable and the two SAS cables are providedwith the Data Mediator High Availability kit.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 33/113
Data Mediator Installation & Administration Guide 4-1
Chapter 4: Configuring Data Mediator
Data Mediator ConfigurationIrrespective of whether you are working with basic Data Mediator architecture or a highavailability Data Mediator cluster, each Data Mediator server that you are working with
must be prepared for use. Four initial steps are required
1. Configure network parameters
2. Configure device parameters
3. Configure IMM parameters
4. Reboot the Mediator
Configuring Network Parameters
The following network parameters need to be assigned to each of the Data Mediatorsyou are using:
Data Mediator IP and mask
Data Mediator Gateway IP for network access
DNS Server IP
Host name
NTP Time Server IP
NOTE The default gateway must be configured. If not configured, insome circumstances, you may not be able to add the SMP to theNetXplorer.
Connecting to Data Mediator
Connect a keyboard, mouse and monitor to the front of the Data Mediator server asshown in Figure 4-1.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 34/113
Data Mediator Installation & Administration Guide4-2
Figure 4-1: Connecting the Data Mediator (M3) – Front View
Log in as a root user: username: root password: bagabu
1. Configure the network settings via the CentOS UI.
2. Open a network connection by selecting Administration/Networkfrom the System menu as shown in Figure 4-2 below
Figure 4-2: Open Network Configuration
3. From the “Devices” tab, select the “eth0” interface as shown inFigure 4-3 below.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 35/113
Data Mediator Installation & Administration Guide 4-3
Figure 4-3: Select eth0
Double click the selected interface. The Ethernet Device dialog
appears.
Figure 4-4: Ethernet Device dialog box
4. In the General tab set the IP address, Subnet mask and Default
Gateway in the “Statically set IP addresses” section.
NOTE The Default Gateway MUST be set during the initial configuration.
5. Click OK to save and return to the Network Configuration dialog.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 36/113
Data Mediator Installation & Administration Guide4-4
Figure 4-5: Network Configuration dialog box, Hosts tab
6. Open the Hosts tab and click New to create a new Host.
The Add/Edit Hosts dialog appears.
Figure 4-6: Network Configuration dialog box, Add/Edit Hosts dialog
7. Enter the IP Address, Host name and Alias for the new host andclick OK to return to the Network Configuration dialog. The new
host will appear in the Hosts tab. To edit an existing Host, clickthe Edit button.
Figure 4-7: Network Configuration dialog box, DNS tab
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 37/113
Data Mediator Installation & Administration Guide 4-5
8. Open the DNS tab and enter the Host name and DNS IPaddresses.
NOTE The Hostname entered in the DNS tab must be the same as was
added in the Hosts tab.9. Select Save from the File menu to save all changes.
10. Restart the network process via the Devices tab by selecting eth0and then Deactivate followed by Activate.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 38/113
Data Mediator Installation & Administration Guide4-6
Configuring Device Parameters
Once you have installed the MD package and configured the network parameters of theData Mediator, you need to configure the appropriate device parameters in order to
inform the Data Mediator server what functional elements should be enabled.
The dev-setup command informs the Data Mediator what its identity is. You can seethe list of available parameters by running the command “dev_setup.sh” without
parameters.
NOTE If a Data Mediator has been shipped directly from Allot, thisprocess has already been done and should be skipped.
Usage
dev_setup.sh [ -v ] [ -m MODE ] [ -f ] [ -a ]
-v View device setup properties and license details
-m <MODE> Device mode to set
-f Force change of mode and key
-a High availability mode and key
The proper MODE to use in the command to configure a Data Mediator isdatamediation. Once that is set, reboot the Data Mediator.
Once the device is set to datamediation you will need to open the NetXplorer GUI toassign the Data Mediator to that NetXplorer.
Example
dev_setup.sh -m datamediation
Setting device mode to datamediation...
Request completed successfully.
Device mode successfully changed to datamediation.
Please reboot your device.
Configuring IMM Parameters
The default details of the IMM are as follows:
Default IP: 192.168.70.125
Default User Name: USERID
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 39/113
Data Mediator Installation & Administration Guide 4-7
Default Password: PASSW0RD (where the “0” is not “o” but “zero”)
To configure the network settings of the Integrated Management Module, follow thesteps below:
1. Connect directly from a laptop to the IMM interface on the rear of the DataMediator. The interface is labeled “SYSTEM MGMT” as shown in Figure 4-8
below
Figure 4-8: IMM "System Management" Port
2. Open a web browser. In the address field type the IP address or host name ofthe IMM to which you want to connect.
NOTE If you are logging in to the IMM for the first time after installation,the IMM defaults to DHCP. If a DHCP host is unavailable, it usesthe default static IP address 192.168.70.125. You can obtain theDHCP-assigned IP address or the static IP address from theserver BIOS or from your network administrator.
3. Enter User ID and Password
4. You will be prompted to specify an inactive session timeout value. Choose avalue from the dropdown list and click on Continue.
5. You will see the IMM User Interface, with the default “System Status” in view,as seen in below:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 40/113
Data Mediator Installation & Administration Guide4-8
Figure 4-9: IMM System Status Screen
6. Select “Network Interfaces” from the system tree on the left side of the screen.
7. In the Ethernet section, make sure that “interface” is enabled, and IPv6 DHCP
is disabled. In addition, DDNS status should be set to “Disabled” and “Domain Name Used” should be set to “manual”.
8. In the IPv4 section, make sure that the DHCP field is set to: “Disabled – UseStatic IP configuration”. Assign an IP, mask and default gateway as seen above
and click Save. You can now access the IMM remotely using these networksettings.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 41/113
Data Mediator Installation & Administration Guide 4-9
Figure 4-10: IMM Network Interfaces
Rebooting the Data Mediator
To reboot the Data Mediator server, log in as a root user and enter: reboot
Request completed successfully
Broadcast message from root (Sun Mar 1 16:59:18 2009):
Data Mediator High Availability
The High Availability (HA) solution for Allot’s Data Mediator Platform is based onusing two installations of Data Mediator for high availability applications and a sharedstorage unit for Sybase database files.
Pre-Installation Requirements
Before starting a “high availability” installation, you will need to ensure that both the
Data Mediator servers which you will be using have the same Data Mediator softwareversion.
NOTE: Make sure that both DMs are configured with the dev-setup “-a ”flag for high availability i.e: dev-setup.sh –m datamediator –a
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 42/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 43/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 44/113
Data Mediator Installation & Administration Guide4-12
192.168.168.2 MED-1
x.x.x.x host.example.com host
In this example, x.x.x.x represents the management interface IP address - it should be
different in each Data Mediator node.
Verify a Successful Installation
After running the hainstall script, there are several checks that you can run to verify asuccessful installation.
Verifying the creation of a cluster using the display file system command
df-h
Verify network parameters for the cluster using ifconfig – a
Check the High Availability status by entering the crm_mon command.
Confirm storage connectivity by entering the cat/proc/partitions command.
Enabling and Configuring Data MediationThe process of configuring Subscriber Data Export consists of 4 distinct stages:
1. Enabling Subscriber Data Export license via activation key
2. Adding IP addresses to each Core Controller
3. Adding a Data Mediator
4. Configuring and enabling each type of CDR.
Each of these stages is examined in detail below.
Enabling Subscriber Data Export License
In order for the Subscriber Data Export feature to work, the operator must first have purchased the correct license for each Core Controller blade on one or more of theService Gateways in the network. Only those Service Gateways for which SubscriberData Export have been enabled will be able to collect usage data to be later exported.
There are two different licenses for subscriber data export functionality:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 45/113
Data Mediator Installation & Administration Guide 4-13
Statistics Export License for Conv, VC, MOU, SDR, UDRexport
HTTP CDR License for HTTP CDR and BDR export
To Verify that the Proper Subscriber Data Export License is Activated:
1. In the Navigation pane, select and right-click the NetEnforcer orService Gateway in the Navigation tree and select Configuration
from the popup menu.
OR
Select the NetEnforcer or Service Gateway in the Navigation tree and thenselect Configuration from the View menu.
ORSelect the NetEnforcer or Service Gateway in the Navigation tree and then
click the Configuration icon on the toolbar.
The Configuration window for the selected NetEnforcer or Service Gateway isdisplayed.
2. Select the Identification & Key tab.
3. In the Activation Key Details pane, scroll down to check ifStatistics Export has been activated for each Core Controller in theService Gateway as shown in Figure 4-11 below. If the correctnumber of Core Controllers do not appear in the Statistics Exportline, contact Allot Customer Support at support@allot.com.
Figure 4-11: Subscriber Data Export License Enabled
NOTE This may also be verified from the CLI using the followingcommand:
gcy key
Enabling Distributed Collection
In multi-blade In-line Platforms Allot has two modes of data collection:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 46/113
Data Mediator Installation & Administration Guide4-14
Centralized Collection: where data is collected on each Core
Controller blade and transferred to the Host Blade (SGSV-100 on
SG-Sigma or SFB-300 in slot 7 on SG-Sigma E) where it is stored.The host blade maintains a list of each data file that is stored.
Distributed Collection: where data is collected on each Core
Controller blade and stored there. The host blade maintains a
central list of each data file, and the core controller informs thehost blade which files should be added or removed from the listand on which blade each one is stored.
Distributed collection must be used with the Data Mediator. In order to enabledistributed collection you will need to assign IP addresses from the managementnetwork to each Core Controller and SFB in the system and then enable direct access to
them. The procedures for doing this are detailed below.
To set the IPs for CC blades in slots 1-5 and 10-14 in two steps:Enter the following commands:
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots1:5
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots10:14
In the commands above x.x.x.x:255.255.0.0 represents the IP:Subnet Mask whilethe y.y.y.y represents the Default Gateway. The – slots value 1:5 means that theslot number will increment up from 1 to 5, while 10:14 means the slot numberwill increment up from 10 to 14.
Using these commands will cause the IP address (x.x.x.x in the example) to
increment up as follows:
CC n will get IP address x.x.x.x
CC n+1 will get IP address x.x.x.x+1
etc.
To set the IP for each CC blade individually:
1. Enter the following commands for the relevant slots:
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 1:1
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 2:2
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 3:3
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 4:4
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 5:5
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 47/113
Data Mediator Installation & Administration Guide 4-15
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots10:10
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots11:11
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots12:12
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots13:13
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots14:14
2. To view the configuration, type the following command:sysadmin@EXC-SBH[1/6]:~$ go config view
blade_mngt_ips
To set the IP for each SFB blade individually:
1. Enter the following commands for the relevant slots:
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 6:6
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 7:7
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 8:8
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 9:9
Output Example
==== Blade Management IPs ====
|Slot |IP |Mask |Gateway
--------------------------------------------------------------
|4 |10.150.6.34 |255.255.0.0 |10.150.0.1
--------------------------------------------------------------
Request completed successfully.
To enable Distributed Collection in the NetXplorer:
1. In the Navigation pane, select and right-click the desired In-linePlatform in the Navigation tree and select Configuration from the
popup menu.
OR
Select the desired In-line Platform in the Navigation tree and then selectConfiguration from the View menu.
OR
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 48/113
Data Mediator Installation & Administration Guide4-16
Select the desired In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
The Configuration window for the selected NetEnforcer or Service Gateway isdisplayed.
2. Select the IP Properties tab.
3. In the Direct Access pane, check the Management IP per CCcheckbox.
Figure 4-12: IP Properties tab
Adding a Data Mediator
Open NetXplorer.1.
In the Navigation pane, right-click Servers in the Network pane in2.
the Navigation tree and select New Data Mediation… from the popup menu.
The Data Mediation Properties - New dialog is displayed.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 49/113
Data Mediator Installation & Administration Guide 4-17
Figure 4-13: Data Mediation Properties
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 50/113
Data Mediator Installation & Administration Guide4-18
Enter the name and IP address of the Data Mediator.3.
In the Source Units area, use the arrow keys to move In-line4.
Platforms and SMPs from the Available to the Selected lists.
Those In-line Platforms and SMPs selected will provide data tothe Data Mediator.
In the Record Types to Collect area, select those record types that5.
this Data Mediator will collect. Multiple data types may beselected, depending upon the capabilities of the platforms selected
as Source Devices as well as the current license key.
The possible data types are VC, Conversations, HTTP Detailed
Records (HDR), Minutes of Use (MOU), Binary Detail Records(BDR), Usage Detail Records (UDR) and Session Detail Records
(SDR).
NOTE SDR collection is only possible if you have included an SMP inthe Selected Sources.
Click Save to add the Data Mediator to the network.6.
Click Save. The Monitoring Collector is added to the Navigation7.
tree. The New Collector operation can take up to a couple ofminutes to complete.
Enabling and Configuring CDR Types
The way that the Data Mediator works with each of the seven types of CDRs (VCs,Conversations, HTTP Detailed Records (HDR), Minutes of Use (MOU), Binary Detail
Records (BDR), Usage Detail Records (UDR) and Session Detail Records (SDR)) must be configured from the Network Configuration dialog within the NetXplorer GUI.
In addition, HDRs, MOU and Conversation records must be enabled on each relevantIn-line Platform individually. SDRs must be enabled on each relevant SMP, which willalso enable UDRs.
For details, see the subsequent chapters outlining the requirements for each CDR type.
NOTE In some circumstances you may wish to customize the format ofthe data exported. Allot provides direct access to theconfiguration file to enable you to do this if required. For moreinformation, see Appendix C on page 13-3.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 51/113
Data Mediator Installation & Administration Guide 5-1
Chapter 5: HTTP Detail Records (HDR)
HTTP CDRs are exported in CSV file format. Every HTTP complete transaction isrepresented by a record in the CSV file.
The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external systemfrequently (every 1-2mins). These settings can be changed from the Data Mediation tabin the NetXplorer GUI.
The Allot system can be configured to purge the files after a pre-defined period of time(default is 24 hours).
Enabling and Configuring HDRs
Enabling HDRs per In-line Platform
1. In the Navigation pane, select and right-click the relevant In-linePlatform in the Navigation tree and select Configuration from the
popup menu.
OR
Select the In-line Platform in the Navigation tree and then selectConfiguration from the View menu.
OR
Select the In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
The Configuration window for the selected In-line Platform is displayed.
2. Select the Service Activation tab.
3. To enable HDRs, select Enable from the HDR Generation
Enablement drop down field.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 52/113
Data Mediator Installation & Administration Guide5-2
Figure 5-1: HDR CDR area in the Service Activation tab
4. In the HDR CDR, the following other fields may be configured.
HDR Generation Rate Limit – This defines the maximum
number of entries generated per second. The maximum possiblevalue depends upon which In-line Platform or Core Controller
blade is being used, as follows:
CC-300 Blade – 10000
CC-200 Blade – 2500
AC-6000 – 7500
AC-1400/AC-3000 - 1000
HDR File Compression – This sets the file compression methodto be used when storing the file on the In-Line Platform. The
possible options are GZIP and no compression. GZIP is selected by default.
5. Save the configuration changes to apply the new settings.
NOTES The following environments do not support HDRs- Asymmetric Deployment- Captive Portal set to “Use Request”
A Websafe blacklist external server will not create HDRs
In Active Redundancy, the record will be generated on theNetEnforcer receiving the HTTP request
HTTP requests created by a steering service will not be includedin HDR
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 53/113
Data Mediator Installation & Administration Guide 5-3
HTTPS traffic is not included in HDRs
Configuring HDRs on the Data Mediator
1. In the Navigation pane, select and right-click the Network in the Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration fromthe View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
The Configuration window for the Network is displayed.
2. Select the Data Mediation tab.
3. To configure HDRs, open the Data Definition tab and select theHDR sub-tab.
Figure 5-2: HDR Data Definition Properties
4. In the HDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file before compression. The default is
100,000 bytes (100 MB) and the maximum possible value is1,000,000,000 bytes (1 GB).
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 54/113
Data Mediator Installation & Administration Guide5-4
File Closing Interval (in seconds) – This defines how long after
being created the export file is closed. The default is 300
seconds (5 minutes) and the maximum is 3600 seconds (1 hour).
File Compression method – This sets the file compressionmethod to be used when exporting the file. The possible optionsare GZIP and no compression. GZIP is selected by default.
Delete files older than (in minutes) – This defines the amount oftime the system retains files. The default is 1440 minutes (24
hours) and the maximum is 4320 minutes (72 hours).
Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis andstorage. The four push method options are SFTP, FTP, SCP and
Copy. If SFTP is selected, the Username, Password, Server and
Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the pathmust be defined. The default is Enabled.
Filtering Rules – See Below.
Filtering Rules
Using the Filtering Rules area, you may elect to Include All Data in your HDRs or toExclude All Data from your HDRs by selecting the appropriate radio button. You may
then add exceptions to the list below by clicking on the Add button.
Figure 5-3: Add Rule dialog
When the Add Rule dialog appears, you may create an exclusion rule by entering aname and clicking Add to add attributes to the rule.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 55/113
Data Mediator Installation & Administration Guide 5-5
Figure 5-4: Add Attribute dialog
Clicking the Add button on the Add Rule dialog allows you to select to add aContentType, Path, Method or Domain and then enter the appropriate value. Click OK
to save your selections.
KEY DESCRIPTION EXAMPLES
ContentType The MIME type ofthis content(MultipurposeInternet Mail
Extensions)
text/html
image/gif
image/jpeg
Path The URI /index.path
/news
Method The desired actionto be performed on
the resource.
GET, CONNECT, POST
Domain The domain name ofa server.
www.cnn.com
www.ynetnews.com
Naming ConventionsHTTP CDRs files are generated according to the following naming conventions:
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Datatype>_<Version>.[<FE>]
FIELD NAME DESCRIPTION NOTES
< Source unit name > The name of the source unit thatgenerated the file
Each name must beunique
<Source file IP> The IP Address of the Core Controller.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 56/113
Data Mediator Installation & Administration Guide5-6
FIELD NAME DESCRIPTION NOTES
<Mediation IP> The IP address of the Mediation unitthat created the file
<Seq> A sequencer that is never reset.
<Date> Creation date of the file in the formyyyymmddhhmmss
<Data type> Represent the file content:
“HDR” – for files containing HTTPCDR records
<Version> Version of the entry format.
Format: V<format number>
The version is set from theDataExportConfig.xml file where therequired output structure is defined.
<FE> File extension.
The file extension is csv.gz for HTTPCDR files.
While the file is being copied a “.tmp”
suffix will be added. This suffix will be removed after the copy has been
completed successfully.
HDR Interface Fields
ATTRIBUTENAME
DESCRIPTION FORMAT
Enforcer Name In-line Platform name which generated the
CDR
String (in quotes)
StartTime Date and Time that the HTTP request
detected in the device.
By default – represented as yyyy-MM-
ddThh:mm:ss+hh:mm
This field can also be set to be represented
as a simple numeric string containing UTC
time in seconds since 1.1.1970
Date String
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 57/113
Data Mediator Installation & Administration Guide 5-7
ATTRIBUTENAME
DESCRIPTION FORMAT
SubscriberID Unique subscriber identifier.
Zero if deployment is without SMP.
Value is external identifier (in mobile
networks) or internal identifier (in Fixed
networks).
String (Numeric String in
mobile environments)
SessionKey Unique identifier of the subscriber session.
The same SessionKey is used for the
Subscriber session in HDRs, SDRs and
UDRs.
Present in subscriber management
environment only. If no SMP is in the
Network, this value appears as 0.
Numeric String
ClientIP IP (IPv4 or IPv6) of the HTTP request
originator
Dotted Numeric String
ClientPort TCP port of the request originator Numeric String
ServerIP IP (IPv4 or IPv6) of the designated HTTP
server.
Dotted Numeric String
ServerPort TCP port of designated HTTP server Numeric String
Service name The Service name mapped from the DPI
application identifier
String (in quotes)
HTTPMethod HTTP method type of the request (GET,HEAD, POST, etc.)
String (in quotes)
Host Domain of the HTTP request, taken from
the Host header
String (in quotes)
URI URI field of the request String (in quotes)
UserAgent UserAgent header from the HTTP request String (in quotes)
Referer Referer header from the HTTP Request String (in quotes)
DownloadContentL
ength
The content length field from the server
response as set by the server application.
Limited to 4GB.
Numeric String
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 58/113
Data Mediator Installation & Administration Guide5-8
ATTRIBUTENAME
DESCRIPTION FORMAT
UploadContentLen
gth
The content length field from the request
as set by the client application. Limited to4GB.
Numeric String
ResponseCode The response code from the server
response
Numeric String
ServerInitialRespo
nseTime
Time interval in milliseconds from
identifying the request until receiving the
server response.
Numeric String
Duration Transaction duration in milliseconds (until
last packet of the response is received)
Numeric String
RequestActualByte
Count
Actual byte count of outgoing traffic as
counted by the DPI engine.
Numeric String
ResponseActualByt
eCount
Actual byte count of incoming traffic as
counted by the DPI engine.
Numeric String
DNT DNT or x-do-not-track header from the
HTTP request
String (in quotes)
Output Example
Figure 5-5: HDR .csv file example
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 59/113
Data Mediator Installation & Administration Guide 6-1
Chapter 6: Minutes of Use (MOU)
MOU CDRs present data for the minutes of use for different over the top VoIP
applications running through the network. MOU CDRs are exported in CSV file format.Every complete transaction is represented by a record in the CSV file.
The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external systemfrequently (every 1-2mins). These settings can be changed from the Data Mediation tab
in the NetXplorer GUI.
The Allot system can be configured to purge the files after a pre-defined period of time(default is 24 hours).
Enabling and Configuring MOU
Enabling MOU per In-Line Platform
1. In the Navigation pane, select and right-click the relevant In-linePlatform in the Navigation tree and select Configuration from the
popup menu.
OR
Select the In-line Platform in the Navigation tree and then selectConfiguration from the View menu.
OR
Select the In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
The Configuration window for the selected In-line Platform is displayed.
2. Select the Service Activation tab.
3. To enable MOU, select Enable from the VOIP Data Collection drop down field in the VOIP Reports area.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 60/113
Data Mediator Installation & Administration Guide6-2
Figure 6-1: VOIP Reports in the Service Activation tab
4. Save the configuration changes to apply the new settings.
Configuring MOU on the Data Mediator
1. In the Navigation pane, select and right-click the Network in the Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration fromthe View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
The Configuration window for the Network is displayed.
2. Select the Data Mediation tab.
3. To configure MOUs, open the Data Definition tab and select theMOU sub-tab.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 61/113
Data Mediator Installation & Administration Guide 6-3
Figure 6-2: MOU Data Definition Properties
4. In the MOU sub-tab, the following fields may be configured.
File Size Limit (in bytes) – This defines the maximum possible
size of each exported file. The default is 100,000 bytes (100MB) and the maximum possible value is 1,000,000,000 bytes (1GB).
File Closing Interval (in seconds) – This defines how long after
being created the export file is closed. The default is 300
seconds (5 minutes) and the maximum is 3600 seconds (1 hour).
File Compression method – This sets the file compression
method to be used when exporting the file. The possible optionsare GZIP and no compression. GZIP is selected by default.
Delete files older than (in minutes) – This defines the amount of
time the system retains files. The default is 1440 minutes (24hours) and the maximum is 4320 minutes (72 hours).
Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis and
storage. The four push method options are SFTP, FTP, SCP andCopy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and
password must be defined. For FTP and Copy, only the pathmust be defined.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 62/113
Data Mediator Installation & Administration Guide6-4
Naming ConventionsMOU files are generated according to the following naming conventions:
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Datatype>_<Version>.[<FE>]
FIELD NAME DESCRIPTION NOTES
< Source unit name > The name of the source unit thatgenerated the file
Each name must beunique
<Source file IP> The IP Address of the file origin.
<Mediation IP> The IP address of the Mediation unitthat created the file
<Seq> A sequencer that is never reset.
<Date> Creation date of the file in the formyyyymmddhhmmss
<Data type> Represent the file content:
“MOU” – for files containing Minutesof Use records
<Version> Version of the entry format.
Format: V<format number>
The version is set from theconfiguration file where the requiredoutput structure is defined.
<FE> File extension.
The file extension is csv.gz for MOUfiles.
While the file is being copied a “.tmp”suffix will be added. This suffix will
be removed after the copy has beencompleted successfully.
MOU Interface Fields
ATTRIBUTENAME
DESCRIPTION FORMAT
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 63/113
Data Mediator Installation & Administration Guide 6-5
ATTRIBUTENAME
DESCRIPTION FORMAT
Start Time Start of the time interval measured by this record.
By default this is represented as yyyy-MM-
ddThh:mm:ss+hh:mm
Can also be set to be represented as a simple
numeric string containing UTC time in seconds
since 1.1.1970
Date String
End Time End of the time interval measured by this record
By default this is represented as yyyy-MM-
ddThh:mm:ss+hh:mm
Can also be set to be represented as a simple
numeric string containing UTC time in secondssince 1.1.1970
Date String
Service Plan Name Service plan name String (in quotes)
Service Name The name of the application mapped from the DPI String (in quotes)
Total seconds VoIP usage in seconds Numeric (D)
Output Example
Figure 6-3: MOU .csv file example
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 64/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 65/113
Data Mediator Installation & Administration Guide 7-1
Chapter 7: VC Records
NOTE VC Records are automatically enabled for all In-line Platforms.
This information concerns traffic that is classified into different rules in the Allotsystem. The data is based on the service plan and the subscriber. Depending on theimplementation, VC statistics may provide policy enforced application or application
group volumes based on the service plan definitions for traffic that needs to be shaped, blocked, steered or TOS-marked. They are extracted from the in-line traffic and arecompiled on the in-line platform (e.g: SG-Sigma)
VC CDRs are exported in CSV file format. Every complete transaction is represented by a record in the CSV file.
The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external system
frequently (every 1-2mins). These settings can be changed from the Data Mediation tabin the NetXplorer GUI.
The Allot system can be configured to purge the files after a pre-defined period of time(default is 24 hours).
Configuring VCs on the Data Mediator1. In the Navigation pane, select and right-click the Network in the
Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration fromthe View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
The Configuration window for the Network is displayed.
2. Select the Data Mediation tab.
3. To configure VCs, open the Data Definition tab and select the VCsub-tab.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 66/113
Data Mediator Installation & Administration Guide7-2
Figure 7-1: VC Data Definition Properties
4. In the VC sub-tab, the following fields may be configured.
File Size Limit (in bytes) – This defines the maximum possible
size of each exported file. The default is 100,000 bytes (100MB) and the maximum possible value is 1,000,000,000 bytes (1GB).
File Closing Interval (in seconds) – This defines how long after
being created the export file is closed. The default is 300
seconds (5 minutes) and the maximum is 3600 seconds (1 hour).
File Compression method – This sets the file compression
method to be used when exporting the file. The possible optionsare GZIP and no compression. GZIP is selected by default.
Delete files older than (in minutes) – This defines the amount oftime the system retains files. The default is 1440 minutes (24
hours) and the maximum is 4320 minutes (72 hours).
Push Properties – Push properties, once enabled, define how the
export files are sent to their proper locations for analysis and
storage. The four push method options are SFTP, FTP, SCP andCopy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and
password must be defined. For FTP and Copy, only the pathmust be defined.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 67/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 68/113
Data Mediator Installation & Administration Guide7-4
ATTRIBUTE NAME DESCRIPTION FORMAT
Enforcer Name Device name which generated the
CDR
String (in quotes)
Start Time Start of the time interval measured bythis record.
By default this is represented as yyyy-
MM-ddThh:mm:ss+hh:mm
Can also be set to be represented as asimple numeric string containing UTCtime in seconds since 1.1.1970
Date String
End Time End of the time interval measured by
this record.
By default this is represented as yyyy-MM-ddThh:mm:ss+hh:mm
Can also be set to be represented as a
simple numeric string containing UTCtime in seconds since 1.1.1970
Date String
Line Name representing the Line String (in quotes)
Pipe Name representing the Pipe String (in quotes)
VC Name representing the VirtualChannel
String (in quotes)
Instance Type See Note 1 Numeric String
Instance Key 1 See Note 1 Numeric String
Instance Key 2 See Note 1 Numeric String
Net Unit ID Cell ID Numeric String
Live connections Number of live Connection in thetime interval
Numeric String
Dropped connections Number of dropped Connection in thetime interval
Numeric (D)
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 69/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 70/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 71/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 72/113
Data Mediator Installation & Administration Guide8-2
2. Select the Data Mediation tab.
3. To configure Conversations CDRs, open the Data Definition taband select the Conversations sub-tab.
Figure 8-1: Conversations Data Definition Properties
4. In the Conversations sub-tab, the following fields may beconfigured.
File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file. The default is 5 minutes and you may
opt for bytes (100 MB) and the maximum possible value is1,000,000,000 bytes (1 GB).
Aggregation Period (min) – This defines how long after beingcreated the export file is closed. The default is 5 minutes and
you may opt for 10, 15, 30 or 60 minutes from the drop downlist. This value will affect ALL record types, not justConversations.
File Compression method – This sets the file compression
method to be used when exporting the file. The possible optionsare GZIP and no compression. GZIP is selected by default.
Delete files older than (in minutes) – This defines the amount of
time the system retains files. The default is 1440 minutes (24
hours) and the maximum is 4320 minutes (72 hours).
Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis and
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 73/113
Data Mediator Installation & Administration Guide 8-3
storage. The four push method options are SFTP, FTP, SCP and
Copy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and
password must be defined. For FTP and Copy, only the path
must be defined.
Prioritizing Conversations Statistics
It is possible for the operator to prioritize the Conversation records by application asfollows:
Golden: Conversation CDRs are exported in full detail due to highinterest.
Excluded: Conversation CDRs are exported in an abridged format
due to lack of interest (only number of connections and throughput
is saved, with no rule ID and client IP).
Normal: Conversation CDRs are exported in summary detail. This
priority level is set to applications by default.
All Conversation records are collected in detail and are summarized to reduce thestorage space required. This feature allows the operator determine which applicationsshould be kept and exported in full Conversation detail, which applications can besummarized and which are exported in a highly abridged format. Sufficient storagespace is then set based on these priorities.
The operator may further define whether the Conversation data should be exported in 5-minute or 1-hour aggregations.
All Conversation records are collected in detail and are summarized to reduce thestorage space required. This feature allows the operator determine which applications
should be kept and exported in full Conversation detail, which applications can besummarized and which need not be exported at all. Sufficient storage space is then set
based on these priorities.
Assigning Priority
Priority levels need to be configured via the CLI using the following commands:
To set the priority level:
go add app_collection [app_name] –priority golden/excluded
app_name – service catalogue name entry (excluding service groups)
To show the priority list by application:
go list app_collection
Output example
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 74/113
Data Mediator Installation & Administration Guide8-4
sysadmin@AC3K-5:~$ go list app_collectionGolden services:
ViberSkypeTango
LineMGCPYahoo ChatJabberGoogle PlayiTunesFacebook ChatBaidu HiQQ ChatTwitterFacebookYandexVKontakteGmailCall Of Duty
GoogleTalkWhatsAppMailRuYahoo MailDiabloIIILineage2Kakao
Excluded services:BGPDNSEGPICMPIGMPOSPF
RIPARPPPPoE-DiscoveryPPPoE-ControlDHCP-ClientDHCP-ServerNTPIS-ISLACP
Request completed successfully.sysadmin@AC3K-5:~$
To change/remove the configuration for a specific application:
go update app_collection [app name] –priority golden/excludedgo delete app_collection [app name] –priority golden/excluded
Naming Conventions
Conversations records are generated according to the following naming conventions:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 75/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 76/113
Data Mediator Installation & Administration Guide8-6
ATTRIBUTENAME
DESCRIPTION FORMAT
Start Time
Start of the time interval measured bythis record.
By default this is represented as yyyy-MM-ddThh:mm:ss+hh:mm
Can also be set to be represented as asimple numeric string containing UTC
time in seconds since 1.1.1970
Date String
End Time
End of the time interval measured bythis record.
By default this is represented as yyyy-
MM-ddThh:mm:ss+hh:mm
Can also be set to be represented as a
simple numeric string containing UTCtime in seconds since 1.1.1970
Date String
Internal IP Internal IP address; both IPv4 andIPv6 are supported
Dotted Numeric String
External IP External IP address; both IPv4 andIPv6 are supported
Dotted Numeric String
Service Name The name of the application mapped
from the DPI
String (in quotes)
Line Name representing the Line Numeric String
Pipe Name representing the Pipe Numeric String
VC Name representing the Virtual Channel Numeric String
Instance Type See Note 1 Numeric String
Instance Key 1 See Note 1 Numeric String
Instance Key 2 See Note 1 Numeric String
Net Unit ID Cell ID Numeric String
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 77/113
Data Mediator Installation & Administration Guide 8-7
ATTRIBUTENAME
DESCRIPTION FORMAT
Live connections Number of live Connection in the timeinterval Numeric String
Dropped connections Number of dropped Connection in thetime interval Numeric (D)
New connections Number of new Connection in the
time interval Numeric (D)
Octets in Byte count of incoming traffic Numeric (D)
Octets out Byte count of outgoing traffic Numeric (D)
Note 1:
INSTANCE TYPE INSTANCE KEY 1 INSTANCE KEY 2
0 Empty, no instance Empty
1 IP address (IPv4 or IPv6) asPipe instance
Prefix for IPv6
2 IP Address (IPv4 or IPv6) asVC instance
Prefix for IPv6
3 Host group name as Pipeinstance
Empty
4 Host group name as VCinstance
Empty
5 Subscriber Identifier as Pipeinstance
Session Key
6 Subscriber Identifier as VCinstance
Session Key
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 78/113
Data Mediator Installation & Administration Guide8-8
Output Example
Figure 8-2: CONV .csv file example
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 79/113
Data Mediator Installation & Administration Guide 9-1
Chapter 9: Session Detail Records (SDR)
SDRs are extracted from the AAA function (e.g: RADIUS) and are compiled on theSMP Server. An SDR is created for every segment of a session (called a sub-session) on
a periodic basis (20m by default. To change the interval, contact support@allot.com). Asession is broken into a sub-session whenever there is a change in the session data (e.g:switch of service plan) or by time interval.
SDRs are exported in CSV file format. Every complete transaction is represented by arecord in the CSV file.
The CSV files are pushed uncompressed or compressed in GZIP format from the DataMediator via SCP, SFTP, Copy or FTP (Default is None) to the external system
frequently (every 1-2mins). These settings can be changed from the Data Mediation tabin the NetXplorer GUI.
The Allot system can be configured to purge the files after a pre-defined period of time(default is 24 hours).
NOTE SDRs are only available from an SMP, not from an In-linePlatform.
Enabling SDRs per SMP
Session Detail Records (SDRs) must be generated on the SMP server to extract therelevant information from the RADIUS message. This is enabled via the NetXplorerGUI.
1. In the Navigation pane of the NetXplorer User interface, right-
click the Network and select Configuration from the popup menu
OR
Select Network in the navigation pane, and then chooseConfiguration from the Actions menu.
The network tabs will be displayed in the Applications pane.
2. Select the SMP tab. In the SMP preferences section, 4 additionaltabs will appear.
3. Select “Session Parameters”. From the Session Parameters tab,determine the extra RADIUS parameters which you wish to beincluded in the SDRs. Check whichever fields you wish toinclude, as shown in Figure 9-1 below.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 80/113
Data Mediator Installation & Administration Guide9-2
Figure 9-1: Adding Additional RADIUS Parameters to the SDR
NOTES You can also set a default value for some fields. This defaultvalue will be included in each SDR for the particular fieldspecified, should no actual value be received.
The AVPs which you select in this dialog to be included in theSDR must have first been configured to be extracted from theRADIUS message in the radius_params.conf file. The procedurefor extracting these AVPs is detailed in Error! Reference source not
ound.
4. Select the “Policy and Charging” tab. Set the SDR CollectionStatus field to “On”
NOTE Unless SDR Collection Status is defined as On in the Policy andCharging Tab, no SDRs will be generated even if SDR Export isenabled in other areas of the GUI.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 81/113
Data Mediator Installation & Administration Guide 9-3
Figure 9-2: Policy and Charging Tab
5. You will be prompted to reboot the SMP server, whereupon SDRcollection will commence.
Configuring SDRs on the Data Mediator1. In the Navigation pane, select and right-click the Network in the
Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration fromthe View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
The Configuration window for the Network is displayed.
2. Select the Data Mediation tab.
3. To configure Conversations CDRs, open the Data Definition taband select the Conversations sub-tab.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 82/113
Data Mediator Installation & Administration Guide9-4
Figure 9-3: SDR Data Definition Properties
4. In the SDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) – This defines the maximum possible
size of each exported file. The default is 100,000 bytes (100MB) and the maximum possible value is 1,000,000,000 bytes (1GB).
File Closing Interval (in seconds) – This defines how long after
being created the export file is closed. The default is 300
seconds (5 minutes) and the maximum is 3600 seconds (1 hour).
File Compression method – This sets the file compressionmethod to be used when exporting the file. The possible options
are GZIP and no compression. GZIP is selected by default.
Delete files older than (in minutes) – This defines the amount of
time the system retains files. The default is 1440 minutes (24hours) and the maximum is 4320 minutes (72 hours).
Push Properties – Push properties, once enabled, define how the
export files are sent to their proper locations for analysis and
storage. The three push method options are SFTP, FTP andCopy. If SFTP is selected, the Username, Password, Server and
Path must be defined. For FTP and Copy, only the path must bedefined.
Naming ConventionsSDRs files are generated according to the following naming conventions:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 83/113
Data Mediator Installation & Administration Guide 9-5
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Datatype>_<Version>.[<FE>]
FIELD NAME DESCRIPTION NOTES
< Source unit name > The name of the source unit that
generated the file
Each name must be unique
<Source file IP> The IP Address of the file origin.
<Mediation IP> The IP address of the Mediation unit that
created the file
<Seq> A sequencer that is never reset.
<Date> Creation date of the file in the form
yyyymmddhhmmss
<Data type> Represent the file content:
“SDR” – for files containing HTTP CDR
records
<Version> Version of the entry format.
Format: V<format number>
The version is set from the configuration
file where the required output structure is
defined.
<FE> File extension.
The file extension is csv.gz for VC files.
While the file is being copied a “.tmp”
suffix will be added. This suffix will be
removed after the copy has been
completed successfully.
SDR Interface Fields
ATTRIBUTE NAME DESCRIPTION FORMAT
Enforcer Name Device name which generated the CDR string (in quotes)
Record creation
timestamp
SDR generation timestamp Date String
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 84/113
Data Mediator Installation & Administration Guide9-6
ATTRIBUTE NAME DESCRIPTION FORMAT
Subscriber ID Unique subscriber identifier
Zero if deployment is without SMP.
Value is external identifier (in mobile networks)
or internal identifier (in Fixed networks).
Numeric String
Session Key Unique identifier of the subscriber session. The
same SessionKey is used for the Subscriber
session in HDRs, SDRs and UDRs.
Present in subscriber management environment
only. If no SMP is in the Network, this value
appears as 0.
Numeric String
Sub-Session Key Unique sub-session identifier per subscriber
session. Sub session is created if there is a
change in the service plan of the subscriber
during the session.
Present in subscriber management environment
only
Numeric String
Sub Session Start Time Start time of the sub session Date String
Sub Session End Time End time of the sub session Date String
Service Plan Service plan name String (in quotes)
Open reason Reason for opening the session. One of the
following values:
START
SERVICE_PLAN_CHANGE
INFO_CHANGE
SCHEDULE
String (in quotes)
Close reason Reason for closing the session. One of the
following values:
SERVICE_PLAN_CHANGE
INFO_CHANGESCHEDULE
CLOSE
String (in quotes)
IMEI This field contains the International Mobile
Equipment Identity (IMEI) of the equipment
served, if available (3GPP-IMEISV)
Numeric String
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 85/113
Data Mediator Installation & Administration Guide 9-7
ATTRIBUTE NAME DESCRIPTION FORMAT
SGSN Initial SGSN addresses used received on session
start (3GPP-SGSN-Address)
Numeric String
IMEI-TAC The TAC part of the IMEI for device type
identification (3GPP-IMEISV)
Numeric String
RAT Radio Access Type used to start the session
(3GPP-RAT-Type)
Numeric String
Source IPv4 Source address of the session in IPv4 format Dotted Numeric
String
Source IPv6 Source address of the session in IPv6 format Dotted Numeric
String
APN Access Point through which the session started
(Called-Station-Id)
String (in quotes)
Location Initial location where the session started (3GPP-
User-Location-Info)
String (in quotes)
MCC MNC Mobile Country Code and Mobile Network
Code (MCC-MNC) of the subscriber's home
operator (3GPP-IMSI-MCC-MNC)
String (in quotes)
CustomAttr1 Attributes that can extracted on-demand by the
end-user
String (in quotes)
CustomAttr2 Attributes that can extracted on-demand by the
end-user
String (in quotes)
CustomAttr3 Attributes that can extracted on-demand by the
end-user
String (in quotes)
Custom Attributes
To define the three custom attributes (CustomAttr1, CustomAttr2 and CustomAttr3),the fields must be added to the radius-dictionary file on each SMP and then added to the
DataExportConfig.xml file.
These three custom fields will then be extracted from the radius message and saved inthe SDR.
For example:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 86/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 87/113
Data Mediator Installation & Administration Guide 9-9
Service Plan ID
Every service plan configured in the system has an internal ID number. To determinethe internal ID number of each service plan, run the pccAdmin.sh – p command on the
SMP. One of the logs created is the ServicePlanCatalog.log. This log will list the ID ofeach Service Plan created, mapping between ID and Service plan name as shown in thesample output below:
less ServicePlanCatalog.log
===== Service Plan Catalog =====
Service Plan Catalog Size= 26
Service Plan Id= 2 ,Service Plan Name= Block Service Plan ,Gx SessionExclude Group Id= 1025
Service Plan Id= 7 ,Service Plan Name= gold ,Gx Session Exclude GroupId= 1030
Service Plan Id= 1 ,Service Plan Name= Unlimited Service Plan ,GxSession Exclude Group Id= 1024
Service Plan Id= 9 ,Service Plan Name= Unlimited ServicePlan_1318349232984 ,Gx Session Exclude Group Id= 0
Service Plan Id= 19 ,Service Plan Name= Block Service Plan_1318349038792,Gx Session Exclude Group Id= 0
Service Plan Id= 6 ,Service Plan Name= vc_sp3 ,Gx Session Exclude GroupId= 1029
Open Reason / Close Reason
The open reason and close reason detail why a new sub-session (represented by a newline within the SDR) has been opened. A value between 1 and 6 is used to detail the
reason, in accordance with the table below:
VALUE DEFINITION
1 START Session radius start message received. The first sub-session recordof a session will always have an open reason of Start.
2 START_REPORT You will only see this value if Allot Customer Support have set the“Session Report on Start” status configuration to "enable". In thiscase, a record with open reason Start is created. The close reasonfor such records is Start_Report. Start_Report will then also be
used as the open reason of the following sub-session.
3 SP_CHANGE A new SP is assigned to the session.
4 INFO_CHANGE Any session information change received from Radius. Used as bothclose and open reason.
5 SCHEDULE You will only see this value if Allot Customer Support have
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 88/113
Data Mediator Installation & Administration Guide9-10
VALUE DEFINITION
changed the value of the “Sub-session lifetime” field. By default theconfiguration is 24hr. When the duration of a sub-session reaches
the configured time, the sub-session is closed by the SMP with aSchedule record type and a new sub-session is opened. This is usedas both close and open reason.
6 CLOSE Session radius stop message received.
IMEISV Number
The IMEISV number is extracted from the RADIUS message. IMEISV contains 15digits and has the following format: AA-BBBBBB-CCCCCC-D
AA: These two digits are for the Reporting Body Identifier, indicating the GSMA
approved group that allocated the TAC (Type Allocation Code).
BBBBBB: The remainder of the TAC
CCCCCC: Serial sequence of the Model D: Luhn check digit of the entire model or 0 (This is an algorithm that validates
the ID number)
Output Example
Figure 9-4: SDR .csv file example
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 89/113
Data Mediator Installation & Administration Guide 10-1
Chapter 10: Usage Detail Records (UDR)
NOTE UDRs are automatically enabled if SDRs are enabled. If SDRs are
not being generated by one or more SMPs, then UDRs cannot begenerated.
UDRs collect usage information per session as well as usage information perapplication within a session.
UDR information is extracted from the in-line traffic and is compiled on the in-line platform (e.g: SG-Sigma). UDRs are created on a periodical basis (1hr by default. Tochange the interval, contact support@allot.com) and count the sub-session uplink and
downlink volumes of traffic. When not interrupted by session changes, these records arealigned by the hour (e.g: 12.00, 13.00, 14.00 etc.)
UDRs are exported in CSV file format. Every complete transaction is represented by a
record in the CSV file.
The CSV files are pushed compressed in GZIP format from the Data Mediator to theexternal system frequently (every 1-2mins).
The Allot system can be configured to purge the files after a pre-defined period of time(default is 24 hours).
NOTE A UDR is generated for each monitored application configured, inaddition to a UDR with total usage for that session.
Allot’s in-line platforms use proprietary “monitoring rules” tocount the number of bytes used per subscriber for eachconfigured application and for the total bytes used by the
subscriber’s session if configured to do so. The in -line platformwill use a single monitoring rule per each application (and for thesession counted bytes if configured to do so) defined for UDRcollection.
Allot supports a total of 13 monitoring rules per subscriber. EveryUDR monitoring rule is coupled by an SDR monitoring rule.Therefore the total number of applications which can bemonitored per subscriber is 6.
Monitoring Rules are also consumed when implementing Gxbased volume reporting, an online charging policy, an offlinecharging policy and service flow detection (event based triggers.
Configuring UDRs on the Data Mediator1. In the Navigation pane, select and right-click the Network in the
Navigation tree and select Configuration from the popup menu.
OR
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 90/113
Data Mediator Installation & Administration Guide10-2
Select the Network in the Navigation tree and then select Configuration fromthe View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
The Configuration window for the Network is displayed.
2. Select the Data Mediation tab.
3. To configure UDRs, open the Data Definition tab and select theUDR sub-tab.
Figure 10-1: UDR Data Definition Properties
4. In the UDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file. The default is 100,000 bytes (100MB) and the maximum possible value is 1,000,000,000 bytes (1GB).
File Closing Interval (in seconds) – This defines how long after
being created the export file is closed. The default is 300seconds (5 minutes) and the maximum is 3600 seconds (1 hour).
File Compression method – This sets the file compressionmethod to be used when exporting the file. The possible options
are GZIP and no compression. GZIP is selected by default.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 91/113
Data Mediator Installation & Administration Guide 10-3
Delete files older than (in minutes) – This defines the amount oftime the system retains files. The default is 1440 minutes (24
hours) and the maximum is 4320 minutes (72 hours).
Push Properties – Push properties, once enabled, define how theexport files are sent to their proper locations for analysis andstorage. The four push method options are SFTP, FTP, SCP and
Copy. If SFTP is selected, the Username, Password, Server andPath must be defined. If SCP is selected, the username and
password must be defined. For FTP and Copy, only the pathmust be defined.
Monitored applications – See below.
Monitored Applications
The Monitored Applications list is where applications-based UDRs are configured. Theapplications to be included in the UDR must be selected. In order for an application to
be selected, it must have already been added to the Charging Applications catalog.
You may add Monitored Applications to the list by clicking on the Edit button.
Figure 10-2: Change Monitored Applications dialog
When the Change Monitored Applications dialog appears, use the Arrow buttons tomove applications from the Available Applications list to the Selected Applications list.Only those applications that have previously been added to the Charging Applicationscatalog will appear in the Available Applications list.
Click OK to save and return to the UDR tab. Those applications you have selected nowappear in the Monitored Applications list.
Modification of the Selected Applications list while the system is running will affect
only new sessions. Statistics for sessions that started prior to the modification willcontinue reporting according to the previous Selected Applications list.
NOTE A UDR is generated for each monitored application configured, inaddition to a UDR with total usage for that session. The Sessionapplication is created automatically and cannot be removed.
For more information on Charging Applications, see the NetXplorer Operations Guide.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 92/113
Data Mediator Installation & Administration Guide10-4
Naming ConventionsUDR files are generated according to the following naming conventions:
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Datatype>_<Version>.[<FE>]
FIELD NAME DESCRIPTION NOTES
< Source unit name > The name of the source unit thatgenerated the file
Each name must beunique
<Source file IP> The IP Address of the file origin.
<Mediation IP> The IP address of the Mediation unitthat created the file
<Seq> A sequencer that is never reset.
<Date> Creation date of the file in the formyyyymmddhhmmss
<Data type> Represent the file content:
“UDR” – for files containing UDRrecords
<Version> Version of the entry format.
Format: V<format number>
The version is set from theconfiguration file where the requiredoutput structure is defined.
<FE> File extension.
The file extension is csv.gz for UDRfiles.
While the file is being copied a “.tmp”suffix will be added. This suffix will
be removed after the copy has beencompleted successfully.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 93/113
Data Mediator Installation & Administration Guide 10-5
UDR Interface Fields
ATTRIBUTE NAME DESCRIPTION FORMAT
Subscriber ID Unique subscriber identifier in
subscriber management
environment as received from
SMP.
Zero if deployment is without
SMP.
Value is external identifier (in
mobile networks) or internal
identifier (in Fixed networks).
Numeric String
Session Key Unique identifier of the
subscriber session. The sameSessionKey is used for the
Subscriber session in HDRs,
SDRs and UDRs.
Present in subscriber
management environment only.
If no SMP is in the Network, this
value appears as 0.
Numeric String
Sub-Session Key Unique sub-session identifier per
subscriber session. Sub session is
created if there is a change in the
service plan of the subscriberduring the session.
Present in subscriber
management environment only
Numeric String
Monitored Application Name The name of the monitored
application as defined by the
end-user in the application
groups catalogs
String (in quotes)
Start Time Start of the time interval for
which the statistics measures are
given (e.g. Octets in)
Date String
End Time End of the time interval for
which the statistics measures are
given (e.g. Octets in)
Date String
Octets in Byte count of incoming traffic
counted by the DPI engine.
Numeric String
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 94/113
Data Mediator Installation & Administration Guide10-6
ATTRIBUTE NAME DESCRIPTION FORMAT
Octets outByte count of outgoing traffic
counted by the DPI engine. Numeric (D)
Output Example
Figure 10-3: UDR .csv file example
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 95/113
Data Mediator Installation & Administration Guide 11-1
Chapter 11: Binary Data Records (BDR)
Enabling and Configuring BDRs on the In-LinePlatform
To enable BDR identification and data records generation, use the following procedure:
1. Connect to the in-line platform, as sysadmin
2. Create a directory somewhere called WAConfig
3. Copy the configuration files into that directory. i.e.cp /opt/allot/conf/WebAnalytics/* ./WAConfig
4. Edit your LOCAL ./WAConfig/server.cfg
5. Change the relevant parameters as in the following example:
#
# Pilot Packet configuration file
#
####################################################
# Connection Tuple
# protocol list: 0-TCP / 1-UDP / 2-ANY
# port: 0-65535 (0 for ANY)
# IP: a valid ipv6 address (::ffff:0.0.0.0 for ANY)
####################################################
intIp="::ffff:127.0.0.1"
intPort=0
extIp="::ffff:255.255.255.255"
extPort=8080
protocol=1
####################################################
# DPI Parameters
####################################################
# establish the connection: 0-Parsed / 1-Established
shouldEstablish=0
# DPI connection timeout optimization ( 0 - disabled / 1 - enabled )
special_timeout=1
# predefined servers mode ( 0 - disabled / 1 - enabled )
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 96/113
Data Mediator Installation & Administration Guide11-2
predefined_servers_enable=1
###################################################
# special tuple parameters
###################################################
behaviour=0
###################################################
# WebAnalytics
###################################################
hdr_enable=1
gcdr_enable=1
Legend gcdr_enable – Enable(1) or disable(0) BDRs.
IMPORTANT The default configuration is for BDR to be disabled. Forcustomers using the BDR feature, this flag HAS to be changed to1.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 97/113
Data Mediator Installation & Administration Guide 11-3
operation_mode – 1 is enabled, 0 is disabled. This disables theentire list of servers.
extIp / intIp – One of these fields is mandatory. The other isoptional. Therefore, there should be at least one IP address
provided here.
intPort , extPort – Optional.
protocol – Indicates whether it's TCP (0) or UDP (1) or ANY (2).
For BDRs you should use UDP, 1 , as in the example above.
shouldEstablish – Must be 0 for BDRs.
appId – Must be 407 , which is the AppID for "Other IM
Applications" which was altered on the previous instructions.
hdr_enable – Not relevant, this feature is controlled via the
NetXplorer GUI.
special_timeout – 1 = Enabled, 0 = disabled. This enables the useof HDR duration and a real byte count for timeout purposes. After
editing this parameter, restart the DataSrv (using keeperMgr -SW -
R DataSrv) or reboot the host blade.
IMPORTANT Use 0 if HDR duration or a real byte count fields are not required.
6. After changing the configuration, please apply using the followingcommand:
/opt/allot/bin/wa-update.sh <WACONFIG DIRECTORY>
Output Example
# /opt/allot/bin/wa-update.sh /home/sysadmin/WAConfig
Creating a tar file...
genericcdr.xml
webanalytics.xml
webanalytics.xsd
Updating CCs...
Done!
7. This will copy the files in the directory into/opt/allot/conf/WebAnalytics and apply the configuration.
Configuring BDR Export on the NetXplorer1. To enable BDR configuration from the NetXplorer GUI, contact
Allot Customer Support.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 98/113
Data Mediator Installation & Administration Guide11-4
2. Once BDRs have been enabled in the NetXplorer GUI, select andright-click the Network in the Navigation tree and selectConfiguration from the popup menu.
ORSelect the Network in the Navigation tree and then select Configuration fromthe View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
The Configuration window for the Network is displayed.
3. Select the Data Mediation tab.
4. To configure BDRs, open the Data Definition tab and select theBDR sub-tab.
Figure 11-1: BDR Data Definition Properties
5. In the BDR sub-tab, the following fields may be configured.
File Size Limit (in bytes) – This defines the maximum possiblesize of each exported file before compression. The default is
100,000 bytes (100 MB) and the maximum possible value is1,000,000,000 bytes (1 GB).
File Closing Interval (in seconds) – This defines how long after being created the export file is closed. The default is 300
seconds (5 minutes) and the maximum is 3600 seconds (1 hour).
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 99/113
Data Mediator Installation & Administration Guide 11-5
File Compression method – This sets the file compressionmethod to be used when exporting the file. The possible options
are GZIP and no compression. GZIP is selected by default.
Delete files older than (in minutes) – This defines the amount oftime the system retains files. The default is 1440 minutes (24hours) and the maximum is 4320 minutes (72 hours).
Push Properties – Push properties, once enabled, define how the
export files are sent to their proper locations for analysis andstorage. The four push method options are SFTP, FTP, SCP andCopy. If SFTP is selected, the Username, Password, Server and
Path must be defined. If SCP is selected, the username and password must be defined. For FTP and Copy, only the pathmust be defined. The default is Enabled.
Naming ConventionsBDR files are generated according to the following naming conventions:
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Datatype>_<Version>.[<FE>]
FIELD NAME DESCRIPTION NOTES
< Source unit name > The name of the source unit thatgenerated the file
Each name must beunique
<Source file IP> The IP Address of the file origin.
<Mediation IP> The IP address of the Mediation unitthat created the file
<Seq> A sequencer that is never reset.
<Date> Creation date of the file in the formyyyymmddhhmmss
<Data type> Represent the file content:
“B40” – for files containing BDRrecords
<Version> Version of the entry format.
Format: V<format number>
The current version is V1
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 100/113
Data Mediator Installation & Administration Guide11-6
FIELD NAME DESCRIPTION NOTES
<FE> File extension.
The file extension is bin.gz for BDRfiles.
While the file is being copied a “.tmp”suffix will be added. This suffix will
be removed after the copy has been
completed successfully.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 101/113
Data Mediator Installation & Administration Guide 12-1
Chapter 12: SNMP Interface
Data Mediator TrapsThe Data Mediator sends 5 different types of traps to the Service Provider’s NOC.
NAME DESCRIPTION OID
alDmCopyToFailure This trap is sent whenever a file push
failed. The alarm is raised provided that
a configured interval has passed since
the last time a copytofailure alarm was
raised towards the same target
1.3.6.1.4.1.2603.12.4.1
alDmRecordRateUp This trap is sent if the rate of CDRs
processed per second passes the
configured threshold for each interval.
1.3.6.1.4.1.2603.12.4.2
alDmRecordRateDown This trap is sent if the rate of CDRs processed per second returns to below
the configured threshold for each
interval.
1.3.6.1.4.1.2603.12.4.3
alDmCopytoFailureDown This trap is sent if there was a failure to
push a file to the target host.
The trap will include the failed target
host.
1.3.6.1.4.1.2603.12.4.4
alDNGetFileFailureUp This trap is sent if the “get file”
operation from the in-line platform
failed. The trap will include the relevant
in-line platform and the relevant
filename.
1.3.6.1.4.1.2603.12.4.5
Data Mediator KPIsThe Data Mediator can be polled by an external SNMP server for the purpose of healthmonitoring.
Using the standard HOST-RESOURCES MIB, you can poll each appliance for healthmonitoring and key performance indicators.
The standard LINUX Host-Resources MIB has the OID of 1.3.6.1.2.1.25. It can be polled for information about:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 102/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 103/113
Data Mediator Installation & Administration Guide 12-3
Output RecordsNAME DESCRIPTION OID
alDmKpiOutputFilesCreatedNum The number of files
created by the datamediator since the last
boot
1.3.6.1.4.1.2603.12.3.1.1.1.6
alDmKpiOutputFilesPushSuccess
Num
The number of filessuccessfully pushed by
the data mediator to theoperator’s SFTP serversince the last boot
1.3.6.1.4.1.2603.12.3.1.1.1.7
alDmKpiOutputFilesPushFailNum The attempts to pushfiles by the data
mediator to the
operator’s SFTP serversince the last boot whichfalied (note: this is notthe number of files
which have not been pushed)
1.3.6.1.4.1.2603.12.3.1.1.1.8
alDmKpiOutputFilesPendingPush Files to be pushed – thetotal number of files
waiting to be pushed.
1.3.6.1.4.1.2603.12.3.1.1.1.10
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 104/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 105/113
Data Mediator Installation & Administration Guide 13-1
Chapter 13: Appendices
Appendix A: Data Mediator Logs
Datamediation.log – a log providing all system messages at twoconfigurable levels, info and debug (debug mode will printadditional information in the log).
App-datamediation.log – a user oriented log, prints messages at 3levels:
failure (logging all failures to push to target or pull from
source)
push (logging those files successfully pushed)
all (logging all files successfully pushed, pulled or createdon the Data Mediator)
An entry in the App-datamediation.log contains the followinginformation:
Name or address of the source unit
Name or address of the target unit
File name
Transfer start date and time
Transfer end date and time
File size in Mbytes
Number of records in file - for stored or pushed files
Status – success or failure
Reason – description of cause of failure
Appendix B: Upgrade Procedure
Use the following procedures to upgrade your Data Mediator devices:
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 106/113
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 107/113
Data Mediator Installation & Administration Guide 13-3
6. On both nodes make sure that there are no DB processes running by using the following command:
ps ax |grep -v grep |grep dbsrv -c
The output should be 0.
7. On both nodes make sure no storage is mounted by entering thefollowing command:
mount |grep dm-1 –c
The output should be 0.
8. On DM-0 run the following command:
mount /dev/dm-1 /opt/sybase/data
9. On DM-0 run the following command:
rpm -Uvh <filename>.rpm
Where < filename> is the name of the new MD installation package stored in a temp folder on the Data Mediator/
10. On DM-1 run the following command:
rpm -Uvh <filename>.rpm
Where < filename> is the name of the new MD installation package stored in a temp folder on the Data Mediator/
11. Reboot both nodes.
Appendix C: Customizing Data ExportIn some circumstances you may wish to customize the format of the data exported.Allot provides direct access to the configuration file to enable you to do this via thefollowing process.
1. In the Navigation pane, select and right-click the Network in the Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration fromthe View menu.
OR
Select the Network in the Navigation tree and then click the Configuration icon on the toolbar.
The Configuration window for the Network is displayed.
2. Select the General tab.
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 108/113
Data Mediator Installation & Administration Guide13-4
3. The current DataExportConfig.xml file is displayed. This filecontrols what fields are included in the exported CDR for each of
the data types. If no custom file has been created, a default file isloaded and displayed here.
4. To customize the contents of the export files, you may compose anew version of the DataExportConfig.xml file following thetemplate shown below. Use the Interface fields as listed for each
CDR type in the following chapters. Click Upload and browse tothe location of the new DataExportConfig.xml to load it.
NOTE The DataExportConfig.xml cannot be edited from the NetXplorerGUI, it is only displayed for reference.
DataExportConfig.xml Template
<DataExportConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="./DataExportConfig.xsd">
<DataType name="HDR">
<FilteringRules defaultAction="include">
<Rule action="exclude">
<Condition type="ContentType">image</Condition>
<Condition type="ContentType">text</Condition>
<Condition type="Host">www.google.com</Condition>
</Rule>
<Rule action="exclude">
<Condition type="Host">www.ynet.co.il</Condition>
<Condition type="ContentType">application/x-javascript</Condition>
</Rule>
</FilteringRules>
<OutputStructure version="1">
<!-- Incremented when structure is modified -->
<Field ID="2" Type="RequestHeader" Name="Host"MaxLength="70"/>
<Field ID="3" Type="StartTime"Format="Numeric/DateTime/String/IPV4/IPV6"/>
<Field ID="4" Type="SubscriberID"Obfuscate="None/Hash/Encrypt"/>
<Field ID="50" Type="ClientIP"/>
<Field ID="51" Type="ClientPort"/>
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 109/113
Data Mediator Installation & Administration Guide 13-5
<!-- Field ID is global to the entire file and unique.
The same ID cannot appear twice in the file, unless it is representing theexact same field. In addition, new fields should always be defined asincremented IDs, to allow backward compatibility -->
</OutputStructure>
</DataType>
<DataType name="SDR">
<FilteringRules defaultAction="include"/>
<OutputStructure version="1">
<Field ID="50" Type="ClientIP"/>
<Field ID="51" Type="ClientPort"/>
<Field ID="67" Type="CustomAttr1" RadiusAttr=”26”
RadiusVendorID=”2603” RadiusVendorAttr=”8”/>
</OutputStructure>
</DataType>
</DataExportConfig>
HDR XML Format
NOTE It is recommended that the OutputStructure version value of thesection be increased by one before saving if any changes weremade to this section.
- <DataType name="HDR "> <FilteringRules defaultAction="include" /> - <OutputStructure version="4"> <Field ID="2" Type="EnforcerID" /> <Field ID="1" Type="StartTime" /> <Field ID="3" Type="SubscriberID" /> <Field ID="4" Type="SessionKey" /> <Field ID="5" Type="ClientIP" /> <Field ID="6" Type="ClientPort" /> <Field ID="7" Type="ServerIP" /> <Field ID="8" Type="ServerPort" /> <Field ID="9" Type="ServiceID" /> <Field ID="10" Type="HTTPMethod " MaxLength="10" /> <Field ID="11" Type="RequestHeader" Name="Host" MaxLength="70" /> <Field ID="12" Type="URI" MaxLength="150" />
<Field ID="13" Type="RequestHeader" Name="User-Agent" MaxLength="150" /> <Field ID="14" Type="RequestHeader" Name="Referer" MaxLength="150" /> <Field ID="15" Type="ResponseHeader" Name="Content-Type" MaxLength="50" /> <Field ID="16" Type="DownloadContentLength" /> <Field ID="17" Type="UploadContentLength" /> <Field ID="18" Type="ResponseCode" MaxLength="20" /> <Field ID="19" Type="ServerInitialResponseTime" /> <Field ID="20" Type="Duration" /> <Field ID="21" Type="RequestActualByteCount" />
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 110/113
Data Mediator Installation & Administration Guide13-6
<Field ID="22" Type="ResponseActualByteCount" /> <Field ID="23" Type="RequestHeader" Name="DNT,x-do-not-track" MaxLength="1"
/> </OutputStructure> </DataType>
UDR XML Format
NOTE It is recommended that the OutputStructure version value of thesection be increased by one before saving if any changes weremade to this section.
- <DataType name="UDR "> <FilteringRules defaultAction="include" />
- <OutputStructure version="1"> <Field ID="3" Type="SubscriberID" /> <Field ID="4" Type="SessionKey" /> <Field ID="39" Type="SubSessionKey" />
<Field ID="40" Type="ChargingApplicationID" /> <Field ID="1" Type="StartTime" /> <Field ID="59" Type="EndTime" /> <Field ID="32" Type="OctetsIn" /> <Field ID="33" Type="OctetsOut" />
</OutputStructure> </DataType>
MOU XML Format
NOTE It is recommended that the OutputStructure version value of thesection be increased by one before saving if any changes weremade to this section.
- <DataType name=" MOU"> <FilteringRules defaultAction="include" />
- <OutputStructure version="1"> <Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="45" Type="ServicePlanID" /> <Field ID="9" Type="ServiceID" /> <Field ID="58" Type="SessionDuration" />
</OutputStructure>
VC XML Format
NOTE It is recommended that the OutputStructure version value of the
section be increased by one before saving if any changes weremade to this section.
<DataType name=" VC"> <FilteringRules defaultAction="include" />
- <OutputStructure version="1"> <Field ID="2" Type="EnforcerID" />
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 111/113
Data Mediator Installation & Administration Guide 13-7
<Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="25" Type="LineID" /> <Field ID="26" Type="PipeID" />
<Field ID="27" Type="
VCID" /> <Field ID="28" Type="InstanceType" />
<Field ID="29" Type="InstanceKey1" /> <Field ID="30" Type="InstanceKey2" /> <Field ID="31" Type=" NetUnitID" /> <Field ID="32" Type="LiveConnections" /> <Field ID="33" Type="DroppedConnections" /> <Field ID="34" Type=" NewConnections" /> <Field ID="37" Type="OctetsIn" /> <Field ID="38" Type="OctetsOut" /> </OutputStructure> </DataType>
Conversations XML FormatNOTE It is recommended that the OutputStructure version value of the
section be increased by one before saving if any changes weremade to this section.
- <DataType name="CONV "> <FilteringRules defaultAction="include" />
- <OutputStructure version="1"> <Field ID="2" Type="EnforcerID" /> <Field ID="23" Type="PeriodStartTime" /> <Field ID="24" Type="PeriodEndTime" /> <Field ID="5" Type="ClientIP" /> <Field ID="7" Type="ServerIP" />
<Field ID="9" Type="ServiceID" /> <Field ID="25" Type="LineID" /> <Field ID="26" Type="PipeID" /> <Field ID="27" Type=" VCID" /> <Field ID="28" Type="InstanceType" /> <Field ID="29" Type="InstanceKey1" /> <Field ID="30" Type="InstanceKey2" /> <Field ID="31" Type=" NetUnitID" /> <Field ID="32" Type="LiveConnections" /> <Field ID="34" Type=" NewConnections" /> <Field ID="37" Type="OctetsIn" /> <Field ID="38" Type="OctetsOut" />
</OutputStructure> </DataType>
SDR XML Format
NOTE It is recommended that the OutputStructure version value of thesection be increased by one before saving if any changes weremade to this section.
- <DataType name="SDR ">
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 112/113
Data Mediator Installation & Administration Guide13-8
<FilteringRules defaultAction="include" /> - <OutputStructure version="1"> <Field ID="41" Type="EnforcerDBID" /> <Field ID="42" Type="RecordCreationTimestamp" /> <Field ID="3" Type="SubscriberID" />
<Field ID="4" Type="SessionKey" /> <Field ID="39" Type="SubSessionKey" /> <Field ID="43" Type="SubSessionStartTime" /> <Field ID="44" Type="SubSessionEndTime" /> <Field ID="45" Type="ServicePlanID" /> <Field ID="46" Type="OpenReason" /> <Field ID="47" Type="CloseReason" /> <Field ID="48" Type="SessionIMEI" /> <Field ID="49" Type="SessionSGSN" /> <Field ID="50" Type="SessionTAC" /> <Field ID="51" Type="SessionRAT" /> <Field ID="52" Type="SessionIPV4" /> <Field ID="53" Type="SessionIPV6" /> <Field ID="54" Type="SessionAPN" /> <Field ID="55" Type="SessionLocation" />
<Field ID="56" Type="SessionMCC" /> </OutputStructure> </DataType>
Using Obfuscation
Obfuscation can be defined for any attribute in DataExportConfig.xml by editing and
uploading the file. The function will be executed as the last function after any othertransformation function that is defined for the attribute. As an example, Subscriber IDwould first be transformed from binary to a string, and then would be obfuscated (if sorequested).
Two obfuscation methods are supported:
Hash – irreversible, using SHA-1 algorithm
Encrypt – reversible, using AES 128 bit algorithm
Example
In this example, the HDR section of the DataExportConfig.xml file has been edited todefine with obfuscation for the SubscriberID and URI attributes, appearing in red:
- <DataType name="HDR"><FilteringRules defaultAction="include" />
- <OutputStructure version="4"><Field ID="2" Type="EnforcerID" /><Field ID="1" Type="StartTime" /><Field ID="3" Type="SubscriberID" Obfuscate="Hash" /><Field ID="4" Type="SessionKey" /><Field ID="5" Type="ClientIP" /><Field ID="6" Type="ClientPort" /><Field ID="7" Type="ServerIP" /><Field ID="8" Type="ServerPort" />
8/13/2019 Data Mediator Install and Admin Guide R2 (13.1.10)
http://slidepdf.com/reader/full/data-mediator-install-and-admin-guide-r2-13110 113/113
<Field ID="9" Type="ServiceID" /><Field ID="10" Type="HTTPMethod" MaxLength="10" /><Field ID="11" Type="RequestHeader" Name="Host" MaxLength="70" /><Field ID="12" Type="URI" MaxLength="150" Obfuscate="Encrypt" /><Field ID="13" Type="RequestHeader" Name="User-Agent" MaxLength="150"
/><Field ID="14" Type="RequestHeader" Name="Referer" MaxLength="150" /><Field ID="15" Type="ResponseHeader" Name="Content-Type"
MaxLength="50" /><Field ID="16" Type="DownloadContentLength" /><Field ID="17" Type="UploadContentLength" /><Field ID="18" Type="ResponseCode" MaxLength="20" /><Field ID="19" Type="ServerInitialResponseTime" /><Field ID="20" Type="Duration" /><Field ID="21" Type="RequestActualByteCount" /><Field ID="22" Type="ResponseActualByteCount" /><Field ID="23" Type="RequestHeader" Name="DNT,x-do-not-track"
MaxLength="1" /></OutputStructure></DataType>