Data Mediator

Installation and Administration Guide

Data Mediator Installation and Administration Guide

Version History

Doc Version Product Date Updates

r16b2 MD16.5 10/06/20 Section 2.1, Managing

Passwords Added

r16b1 MD16.5 30/04/20 GA Release

IPv6 Support added
throughout
netmenu updated
(section 3.1.1)
HDR generation on
Blocked Traffic added
(Section 4.1.1)

r8b3 MED16.1.50 24/11/19 SR630 Servers added

to Appendix H,
Procedure for changing
NX to DM
communication to
HTTPS added.

r8b2 MED16.1.50 30/09/19 SR630 Servers added

to Section 2, BMC
information added in
Appendix B. HDR
Generation on Blocked
Traffic added to
Section 4.

r8b1 MED16.1 30/05/19 Configuring the Data

Mediator for Data in
Motion Encryption
(GDPR) added.

r7b6 MED15.1 22/01/19 Rebranded, Interface

Tables removed,
general updates. Smart
Data Source added.
SNMP Interface moved
to SNMP Guide.

r7b5 MED15.1 02/08/18 General Updates

r7b4 MED15.1 30/01/18 Virtual installation

updated

r7b1 MED15.1 07/11/17 GA

Data Mediator Installation and Administration Guide ii

Data Mediator Installation and Administration Guide

Data Mediator Installation and Administration Guide iii

Data Mediator Installation and Administration Guide

Contents
1 Overview........................................................................................................................ 1-1
1.1 Hardware .......................................................................................................... 1-2
1.2 Data Types ........................................................................................................ 1-3
Aggregation vs. Event Records ............................................................ 1-4
Record Export Values .......................................................................... 1-6
DM Performance Rates per Hour ........................................................ 1-7
1.3 Data Flow – System Level ................................................................................. 1-8
1.4 Data Flow – Data Mediator............................................................................... 1-9
1.5 Output File Structure ...................................................................................... 1-11
1.6 Glossary........................................................................................................... 1-12
2 Deploying Data Mediator .............................................................................................. 2-1
2.1 Managing Passwords ........................................................................................ 2-1
SSH Admin Password ........................................................................... 2-1
SSH Root Password .............................................................................. 2-1
BMC Password ..................................................................................... 2-2
IMM Password..................................................................................... 2-3
2.2 Physical Deployment ........................................................................................ 2-5
Physically Connecting to the Network ................................................ 2-5
2.3 Virtual Deployment......................................................................................... 2-11
Pre-Requisites.................................................................................... 2-11
Installing the Virtual DM Template ................................................... 2-13
3 Initial Configuration ....................................................................................................... 3-1
3.1 Hardware Configuration ................................................................................... 3-1
Configuring Network Parameters ....................................................... 3-1
Configuring Device Parameters ........................................................... 3-3
Configuring Server Management Parameters .................................... 3-4
Rebooting the Data Mediator ........................................................... 3-10
3.2 High Availability Configuration ....................................................................... 3-10
Pre-Installation Requirements .......................................................... 3-11
Installing the HA Software ................................................................. 3-11
Verify a Successful Installation .......................................................... 3-13

Data Mediator Installation and Administration Guide iv

Data Mediator Installation and Administration Guide

3.3 Enabling and Configuring Data Mediation...................................................... 3-14

Enabling Subscriber Data Export License .......................................... 3-14
Enabling Distributed Collection ......................................................... 3-14
CMTS Awareness ............................................................................... 3-18
Adding a Data Mediator .................................................................... 3-27
Configuring the Data Mediator for Data in Motion Encryption (GDPR) 3-
29
Enabling HTTPS Communication with the NetXplorer ...................... 3-31
4 Enabling CDR Types ....................................................................................................... 4-1
4.1 Enabling HDRs per In-line Platform .................................................................. 4-1
HDR Generation on Blocked Traffic .................................................... 4-2
4.2 Enabling VDRs per In-line Platform .................................................................. 4-3
4.3 Enabling MOU per In-Line Platform.................................................................. 4-4
4.4 Enabling and Configuring Conversations .......................................................... 4-5
Enabling Conversations Records per In-line Platform......................... 4-5
Prioritizing Conversations Statistics .................................................... 4-5
4.5 Enabling Real-Time Reports (CONV-RTS) per NetXplorer ................................ 4-8
4.6 Enabling SDRs per SMP ................................................................................... 4-10
4.7 Enabling HTTP Reports per NetXplorer .......................................................... 4-12
5 Smart Data Source ......................................................................................................... 5-1
5.1 Output Records ................................................................................................. 5-2
Input Record Filtering Rules ................................................................ 5-2
Creating/Editing Output Records ........................................................ 5-5
5.2 Output Profiles ................................................................................................ 5-16
Creating/Editing/Duplicating Output Profiles ................................... 5-17
5.3 Default Profiles ............................................................................................... 5-20
Allot Default Profile ........................................................................... 5-20
Allot Default ClearSee Metrics Profile ............................................... 5-21
Allot Default ClearSee Analytics Profile ............................................ 5-22
Allot Default ClearSee Real Time Profile ........................................... 5-23
5.4 Global Output Trigger ..................................................................................... 5-23
5.5 Input Definitions ............................................................................................. 5-25
Configuring HDR Filtering Rules ........................................................ 5-25

Data Mediator Installation and Administration Guide v

Data Mediator Installation and Administration Guide

Configuring VDR Filtering Rules......................................................... 5-27

Configuring the SDR SubSession Lifetime ......................................... 5-29
Configuring UDR Monitored Applications ......................................... 5-30
Configuring CMDR Time Intervals ..................................................... 5-32
6 Appendices .................................................................................................................... 6-1
6.1 Appendix A – Binary Data Records (BDR) ......................................................... 6-1
Enabling and Configuring BDRs on the In-Line Platform..................... 6-1
Configuring BDR Export on the NetXplorer ......................................... 6-3
Naming Conventions ........................................................................... 6-4
6.2 Appendix B - Using the BMC ............................................................................. 6-6
Home ................................................................................................... 6-6
BMC Configuration .............................................................................. 6-7
6.3 Appendix C – Using the IMM ............................................................................ 6-7
System Status ...................................................................................... 6-9
Server Management .......................................................................... 6-10
IMM Management ............................................................................ 6-10
6.4 Appendix D – Server Memory Upgrades ........................................................ 6-11
6.5 Appendix E - Data Mediator Logs ................................................................... 6-12
6.6 Appendix F – RTT Working Modes.................................................................. 6-13
Setting the Working Mode ................................................................ 6-13
6.7 Appendix G - File Locations............................................................................. 6-14
6.8 Appendix H - Server Hardware Specifications ................................................ 6-15
P/N PA-MED-SRV ............................................................................... 6-15
P/N PA-MED-SRV-DC ......................................................................... 6-15
P/N PA-MED-SRV-HAP ....................................................................... 6-15
P/N PA-MED-SRV-GEN5..................................................................... 6-16
P/N PA-MED-SRV-HAP-GEN5 ............................................................ 6-16

Data Mediator Installation and Administration Guide vi

1 Overview
In addition to its powerful core functionality, the Allot solution elements can serve
as a comprehensive data source. Working together with the Data Mediator, data
can be compiled, collected, mediated and exported to external systems for the
purpose of storage or analysis.
Various types of statistics and detail records are generated by the Allot solution.
The data must be collected periodically from the Data Source (Allot In-Line
Platforms or Subscriber Management Platforms), transformed into a suitable
format and then pushed to target devices that will process or store the data
according to the needs of each individual customer.
The Allot Data Mediator is an infrastructure component that provides:
• Single and unified infrastructure for data mediation between producers
and consumers, where consumers may be internal or external to the
system
• Shorter implementation time to support collection, transformation and
transfer of new data types
• Faster compliance with customer needs by supporting pluggable
adapters without modifying generic infrastructure
Data Mediator Installation and Administration Guide

1.1 Hardware
The Allot Data Mediator is a 1U high appliance based on an IBM M4 or M3 server.
The appliance is shipped from Allot with the appropriate Linux based CentOS
operating system and Allot software already installed.
The Allot Data Mediator is available in the following configurations:
• PA-MED-SRV-GEN6 or PA-MED-SRV (AC standalone)
• PA-MED-SRV-HAP (AC HAP)
• PA-MED-SRV-HAP-DC (DC HAP)
A Data Mediator High Availability cluster is made up of two AC or DC Data
Mediators along with a Storage device. The Storage device supports the same
power supply (AC or DC) as the Data Mediators it is clustered with.

Data Mediator Installation and Administration Guide 1-2

Data Mediator Installation and Administration Guide

1.2 Data Types

Allot in-line platforms, together with the Allot Data Mediator, serve as a data
source for the following data types:
• Conversation: Bandwidth per connection, service. Customization of
preferred and excluded application
• Session (SDR/UDR): Bandwidth per subscribers. Customization of
filtering rules
• HDR: Web activity usage per subscriber. Define Application to monitor
• VC: Bandwidth per policy element
• MOU: VOIP usage per service plan and service
• BDR: Binary raw data
• VDR: Video data
• CMDR: Cable Modem data
• CMBM: CMTS BG Channel Map data
• CMCS: CMTS Channel Statistics
• HTTP: Web Traffic Information
• CONV-RTS: Generated on demand when a user requests a Real Time
report for a given subscriber/instance.
• CONV-RTU: Constantly generated, giving information per static policy
(no subscribers) with drill down to applications/ASN.
• WSP Buckets: Data generated concerning users/subscriber activity by
WebSafe Personal
This data can be used to generate many different types of statistics, including:
1. Usage statistics - based on Conversation records extracted from in-line
traffic per flow, Allot can provide 360 view of network, subscriber, and
app usage for monitoring and planning
2. Mobile statistics - based on Mobile session records (SDR/UDR) extracted
from RADIUS from Allot SMP, Allot can provide statistics on Mobile
handset usage useful for new handset introduction and monitoring as
well as monitor Mobile network load e.g. by analyzing session bitrates
3. VoIP statistics – based on MOU records, Allot can monitor the leakage of
Voice calls to OTT VoIP in minutes

Data Mediator Installation and Administration Guide 1-3

Data Mediator Installation and Administration Guide
4. Web statistics – based on HTTP transaction level records extracted from
HTTP traffic (covering also SSL), Allot can provide Web traffic analysis
and subscriber profiling and trends and allow its customer to generate
new revenue streams
5. Policy statistics – based on VC records extracted from in line classified
traffic providing app usage per the policy elements managed by the
system (subscribers or service plans) for example provide subscriber
service plan quota usage, for service plan tuning and capacity planning
6. Video statistics – based on video records (VDR) extracted from in-line
traffic per watch, Allot can provide high granularity full coverage usage
and QoE KPIs for the #1 OTT application.
7. Cable statistics – based on CMTS session records (CMDR/CMCS/CMBR)
extracted from start stop message, Allot can provide statistics on CMTS
host, MAC domain interface, channel/bonding and DOCSIS version usage
Allot data feeds are delivered with minimum impact on DPI performance.
Records include a unique identifier which allows association between the
different statistical record types for creating a single enriched record.

Aggregation vs. Event Records

All data types produce records which are comprised of parameters and counters.
There are two basic kinds of records:
• Aggregation Records, in which the trigger for release is time bound (30
seconds or 300 seconds), and the parameters serve as keys for
aggregation of the counters. Aggregation records are produced by
CONV, VC and MOU data types.
• Event Records, in which the trigger for release is an event and there is
no aggregation of the counters. Event based rcords are produced by
SDR. HDR, HDR, VDR and CMDR data types.

Data Mediator Installation and Administration Guide 1-4

Data Mediator Installation and Administration Guide
Aggregation Records

Above is an example of an Aggregation record of the CONV data type.

1. A conversation is started between a client and a server for a service
which triggers the creation of a record.
2. The keys (ClientIP, ServerIP, ServiceID) identify the record.
3. The record is created after aggregating the Metrics (OctetsIn, OctetsOut)
for 5 minutes

Event Records

Above is an example of an Event record of the HDR data type.

1. A client sends a request for a Web page and many responses are
returned. Each response triggers the creation of an Event-drive record
2. The fields (ClientIP, URI, ServiceID, Request.Header.Host) describe the
event.

Data Mediator Installation and Administration Guide 1-5

Data Mediator Installation and Administration Guide
3. Metrics (RequestActualByteCount, ResponseActualByteCount) may
appear but are not aggregated.

Record Export Values

The following table lists how many records of each type can be exported on the
device (NetEnforcer/Service Gateway) itself. It is assumed that each device is
running the most recent GA version.
For SG-Tera, SG-Sigma and SG-Sigma E Service Gateways, the following values are
per Core Controller.
Note: 1 Data Mediator supports up to 20 CC blades.
PER
SIGMA E SG-Tera
INTERVAL AC1400 AC3000 AC6000 SIGMA
(CC-300) (CC-400)
(SECONDS)
CC per NE 1 1 1 4 10 10
CPU per CC 1 1 1 1 2 2

UDR 300 180,000 180,000 360,000 480,000 250,000 750,000

HDR 1 1,000 1,000 7,500 2,500 10,000 20,000

VDR 1 160 160 480 600 600 600

VC 300 125,000 125,000 400,000 400,000 1,000,000 A total of
8,000,000
divided
Conversation 300 125,000 125,000 400,000 400,000 1,000,000 between
VC and
Conv.
MOU 300 480,000 480,000 480,000 480,000 480,000 480,000

Data Mediator Installation and Administration Guide 1-6

Data Mediator Installation and Administration Guide

SG-VE SG-VE
PER INTERVAL
(Large (XSmall SG-9100 SG-9500 SSG600 SSG800
(SECONDS)
Template) Template)
CC per NE 1 1 1 1 1 1
CPU per CC 1 1 1 1 1 1

UDR 1 2500 300 2500 5000 100 300

HDR 1 20,000 2400 20,000 40,000 9,600 16,000
VC 300 A total of A total of A total of A total of
A total of A total of
4,000,000 8,000,000 800,000 2,400,000
960,000 8,000,000
divided divided divided divided
divided divided
Conversation 300 between between between between
between VC between VC
VC and VC and VC and VC and
and Conv. and Conv.
Conv. Conv. Conv. Conv.
MOU 1 66 66 66 66 66 66

PER INTERVAL
SG-9008 SG-9700 SSG-500 SSG-200
(SECONDS)
CC per NE 1 1 1 1
CPU per CC

UDR 1 600 16,500 600 600

HDR 1 4,800 130,000 4,800 4,800
VC 300 A total of A total of A total of A total of
2,000,000 53,000,000 2,000,000 2,000,000
divided divided divided divided
Conversation 300 between VC between VC between VC between VC
and Conv. and Conv. and Conv. and Conv.
MOU 1 66 66 66 66

DM Performance Rates per Hour

The table below provides performance numbers in two working modes – with
and without 1 hour aggregation for the conversation records.

Records Per Hour Records Per Hour

Without Aggregation With Aggregation

UDR 360M 180M

Data Mediator Installation and Administration Guide 1-7
Data Mediator Installation and Administration Guide

Records Per Hour Records Per Hour

Without Aggregation With Aggregation

HDR 720M 360M

VC* 288M 144M
Conversation 360M 180M
SDR 22M 11M
MOU 400K 200K
* The In-line platform limits the number of exported VC’s to the # of supported
VC in the distributed collector (4M per 5 Minutes).
The Aggregation value affects all exported records.

1.3 Data Flow – System Level

Figure 1-1: System Level Data Flow

• Step 1- The RADIUS server performs the AAA functionality for

the new subscriber session.

• Step 2 – The RADIUS Server passes the RADIUS accounting

message to the SMP.

• Step 3 – The SMP extracts relevant subscriber information from

the RADIUS message and passes this on to the In-line Platform.
It also generates SDR buckets which aggregate this information
on the SMP.

Data Mediator Installation and Administration Guide 1-8

Data Mediator Installation and Administration Guide
• Step 4a – The subscriber is granted access to the Internet and
subscriber traffic flows through the In-line platform. The In-line
platform records the relevant subscriber data (in a series of
different data buckets) and…

• Step 4b – …forwards the subscriber’s traffic to the internet

according.

• Step 5 – The Data Mediator polls the various data sources (in
this case the In-line Platform and the SMP) for the configured
data buckets,

• Step 6 – The Data Mediator formats the data and sends it on to

the Data Warehouse or third-party analytics system.

1.4 Data Flow – Data Mediator

The Data Mediator performs three key functions:

• Collects data from in-line platforms

• Transforms the data to an output format

• Pushes the data to the required targets

2 Purge
(6)
Transformer
Collector (3)

1
Transformed Push (5)
File List Output (4)

Data Data
Source Data Mediator Target

Data Mediator Installation and Administration Guide 1-9

Data Mediator Installation and Administration Guide

Figure 1-2: Data Mediator Data Flow

The Data Mediator collects data from a list of configured Allot data sources.
Periodically, the Data Mediator retrieves (1) the list of potential files to collect from
each data source (in-line platform or SMP)
NOTE It is the responsibility of the Data Mediator to identify the
files that have not been processed yet, and to avoid
duplicate processing of files.
New files are retrieved by the Collect (2) function, and depending on the type of
the collected data, a suitable Transform (3) function is executed. The Data
Mediator supports different transform adapters per data type to facilitate
changing adapters without making any change to the infrastructure.
Transformations are either of the following types:

• Simple assignments from input to output

• Data type conversions – binary to textual (e.g. date, time, IP

address)
A transformed file is stored in the Data Mediator storage (4).
The system supports a push mechanism towards the Target consumers. Therefore,
files are pushed by the Push (5) function.

• At the completion of the life cycle, transformed files are purged

from the Data Mediator storage by the Purge (6) function,
according to the retention policy for the data type.

Data Mediator Installation and Administration Guide 1-10

Data Mediator Installation and Administration Guide

1.5 Output File Structure

Data Mediator output files are in CSV format. Every output string (including
dictionary lookups such as service name and date formatting) is surrounded with
quotations. Obfuscation results (both hash and encrypt), null values, zeroes, empty
strings, IP, numeric formatting and -1 (null resolution or no input value) are not
marked with quotations.
Obfuscation results (both hash and encrypt) are not marked with “” in the output
CSV. are also not marked with “”.
Commas separating attributes in the CSV file are not surrounded by spaces.
The Data Mediator performs the following adaptations to the records:
• Precedes every “ with an escape character (by default, inverted
commas).
• Precedes an escape character with a second escape character.
• Replaces with a space every character not in range.
The escape character can be set via the Configuration CLI using the following
command:
-setDmProfileEscapeChar -profileName <NAME> -escapeChar <CHAR>
Possible CHAR values are:
' \ / ` % | * ? < > ~, ; ! # $
Exclusions in syntax for these delimiter characters:
DELIMITER SYNTAX
* */
\ \”\
| \”|
> \”>
< \”<
“ default

Data Mediator Installation and Administration Guide 1-11

Data Mediator Installation and Administration Guide

1.6 Glossary
TERM DEFINITION
BDR Binary Detail Records
CDR Call Data Record, a generic term used to indicate any of the
seven data types translated by the Data Mediator
CMBM CMTS BG Channel Map. CMBM provide statistics concerning
traffic over Cable Modems that is compiled via a CMTS
connection by the SMP. It includes information concerning
congestion and usage on CMTS bonding groups, which
consist of multiple channels.
CMCS CMTS Channel Statistics. CMCS provide statistics concerning
traffic over Cable Modems that is compiled via a CMTS
connection by the SMP. It includes information concerning
channel usage and congestion. Each channel consists of a
single RF signal.
CMDR Cable Modem Data Record. This information concerns traffic
over Cable Modems that is compiled via a CMTS connection
by the SMP. It includes information concerning congestion
and usage distribution as well as Subscriber information.
Conversations (CONV) Individual connections. Conversations CDR information
includes data regarding individual connections on the
network. Depending on the implementation, Conversation
statistics may provide policy enforced application or
application group volumes based on the service plan
definitions for traffic that needs to be shaped, blocked,
steered or TOS-marked. These statistics are further broken
down into a DPI-identified service name. They are extracted
from the in-line traffic and are compiled on the in-line
platform (e.g: SG-Sigma).
CONV-RTS Conversations – Real Time Solicited
CONV-RTU Conversations – Real Time Unsolicited
Data Sources Can be in-line platforms (such as the SG-Sigma or
NetEnforcer) and/or the SMP.
DM Data Mediator (Also known as MED)
HDR HTTP Detail Records (HDR) are enhanced HTTP CDRs which
record HTTP transaction-level attributes, as well as
HTTPS/SPDY session-level attributes obtained from the
TLS/SSL handshake (i.e. destination host). The HTTPS/SPDY
records can be sent to ClearSee for analytics purposes.

Data Mediator Installation and Administration Guide 1-12

Data Mediator Installation and Administration Guide

TERM DEFINITION
HTTP
In-line Platform Allot NetEnforcers or Service Gateways
MOU Minutes of Use. MOU CDRs present data for the minutes of
use for different over the top VoIP applications running
through the network.
NX NetXplorer
Pilot Packet See BDR
SDR Session Detail Record. SDRs are extracted from the AAA
function (e.g: RADIUS) and are compiled on the SMP Server.
An SDR is created for every segment of a session (called a
sub-session) on a periodic basis (20m by default. To change
the interval, contact [email protected]). A session is
broken into a sub-session whenever there is a change in the
session data (e.g: switch of service plan) or by time interval.
SMP Subscriber Management Platform
UDR Usage Detail Record. UDRs collect usage information per
session as well as usage information per application within a
session. UDR information is extracted from the in-line traffic
and is compiled on the in-line platform (e.g: SG-Sigma).
UDRs are created on a periodical basis (1hr by default. To
change the interval, contact [email protected]) and count
the sub-session uplink and downlink volumes of traffic.
When not interrupted by session changes, these records are
aligned by the hour (e.g: 12.00, 13.00, 14.00 etc.)
VC Virtual Channel. VC CDR information concerns traffic that is
classified into different rules in the Allot system. The data is
based on the service plan and the subscriber. Depending on
the implementation, VC statistics may provide policy
enforced application or application group volumes based on
the service plan definitions for traffic that needs to be
shaped, blocked, steered or TOS-marked. They are extracted
from the in-line traffic and are compiled on the in-line
platform (e.g: SG-Sigma).
VDR Allot Video Data Records (VDR) contain information on a
video session and can include data on only part of the video
session to allow analysis of certain events within the video
session (e.g. hourly volume, volume per video resolution).
Video data transport can be partitioned into pieces and/or
use different transport protocols as configured by the user
via the NetXplorer GUI.

Data Mediator Installation and Administration Guide 1-13

2 Deploying Data Mediator
2.1 Managing Passwords
In order to facilitate installation and initial configuration, Allot provides default
values for all required passwords.
It is ESSENTIAL for security that these default passwords be changed AS SOON AS
POSSIBLE. In this section each default password is listed, along with instructions on
how to change it.

SSH Admin Password

Allot provides end-users with SSH access to the system via a user privilege called
“admin”.
Note: Allot STRONGLY recommends that the default passwords are changed to
ensure a minimum level of security.
• User Name: admin
• Default Password: allot

To change the SSH Admin password:

1. Log into the system via SSH.
2. Enter admin for the login and the admin password and then press
<Enter>.
3. Enter passwd and then press <Enter>.
4. You will be asked to enter the current password and click <Enter>
5. When prompted enter a new password and press <Enter>. The password
must be between 5 and 8 characters. You can use a combination of upper
and lower case letters and numbers.
6. Re-enter the new password and press <Enter>.

SSH Root Password

The SSH Root password is required for certain actions in the CLI and gives complete
access to the system. Therefore it should only be given to trusted users.
The default values are as follows:
NOTE Customers are strongly advised to change default passwords on first login. Not
doing so represents a security risk.
• User Name: root
• Default Password: bagabu
 Deploying Data Mediator
Changing the Root Password
1. Access the Server using the SSH Admin log in and password.
2. Switch to SSH Root user with the following command:
su -
3. Enter the root password, and then press <Enter>.
4. Enter passwd and then press <Enter>.
5. Enter a new password and press <Enter>. The password must be between
5 and 8 characters. You can use a combination of upper and lower case
letters and numbers.
6. Re-enter the new password and press <Enter>.

BMC Password
The BMC is the Server Management Software used on Lenovo servers and is
provided with all new Servers from Allot. It is a way to access and manage the DM
Server remotely.
NOTE Customers are strongly advised to change default passwords on first login. Not
doing so represents a security risk.
The default details of the BMC are:
• Username: USERID
• Password: Password10
• IP: 10.4.4.4
• Subnet: 255.255.0.0
• Default GW: 10.4.0.1
It is possible that in some units the previous default user name and password may
be required. These were as follows:
• Previous Default IP: 192.168.70.125
• Previous Default User Name: USERID
• Previous Default Password: PASSW0RD (where the “0” is not “o” but
“zero”)
To change the BMC password, follow the steps below:

Data Mediator Installation and Administration Guide 2-2

 Deploying Data Mediator

Figure 2-1: BMC Initial Screen

1. Bring up or reboot the server.
2. Log into the BMC using the default or current password.
3. From the Lenovo XClarity Administrator side bar, select BMC
Configuration, and then User/LDAP.
4. Open the Local Users tab.
5. Click the Edit icon next to USERID (or the relevant user).
6. In the User Credentials and Authority area, enter a new Password in the
field, then enter it again in Confirm Password.
7. Click Apply.

IMM Password
Some older units (made by IBM) may continue to use the IMM Server management
system in place of the BMC.
NOTE Customers are strongly advised to change default passwords on first login. Not
doing so represents a security risk.
The default details of the IMM are:
• Default IP: 192.168.70.125
• Default User Name: USERID
• Default Password: PASSW0RD (where the “0” is not “o” but “zero”)
To change the IMM password, follow the steps below:
1. Bring up or reboot the server.
Data Mediator Installation and Administration Guide 2-3
 Deploying Data Mediator
2. Log into the IMM using the default or current password.
3. From the IMM menu bar, select IMM Configuration, and then User
Accounts.
4. Open the Local Accounts tab.

Figure 2-2: IMM User Accounts

5. Click USERID (or the relevant user name) to open the User Properties
screen.
6. In the User Credentials area, enter a new Password in the field, then enter
it again in Confirm Password.
7. Click Apply.

Data Mediator Installation and Administration Guide 2-4

 Deploying Data Mediator

Figure 2-3: IMM User Credentials

2.2 Physical Deployment

Physically Connecting to the Network
Connecting a Single Data Mediator
To connect a single Data Mediator to the network, follow the instructions below:
• Connect a straight copper cable from eth1 illustrated below on the rear of
the Data Mediator to a management switch.
• Connect a straight copper cable from the BCM or IMM port illustrated
below on the rear of the Data Mediator to a laptop for direct connectivity
and maintenance (in blue).
• Optionally, you may also connect an additional straight copper cable from
the adjacent port (eth2) to a secondary management switch for backup
purposes.

Data Mediator Installation and Administration Guide 2-5

 Deploying Data Mediator

Figure 2-4: Connecting a Single Data Mediator (SR630 Hardware)

Figure 2-5: Connecting a Single Data Mediator (M5 Hardware)

Connecting an HA Data Mediator Cluster

IBM SR630 Servers

To connect the HA DM Cluster, based on SR630 hardware (2 x DM and an IBM
V5010 storage device with dual controllers), follow the instructions below:

HA DM Cluster with V5010 Storage

Data Mediator Installation and Administration Guide 2-6

 Deploying Data Mediator

Figure 2-6: HA Cluster Network Connections (SR630 and V5010 Hardware)

1. A label is affixed to the top of the V5010 Storage Unit that indicates the
serial number of the supplied DM that must be used as the Primary DM.
Use this information when connecting your DM-HAP system.
2. Use two crossed copper cable to connect between eth3 and eth4 on each
DM to the same port on the second server (illustrated in green above).
This connection serves the pacemaker communication process and its
redundant backup respectively.
3. Use a Serial Attached SCSI (SAS) cable to connect between each DM and
the RAID storage server. These connections and the exact ports required
are illustrated in orange above. The cables connect to the DM via Micro-
SAS connectors.
4. Connect each DM to the management network via eth1 (shown in green)
with an additional link via eth2.
5. Additional management cables should be connected to the IMM module
on each DM for direct connectivity and maintenance (in green).
An additional optional management cable may be connected to the RAID storage
server for storage management and storage traps.

Data Mediator Installation and Administration Guide 2-7

 Deploying Data Mediator
IBM M5 Servers
To connect the HA DM Cluster, based on Gen-5 hardware (2 x DM and an IBM
V5010 storage device with dual controllers), follow the instructions below:

HA DM Cluster with V5010 Storage

Figure 2-7: HA Cluster Network Connections (M5 and V5010 Hardware)

1. A label is affixed to the top of the V5010 Storage Unit that indicates the
serial number of the supplied DM that must be used as the Primary DM.
Use this information when connecting your DM-HAP system.

Data Mediator Installation and Administration Guide 2-8

 Deploying Data Mediator

Figure 2-8: Label on Storage Unit

2. Use two crossed copper cable to connect between eth2 and eth3 on each
DM to the same port on the second server (illustrated in green above).
This connection serves the pacemaker communication process and its
backup respectively.
3. Use a Serial Attached SCSI (SAS) cable to connect between each DM and
the RAID storage server. These connections and the exact ports required
are illustrated in orange above. The cables connect to the DM via Micro-
SAS connectors.
4. Connect each DM to the management network via eth0 (shown in blue)
with an additional link via eth1.
5. Additional management cables should be connected to the IMM module
on each DM for direct connectivity and maintenance (in blue).
An additional optional management cable may be connected to the RAID storage
server for storage management and storage traps.

HA DM Cluster with V3700 Storage

Data Mediator Installation and Administration Guide 2-9

 Deploying Data Mediator

Figure 2-9: HA Cluster Network Connections (M5 and V3700 Hardware)

1. A label is affixed to the top of the V3700 Storage Unit that indicates the
serial number of the supplied DM that must be used as the Primary DM.
Use this information when connecting your DM-HAP system.

Figure 2-10: Label on Storage Unit

2. Use two crossed copper cable to connect between eth2 and eth3 on each
DM to the same port on the second server (illustrated in green above).

Data Mediator Installation and Administration Guide 2-10

 Deploying Data Mediator
This connection serves the pacemaker communication process and its
backup respectively.
3. Use a Serial Attached SCSI (SAS) cable to connect between each DM and
the RAID storage server. These connections and the exact ports required
are illustrated in orange above. The cables connect to the DM via Micro-
SAS connectors.
4. Connect each DM to the management network via eth0 (shown in blue)
with an additional link via eth1.
5. Additional management cables should be connected to the IMM module
on each DM for direct connectivity and maintenance (in blue).
An additional optional management cable may be connected to the RAID storage
server for storage management and storage traps.

2.3 Virtual Deployment

Allot’s Data Mediator can be purchased and delivered as a virtual appliance file
(templates) with the relevant products pre-installed and configured.
Allot Virtual NMS (vNMS) components are provided with two types of template:
• OVF template for deployment in VMWare/ESXi environment
• QCOW2 template for deployment in OpenStack/KVM environment
The template files for each product may be configured to support both larger and
smaller scale environments.

Pre-Requisites
HYPERVISOR

VMWare ESXi VMWare vShpere 5.5 and above

RedHat KVM RedHat RHEL 6.7 and above

OPERATING SYSTEM

CentOS Linux CentOS 6.7, 64-bit x86

Following are the specifications for the DM template that Allot provides. The
template is pre-installed with the relevant product and is configured to support the
workload as specified below.

Data Mediator Installation and Administration Guide 2-11

 Deploying Data Mediator

TEMPLATE VDISK: TEMPLATE VCPU: TEMPLATE RAM: TEMPLATE NICS:

150 GB for OS and 32x Up to 64 GB 1 Network adapter
DM Software
1.1 TB for Data

Data Mediator Installation and Administration Guide 2-12

 Deploying Data Mediator
Installing the Virtual DM Template
For full instructions on installing a Virtual Data Mediator on VMWare or KVM, see
see Allot’s Deployment guides, available on the Allot Knowledge Base.

Data Mediator Installation and Administration Guide 2-13

3 Initial Configuration
3.1 Hardware Configuration
Irrespective of whether you are working with basic Data Mediator architecture or a
high availability Data Mediator cluster, each Data Mediator server that you are
working with must be prepared for use. Four initial steps are required
1. Configure network parameters
2. Configure device parameters
3. Configure IMM parameters
4. Reboot the Mediator

Configuring Network Parameters

The following network parameters need to be assigned to each of the Data
Mediators you are using:
• Data Mediator IP and mask
• Data Mediator Gateway IP for network access
• DNS Server IP
• Host name
• NTP Time Server IP
Note: The default gateway must be configured. If not configured, in some
circumstances, you may not be able to add the SMP to the NetXplorer.

CentOS 7 and later

Connect a keyboard and monitor to the front panel of the DM as shown below.

Figure 3-1: Connecting the Data Mediator (SR630) – Front View

 Initial Configuration
1. Run netmenu.sh using the following command and enter the appropriate
network information when prompted:
/root/netmenu.sh

Figure 3-2: netmenu.sh

NOTE From netmenu.sh you can choose to use IPv4, IPv6 or DualStack (both) for all
connections.
2. Reboot the server.

CentOS 6.4
Connect a keyboard and monitor to the front panel of the DM as shown below.

Figure 3-3: Connecting the Data Mediator (M5) – Front View

1. Copy the netwconf.sh script to the root directory of the server (if it is not
there already), run it using the following command and enter the

Data Mediator Installation and Administration Guide 3-2

 Initial Configuration
appropriate network information when prompted:
/root/netwconf.sh
Output Example
[root@localhost ~]# /root/netwconf.sh
Please type the IP ADDRESS [ 11.11.11.11 ]
10.4.3.65
Please type NETMASK [ 255.255.0.0 ]

Please type the GATEWAY [ 11.11.0.1 ]

10.4.0.1
Please type hostname [ localhost ]
Server1
Please type domain name [ ]
mydomain.com
Please type ip address of DNS [ 198.168.254.2 ]
8.8.8.8

Please check the values entered

The host: Server1 10.4.3.65
NETMASK: 255.255.0.0, DOMAIN: mydomain.com
GATEWAY: 10.4.0.1
DNS: 8.8.8.8
Continue with these values (y/n) [y]?
Y
Please type ip address of additional DNS or press Enter to
continue:
8.8.4.4
Please type ip address of additional DNS or press Enter to
continue:
194.90.1.5
Please type ip address of additional DNS or press Enter to
continue:

Restarting network service...

Done.
2. Reboot the server.

Configuring Device Parameters

Once you have installed the MD package and configured the network parameters
of the Data Mediator, you need to configure the appropriate device parameters in
order to inform the Data Mediator server what functional elements should be
enabled.
The dev-setup command informs the Data Mediator what its identity is. You can
see the list of available parameters by running the command “dev_setup.sh”
without parameters.

Data Mediator Installation and Administration Guide 3-3

 Initial Configuration
Note: If a Data Mediator has been shipped directly from Allot, this process has
already been done and should be skipped.

Usage
dev_setup.sh [ -v ] [ -m MODE ] [ -f ] [ -a ]

-v View device setup properties and license details

-m <datamediation> To set Device mode
-f Force change of mode and key
-a High availability mode and key

The proper MODE to use in the command to configure a Data Mediator is

datamediation. Once that is set, reboot the Data Mediator.
Once the device is set to datamediation you need to open the NetXplorer GUI to
assign the Data Mediator to that NetXplorer.

Example
dev_setup.sh -m datamediation
Setting device mode to datamediation...
Request completed successfully.
Device mode successfully changed to datamediation.
Please reboot your device.

Configuring Server Management Parameters

SR630 Server (CentOS v7.x and later)
NOTE Customers are strongly advised to change default passwords on first login. Not
doing so represents a security risk.
To configure the network settings of the BMC, follow the steps below:
1. Connect directly from a laptop to the BMC interface on the rear of the
Data Mediator.

Data Mediator Installation and Administration Guide 3-4

 Initial Configuration

Figure 3-4: BMC Port

2. Open a web browser. In the address field type the IP address or host name
of the BMC to which you want to connect.

Figure 3-5: BMC Login

3. Enter User ID and Password
4. You will see the BMC User Interface, with the default “Home” in view, as
seen in below:

Data Mediator Installation and Administration Guide 3-5

 Initial Configuration

Figure 3-6: BMC Home Screen

5. Select BMC Configuration and then “Network” from the drop down list on
the left side of the screen.
6. In the Ethernet section, make sure that IPv4 is enabled, and IPv6 DHCP is
disabled.
7. In the IPv4 section, make sure that the Configure IP address settings field
is set to: “Use Static IP Address”. Assign an IP, mask and default gateway
as seen above and click Apply. You can now access the BMC remotely
using these network settings.

Data Mediator Installation and Administration Guide 3-6

 Initial Configuration

Figure 3-7: BMC Network

M5 Server (CentOS v7.x and later)

The default details of the IMM are as follows:
• Default IP: 192.168.70.125
• Default User Name: USERID
• Default Password: PASSW0RD (where the “0” is not “o” but “zero”)
To configure the network settings of the Integrated Management Module, follow
the steps below:
1. Connect directly from a laptop to the IMM interface on the rear of the
Data Mediator. The interface is labeled “SYSTEM MGMT” as shown in
Figure 3-8 below

Data Mediator Installation and Administration Guide 3-7

 Initial Configuration

Figure 3-8: IMM "System Management" Port

2. Open a web browser. In the address field type the IP address or host name
of the IMM to which you want to connect.
Note: If you are logging in to the IMM for the first time after installation, the IMM
defaults to DHCP. If a DHCP host is unavailable, it uses the default static IP
address 192.168.70.125. You can obtain the DHCP-assigned IP address or the
static IP address from the server BIOS or from your network administrator.

Figure 3-9: IMM Login

3. Enter User ID and Password
4. You will be prompted to specify an inactive session timeout value. Choose
a value from the dropdown list and click on Continue.
5. You will see the IMM User Interface, with the default “System Status” in
view, as seen in below:

Data Mediator Installation and Administration Guide 3-8

 Initial Configuration

Figure 3-10: IMM System Status Screen

6. Select “Network” from the IMM Management drop down list on the left
side of the screen.
7. In the Ethernet section, make sure that IPv4 is enabled, and IPv6 DHCP is
disabled. In addition, DDNS status should be set to “Disabled” and
“Domain Name Used” should be set to “manual”.
8. In the IPv4 section, make sure that the Configure IP address settings field
is set to: “Use Static IP configuration”. Assign an IP, mask and default
gateway as seen above and click Save. You can now access the IMM
remotely using these network settings.

Data Mediator Installation and Administration Guide 3-9

 Initial Configuration

Figure 3-11: IMM Network

Rebooting the Data Mediator

To reboot the Data Mediator server, log in as a root user and enter: reboot
Request completed successfully
Broadcast message from root (Sun Mar 1
16:59:18 2009):

3.2 High Availability Configuration

The High Availability (HA) solution for Allot’s Data Mediator Platform is based on
using two installations of Data Mediator for high availability applications and a
shared storage unit for Sybase database files.

Data Mediator Installation and Administration Guide 3-10

 Initial Configuration
Pre-Installation Requirements
Before starting a “high availability” installation, you will need to ensure that both
the Data Mediator servers which you will be using have the same Data Mediator
software version.
Note: Make sure that both DMs are configured with the dev-setup “-a ” flag for high
availability i.e:
dev-setup.sh –m datamediator –a
• Confirm that the same software version is installed on both Data Mediator
units by checking the install log on each Data Mediator server unit, located
at /opt/allot/conf/install_log. You may also run the command dctype.
• Confirm that Network Parameters (IPs, Gateway, DNS, Hostname, etc.) are
defined on both machines and that both nodes have the same subnet and
DG.
• Decide which Data Mediator unit will be designated as DM-0 (initially
active unit) and as DM-1 (initially passive-standby unit). To do this, use a
logical role designation and host name.

Installing the HA Software

CentOS 7 and later
Note: Only a root user can install the HA (High availability) package.
The following procedure should be performed on BOTH nodes.
1. Locate the Allot_HA7_Setup.sh script which should be found in the
/opt/allot/conf/ha directory.
2. Run Allot_HA7_Setup.sh on the Secondary node. Answer the questions
according to your configuration.
For example:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allot High availability for ACP 17.x Setup menu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is MD device, Please answer the following questions
Is this a Primary Machine (y/n) [y]? n
Is this In-Memory Machine (y/n) [y]? y
Do you want to Install or Upgrade (i/u) [i]? i
Does this Machine have External Storage (y/n) [y]? y
(next screen)
+------------------------------------------------------+
| Secondary |
| HOST Name = smpha2-101-231.allot.com |
| GATEWAY = 10.150.0.1 |
| VIP = |
| In-Memory = y eno1-+ |BOND0-Management

Data Mediator Installation and Administration Guide 3-11

 Initial Configuration
| External-Storage = n |-->|10.150.101.231
| Node2 = 10.150.101.231 eno33559296-+ |255.255.0.0
| DNS = 172.18.1.10 |
| |
| eno50338560-+ |BOND1-Cluster
| |-->|192.168.168.2
| eno67109888-+ |255.255.255.0
| |
| |
| -+ |BOND2-Data
| |-->|
| -+ |
| |
+------------------------------------------------------+

# I/F MAC Address IP Address NETMASK Link

~~~ ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~
#) eno1 : 00:0c:29:60:82:6b bond0 255.255.0.0 yes
#) eno33559296 : 00:0c:29:60:82:75 bond0 255.255.0.0 no
#) eno50338560 : 00:0c:29:60:82:7f bond1 255.255.255.0 yes
#) eno67109888 : 00:0c:29:60:82:89 bond1 255.255.255.0 no
#) eno83889152 : 00:0c:29:60:82:93 bond2 255.255.255.0 no
#) eno100668416 : 00:0c:29:60:82:9d bond2 255.255.255.0 yes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

HA Configuration :
~~~~~~~~~~~~~~~~~~
1) Bonding
BOND0/MGT : MGT1=eno1 MGT2=eno33559296
BOND1/Cluster: CLT1=eno50338560 CLT2=eno67109888
BOND2/Data : DAT1= DAT2=
~~~ ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~
2) Continue HA Setup
3) Disable HA and restore SA
4) Quit

3. Start with menu option 1) Bonding. Follow the interactive questions.

4. Select the appropriate Interfaces for Bond0 and define the node IP &
Mask.
5. Select the appropriate Interfaces for Bond1 & Bond2 (in case of in-
memory).
6. When done, return to the main menu, continue with option 2) Continue
HA Setup.
7. When done, continue on Primary node.
8. On the Primary node Run Allot_HA7_Setup.sh. Answer the questions
according to your configuration. Define the Management VIP of the MD
HA.
For example:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allot High availability for ACP 17.x Setup menu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is MD device, Please answer the following questions
Is this a Primary Machine (y/n) [y]? y

Data Mediator Installation and Administration Guide 3-12

 Initial Configuration
Is this In-Memory Machine (y/n) [y]? y
Do you want to Install or Upgrade (i/u) [i]? i
Does this Machine have External Storage (y/n) [y]? y
Please type the IP address of the VIP [] : 10.150.101.232
Please type the Management IP address of the Secondary host [] : 10.150.101.231
9. Continue with menu option #1 (Bonding) follow by option #2 (Continue HA
Setup)
10. When asked please provide the root password.

CentOS 6.4 and later

Note: Only a root user can install the HA (High availability) package.
The following procedure should be performed on BOTH nodes.
1. Locate the HA6x.sh script which should be found in the
/opt/allot/conf/ha directory.
2. On both nodes run the following command:
HA6x.sh -m
3. On both nodes run the following command:
HA6x.sh -i
Note: When prompted, enter the proper network parameters and check the data
before proceeding.
4. On the primary node only run the following command:
HA6x.sh -p
Note: Ignore any warnings at this stage of the configuration.
5. After all of the steps above have been completed, the to complete the
process, run the following command the primary node only:
HA6x.sh –c

Verify a Successful Installation

After running the hainstall script, there are several checks that you can run to
verify a successful installation.
• Verifying the creation of a cluster using the display file system command
df-h
• Verify network parameters for the cluster using ifconfig –a
• Check the High Availability status by entering the crm_mon command.
• Confirm storage connectivity by entering the cat/proc/partitions
command.

Data Mediator Installation and Administration Guide 3-13

 Initial Configuration

3.3 Enabling and Configuring Data Mediation

The process of configuring Subscriber Data Export consists of 4 distinct stages:
1. Enabling Subscriber Data Export license via activation key
2. Enabling Distributed Collection on Multi-bladed Service Gateways
3. Enabling and configuring CMTS Awareness (Optional)
4. Adding a Data Mediator
Each of these stages is examined in detail below.

Enabling Subscriber Data Export License

In order for the Subscriber Data Export feature to work, the operator must first
have purchased the correct license.
There are two different licenses for subscriber data export functionality:
• Statistics Export License for Conv, VC, MOU, SDR, UDR export
• HTTP CDR License for HTTP CDR and BDR export

Enabling Distributed Collection

In multi-blade In-line Platforms Allot has two modes of data collection:
• Centralized Collection: where data is collected on each Core Controller
blade and transferred to the Host Blade where it is stored. The host blade
maintains a list of each data file that is stored.
• Distributed Collection: where data is collected on each Core Controller
blade and stored there. The host blade maintains a central list of each
data file, and the core controller informs the host blade which files should
be added or removed from the list and on which blade each one is stored.
Distributed collection must be used with the Data Mediator. In order to enable
distributed collection you will need to assign IP addresses from the management
network to each Core Controller and SFB in the system and then enable direct
access to them. The procedures for doing this are detailed below.

To set the IPs for CC blades in slots 1-5 and 10-14 in two steps:
Enter the following commands:
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots
1:5

Data Mediator Installation and Administration Guide 3-14

 Initial Configuration
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots
10:14
In the commands above x.x.x.x:255.255.0.0 represents the IP:Subnet Mask while
the y.y.y.y represents the Default Gateway. The –slots value 1:5 means that the slot
number will increment up from 1 to 5, while 10:14 means the slot number will
increment up from 10 to 14.
Using these commands will cause the IP address (x.x.x.x in the example) to
increment up as follows:
• CC n will get IP address x.x.x.x
• CC n+1 will get IP address x.x.x.x+1
• etc.

To set the IP for each CC blade individually:

1. Enter the following commands for the relevant slots:
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 1:1
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 2:2
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 3:3
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 4:4
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 5:5
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots
10:10
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots
11:11
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots
12:12
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots
13:13
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots
14:14
2. To view the configuration, type the following command:
sysadmin@EXC-SBH[1/6]:~$ go config view blade_mngt_ips

To set the IP for each SFB blade individually:

1. Enter the following commands for the relevant slots:
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 6:6
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 7:7

Data Mediator Installation and Administration Guide 3-15

 Initial Configuration
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 8:8
go config blade_mngt_ips -base_ip x.x.x.x:255.255.0.0 -g y.y.y.y -slots 9:9
Output Example
==== Blade Management IPs ====
|Slot |IP |Mask |Gateway
--------------------------------------------------------------
|4 |10.150.6.34 |255.255.0.0 |10.150.0.1
--------------------------------------------------------------
Request completed successfully.

To enable Distributed Collection in the NetXplorer:

1. In the Navigation pane, select and right-click the desired In-line Platform
in the Navigation tree and select Configuration from the popup menu.
OR
Select the desired In-line Platform in the Navigation tree and then select
Configuration from the View menu.
OR
Select the desired In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
The Configuration window for the selected NetEnforcer or Service
Gateway is displayed.
2. Select the IP Properties tab.
3. In the Direct Access pane, check the Management IP per CC checkbox.

Data Mediator Installation and Administration Guide 3-16

 Initial Configuration

Figure 3-12: IP Properties tab

Data Mediator Installation and Administration Guide 3-17

 Initial Configuration
CMTS Awareness
CMTS awareness provides data records of CM subscribers and channel/bonding
group utilization to ClearSee analytics users, or those who use third-party analytics
tools. These data records include CMDR, CMCS and CMBM buckets.
• CMDR (Cable Modem Details Records) - CMDR provide a large range of
CMTS statistics received from the SMP and extracted from start and stop
messages. CMDR is generated once every 20 minutes, on session end and
on service plan change. Optionally, CMDR can be set to be generated on
session start.
• CMCS (CMTS Channel Statistics) – CMCS provide statistics concerning
traffic over Cable Modems that is compiled via a CMTS connection by the
SMP. It includes information concerning channel usage and congestion.
Each channel consists of a single RF signal.
• CMBM (CMTS BG Channel Map) – CMBM provide statistics concerning
traffic over Cable Modems that is compiled via a CMTS connection by the
SMP. It includes information concerning congestion and usage on CMTS
bonding groups, which consist of multiple channels.
With these records, the user can also analyze channels/bonding groups utilization
and provide drill downs in different dimensions (applications, subscribers).
In this section the following steps in configuring and using CMTS Awareness are
described:
• Enabling and Configuring CMTS on the SMP
• Loading CMTS Lists
• Enabling and Configuring CMTS Awareness
• CMTS Interface Status Reports

Enabling and Configuring CMTS on the SMP

Note: In order to make use of the CMTS Awareness feature, an SMP must be
installed in SMF mode. An SMP working in SMF mode will list Subscriber
Mapping as the SMP Type in the Properties dialog.

Enabling CMTS
1. Right click on an appropriate SMP in the Navigation Pane of the
NetXplorer GUI and select Properties.

Data Mediator Installation and Administration Guide 3-18

 Initial Configuration

2. Select the Enable downstream wideband interfaces management

checkbox.
3. Click Save.

Configuring DHCP
1. Open an SSH session to the SMP and open the following file:
/opt/allot/config/DhcpConfig.xml
2. Enter the following values:
<DhcpServiceEnabled value="true"/>
<DhcpV4Support value="true"/>
<InlineGleanerEnabled value="true"/>
<LocalGleanerEnabled value="true"/>
<NeInline ip="InlinePlatform IP" port="5558" />
3. Save the DhcpConfig.xml file.
4. Restart the smprouter process by entering the following command:
KeeperMgr –R smprouter

Data Mediator Installation and Administration Guide 3-19

 Initial Configuration
Configuring IPDR Stream
The solution requires IPDRs to be sent from the CMTS to the SMP. IPDRs may be
either pushed from the SMP or pulled by the DM. This may be configured using the
following procedure:
1. Open an SSH session to the SMP and open the CmtsConfig.xml file.
2. Find the CmtsIpdrCollectorEnabled parameter and set its value. If it is set
as True then the SMP will push IPDRs to the DM (Streaming method). If set
as False then the DM will pull IPDRs from the SMP (Poll Method).

Output Example
<AdminParameters>
<CmtsIpdrCollectorEnabled value="false"/>

Changing the IPDR Destination Folder (Optional)

By default the IPDR destination folder is set as opt/Sybase/data/ipdr. To change
this destination, follow the instructions below:
1. Open an SSH session to the SMP and run the following command:
dbisql_cmd.sh smf
2. Open UPDATE PARAM SET STR_VAL.
3. In the IPDRs Path parameter, enter the folder where you wish IPDRs to be
sent.

Loading CMTS Lists

Note: An SMP must have been configured for SMF Mode when installed in order to
use CMTS Awareness.
CMTS Interfaces are associated to SMPs or SMP Groups though the use of a CSV
List which includes the following information about the CMTS Interface:
• Name
• IPv4 address
• SNMP community string
Each CMTS may be associated to one SMP, but multiple different lists may be
associated with the same SMP.
To associate a CMTS Interface to an SMP:
1. Right click on an SMP or SMP Group in the Navigation Pane of the
NetXplorer GUI and select Properties.

Data Mediator Installation and Administration Guide 3-20

 Initial Configuration

2. Select the CMTS List Tab.

3. Click the Load from File button and browse to the location of the CSV file.
4. Click Save.

Enabling and Configuring CMTS Awareness on the Network

1. In the Navigation pane, select and right-click the Network in the
Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration
from the View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
The Configuration window for the Network is displayed.
2. Select the Net Awareness tab and select CMTS from the drop down menu
for Net Awareness Working Mode.

Data Mediator Installation and Administration Guide 3-21

 Initial Configuration
Note: The CMTS option will only appear in the drop down menu if you have an SMP
installed that is working in SMF mode.

Figure 3-13: New Awareness (CMTS) Tab

3. When the Net Awareness tab is set for CMTS, the following fields may be
configured.
⧫ Channel Usage and Speed Update Interval (in min) – How often CMTS
channel usage and speed data are updated in the SMP (through
SNMP). By default the interval is every 5 minutes but it can be set to
between 1 and 60 minutes.
⧫ CMTS Interface Update Interval (in min) – How often the list of CMTS
Interfaces are updated in the SMP (through SNMP). By default the
interval is every 60 minutes but it can be set to between 5 and 1500
minutes.
⧫ Interface Congestion Clearance Interval – How many Channel Usage
and Speed Updates (see above) that do not indicate congestion are
required to change a Channel from Congested to Not Congested. By
default 3 updates are required but it can be set to between 1 and 10
updates.

Data Mediator Installation and Administration Guide 3-22

 Initial Configuration
⧫ Congestion Threshold (%) – The percentage of the speed of the
interface that needs to be crossed in order for the interface to be
designated as Congested. By default this is 80% but it may be set to
between 2 and 99 percent.
⧫ Congestion Clearance Threshold (%) – The percentage of the speed of
the interface that needs to be crossed in order for the interface to be
designated as Not Congested. By default this is 60% but it may be set
to between 2 and 99 percent.
⧫ Default Action of Interfaces – the default action taken regarding
interfaces.
There are two possible actions:
▪ enforcement – both congestion monitoring and enforcement is
carried out.
▪ monitor_only – no congestion enforcement is carried out
⧫ Interface Input CSV File – It is possible to modify some interface
attributes using a csv input file. This file can only modify existing
interfaces, it can't create new ones.
The Input CSV file format is as follows:
COLUMN # DESCRIPTION ALLOWED VALUES USAGE

1 CMTS name – same name as Must match one of the key

appears in the CMTS list defined CMTS in the
system

2 MAC domain name Informative

MAC domain if index key

3 Interface name Informative

4 Interface type Channel/BG/WB Informative

5 Interface direction US/DS key

6 Interface channel id/list Channel id in hex (for key

single channel) or
concatenation of all
channel ids in hex (for
BG/WB)

7 Congestion threshold [%] 2-99 Input parameter

<empty>: system default

Data Mediator Installation and Administration Guide 3-23

 Initial Configuration
COLUMN # DESCRIPTION ALLOWED VALUES USAGE

8 Congestion clearance 1-98 Input parameter

threshold [%] <empty>: system default

9 Action monitor_only: No Input parameter

congestion management
enforced: congestion
management (and
monitoring)

Figure 3-14: Input csv Example

⧫ Congestion Service Plans – In this field it is possible to define the
service plans used during congestion. Congestion service plans are
defined per service plan (that is the service plan used when there is
no congestion).

There are 3 congestion service plans:

▪ Service Plan for downstream only congestion
▪ Service Plan for upstream only congestion
▪ Service Plan for downstream and upstream congestion

Data Mediator Installation and Administration Guide 3-24

 Initial Configuration
Click Add to open the Congestion Service Plan dialog box and assign
three Congestion Service Plans to an existing Service Plan.

Figure 3-15: Congestion Service Plan dialog box

CMTS Interface Status Report

It is possible to get status reports on one or all active CMTS Interfaces from the
NetXplorer GUI.
1. From the NetXplorer Menu Bar, select Tools > CMTS Interface Status
Report.

The CMTS Interface Status Report dialog will open.

Figure 3-16: CMTS Interface Status Report Dialog

2. Click the checkboxes to filter the Report by Name of CMTS, MAC Domain
Name, Direction (Any, Upstream or Downstream), Interface Type (Any, BG,
Data Mediator Installation and Administration Guide 3-25
 Initial Configuration
Channel or Wideband), Congestion Status (Any, Congested or Cleared) or
Action (Any, Enforced or Monitored Only).
3. Click OK to generate the requested report.
4. The report consists of the following information on the selected CMTS
interfaces:
⧫ CMTS name
⧫ MAC domain name
⧫ MAC domain if index
⧫ Interface name
⧫ Interface type (channel/BG/WB)
⧫ Interface direction (US/DS)
⧫ Interface channel id/list
⧫ Interface NetUnitId
⧫ Interface speed [bps]
⧫ Interface utilization [%]
⧫ Congestion threshold [%]
⧫ Congestion clear threshold [%]
⧫ Congestion status (congested/cleared)
⧫ Action (enforced/monitor_only)

Data Mediator Installation and Administration Guide 3-26

 Initial Configuration

Figure 3-17: CMTS Interface Status Report

Adding a Data Mediator

1. Open NetXplorer.
2. In the Navigation pane, right-click Servers in the Network pane in the
Navigation tree and select New Data Mediation… from the popup menu.
3. The Data Mediation Properties - New dialog is displayed.

Data Mediator Installation and Administration Guide 3-27

 Initial Configuration

Figure 3-18: Data Mediation Properties

4. Enter the name of the Data Mediator.
5. Enter the Network Address of the Data Mediator in the designated field.
This may be an IPv4 address or an IPv6 Address.
6. Select the Enable SSL Between NetXplorer Server and Device checkbox if
you wish the connection between NetXplorer and the DM to be more
secure.
7. In the Source Units area, use the arrow keys to move Service Gateways
and SMPs from the Available to the Selected lists. Those selected will
provide data to the Data Mediator.
8. To collect WSP Buckets, enter the IP address (IPv4 or IPv6) of your
NetworkSecure in the NetworkSecure Details field and click the Right
Button to add it to the Selected sources.

Data Mediator Installation and Administration Guide 3-28

 Initial Configuration
Note: SDR, CMDR, CMCS and CMBM collection is only possible if you have included
an SMP in the Selected Sources.

CMDR, CMCS and CMBM collection requires an SMP in SMF Mode. For more
information, see the SMP Installation and Administration Guide.

WSP Bucket collection is only possible if you have added the NetworkSecure IP
to the Selected Sources.
9. Select a Profile to be associated with this Data Mediator. For more
information on Profiles, see Default Profiles.
10. Click Save to add the Data Mediator to the network.

Configuring the Data Mediator for Data in Motion

Encryption (GDPR)
General Data Protection Regulation (GDPR) is a collection of EU privacy regulations.
GDPR aims primarily to give citizens and residents control over their personal data
and to simplify the regulatory environment for international business by unifying
the regulation within the EU. The regulations affect all organizations that collect
the data of EU residents.
Allot complies with Privacy by Design and Default by ensuring the encryption of
the following via the Data Mediator:
• Data in Motion: All interfaces that pass subject's personal data between
Allot components or from Allot components to customer components, as
follows:
⧫ The collection of the buckets by the Data Mediator (DM) from the
NetworkSecure components
⧫ The sending of the buckets by the DM to ClearSee or the customer
⧫ The sending of the buckets by ClearSee to NetworkSecure’s Central
Reporter
⧫ The Individual Rights APIs (described below)
To configure the DM to encrypt Data in Motion:
1. From the tree, open the Data Mediation Properties dialog box.

Data Mediator Installation and Administration Guide 3-29

 Initial Configuration

2. In the Selected Sources tree, for each NetworkSecure module, do the

following:
⧫ Select the module, and then click Source Properties.
Note: If the NetworkSecure version is not 15.3, then the Source Properties button is
greyed out.

⧫ In the Source Properties dialog box, select HTTPS as Protocol, 443 as

Port, and then User Name and Password the same as the those that
you configured on the NetworkSecure side.

Data Mediator Installation and Administration Guide 3-30

 Initial Configuration
⧫ Click Save, and then, in the Data Mediation Properties dialog box,
click Save.

Enabling HTTPS Communication with the NetXplorer

The communication protocol used for communicating between the Data Mediator
and the NetXplorer can be configured via the /opt/allot/conf/nms_config.xml file.
In order to change from HTTP to HTTPS please modify the following value:
<property name="nxManagementProtocol" value="http" /> <!-- http,
https -->
TO
<property name="nxManagementProtocol" value="https" /> <!-- http,
https -->
After modifying the value, save and restart the DM process using the following
command:
keeperMgr –R datamediation

Data Mediator Installation and Administration Guide 3-31

4 Enabling CDR Types
All of the CDR types must be configured from the Network Configuration dialog
within the NetXplorer GUI.
In addition, CONV-RTS buckets, HDRs, VDRs, MOU and Conversation records must
also be enabled on each relevant In-line Platform individually. SDRs must be
enabled on each relevant SMP, which will also enable UDRs.

4.1 Enabling HDRs per In-line Platform

1. In the Navigation pane, select and right-click the relevant In-line Platform
in the Navigation tree and select Configuration from the popup menu.
OR
Select the In-line Platform in the Navigation tree and then select
Configuration from the View menu.
OR
Select the In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
2. The Configuration window for the selected In-line Platform is displayed.
3. Select the Service Activation tab.
4. To enable HDRs, select Enable from the HDR Generation Enablement drop
down field.

Figure 4-1: HDR CDR area in the Service Activation tab

5. In the HTTP CDR area, the following fields may also be configured.
⧫ HDR Generation Rate Limit – This defines the maximum number of
entries generated per second. The maximum possible value depends
upon which In-line Platform or Core Controller blade is being used,
see the relevant Release Notes for details.
 Enabling CDR Types
⧫ HDR File Compression – This sets the file compression method to be
used when storing the file on the In-Line Platform. The possible
options are GZIP and no compression. GZIP is selected by default.
6. Save the configuration changes to apply the new settings.

HDR Generation on Blocked Traffic

It is possible to configure HDRs to only send buckets based on Blocked traffic.

Enabling the Feature on the NetXplorer

Run the following CLI command on the NetXplorer and then restart the GUI:
ConfigurationCLI.sh -setParam -paramName FEATURE_TOGGLE
-paramSubName hdrAuditBlockedSites -setIntegerParam 1

Enabling the Feature per Device

1. In the Navigation pane of the NetXplorer GUI, select and right-click the
relevant In-line Platform in the Navigation tree and select Configuration
from the popup menu.
OR
Select the In-line Platform in the Navigation tree and then select
Configuration from the View menu.
OR
Select the In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
The Configuration window for the selected In-line Platform is displayed.
2. Select the Service Activation tab.
3. To enable HDRs for Blocked Traffic only, select Enabled for Blocked
Requests from the HDR Generation Enablement drop down field.

Data Mediator Installation and Administration Guide 4-2

 Enabling CDR Types

Figure 4-2: HDR CDR area in the Service Activation tab

Click Save to save the change.

4.2 Enabling VDRs per In-line Platform

1. In the Navigation pane, select and right-click the relevant In-line Platform
in the Navigation tree and select Configuration from the popup menu.
OR
Select the In-line Platform in the Navigation tree and then select
Configuration from the View menu.
OR
Select the In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
2. The Configuration window for the selected In-line Platform is displayed.
3. Select the Service Activation tab.
4. To enable VDRs, select Enable from the VDR Generation Enablement drop
down field.

Data Mediator Installation and Administration Guide 4-3

 Enabling CDR Types

Figure 4-3: VDR CDR area in the Service Activation tab

5. In the VDR CDR, the following other fields may be configured.
⧫ VDR Generation Rate Limit – This defines the maximum number of
entries generated per second. The maximum possible value depends
upon which In-line Platform or Core Controller blade is being used,
see the relevant Release Notes for details.
⧫ VDR File Compression – This sets the file compression method to be
used when storing the file on the In-Line Platform. The possible
options are GZIP and no compression. GZIP is selected by default.
6. Save the configuration changes to apply the new settings.

4.3 Enabling MOU per In-Line Platform

1. In the Navigation pane, select and right-click the relevant In-line Platform
in the Navigation tree and select Configuration from the popup menu.
OR
Select the In-line Platform in the Navigation tree and then select
Configuration from the View menu.
OR
Select the In-line Platform in the Navigation tree and then click the
Configuration icon on the toolbar.
2. The Configuration window for the selected In-line Platform is displayed.
3. Select the Service Activation tab.
4. To enable MOU, select Enable from the VOIP Data Collection drop down
field in the VOIP Reports area.

Data Mediator Installation and Administration Guide 4-4

 Enabling CDR Types

Figure 4-4: VOIP Reports in the Service Activation tab

5. Save the configuration changes to apply the new settings.

4.4 Enabling and Configuring Conversations

Enabling Conversations Records per In-line Platform
Conversations records must be enabled on each relevant In-line Platform by logging
into the platform and entering the following CLI command:
go config data_collect –conv_export enable
The may be disabled by using the following CLI command:
go config data_collect –conv_export disable

Prioritizing Conversations Statistics

It is possible for the operator to prioritize the Conversation records by application
as follows:
• Golden: Conversation CDRs are exported in full detail due to high interest.
• Excluded: Conversation CDRs are exported in an abridged format due to
lack of interest (only number of connections and throughput is saved, with
no rule ID and client IP).
• Normal: Conversation CDRs are exported in summary detail. This priority
level is set to applications by default.

Data Mediator Installation and Administration Guide 4-5

 Enabling CDR Types
All Conversation records are collected in detail and are summarized to reduce the
storage space required. This feature allows the operator determine which
applications should be kept and exported in full Conversation detail, which
applications can be summarized and which are exported in a highly abridged
format. Sufficient storage space is then set based on these priorities.
The operator may further define whether the Conversation data should be
exported in 5-minute or 1-hour aggregations.
All Conversation records are collected in detail and are summarized to reduce the
storage space required. This feature allows the operator determine which
applications should be kept and exported in full Conversation detail, which
applications can be summarized and which need not be exported at all. Sufficient
storage space is then set based on these priorities.

Assigning Priority
Priority levels need to be configured via the CLI using the following commands:

To set the priority level:

go add app_collection [app_name] –priority golden/excluded
app_name – service catalogue name entry (excluding service groups)

To show the priority list by application:

go list app_collection

Golden services:
• Viber
• Skype
• Tango
• Line
• MGCP
• Yahoo Chat
• Jabber
• Google Play
• iTunes
• Facebook Chat
• Baidu Hi

Data Mediator Installation and Administration Guide 4-6

 Enabling CDR Types
• QQ Chat
• Twitter
• Facebook
• Yandex
• VKontakte
• Gmail
• Call Of Duty
• GoogleTalk
• WhatsApp
• MailRu
• Yahoo Mail
• DiabloIII
• Lineage2
• Kakao

Excluded services:
• BGP
• DNS
• EGP
• ICMP
• IGMP
• OSPF
• RIP
• ARP
• PPPoE-Discovery
• PPPoE-Control
• DHCP-Client
• DHCP-Server
• NTP
• IS-IS
• LACP

Data Mediator Installation and Administration Guide 4-7

 Enabling CDR Types
To change/remove the configuration for a specific application:
go change app_collection [app name] –priority golden/excluded
go delete app_collection [app name] –priority golden/excluded

4.5 Enabling Real-Time Reports (CONV-RTS) per

NetXplorer
Note: Real-Time Reporting requires a license key before it can be enabled. For more
information, contact Allot Customer Support.
To Enable the ClearSee Real-Time Monitor’s "User Monitor" or "Client IP Monitor"
reports which require the CONV-RTS buckets you need to add the ClearSee server
IP to the NetXplorer’s Allowed Hosts and enter the appropriate license key.
1. In the Navigation pane, select and right-click the Network in the
Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration
from the View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
2. The Configuration window for the NetXplorer is displayed.
3. Select the Servers tab.
4. Add the ClearSee server IP to the "Allowed Hosts" list or select "Allow all
Hosts".

Data Mediator Installation and Administration Guide 4-8

 Enabling CDR Types

Figure 4-5: Servers tab

5. Click Save.
6. Log into an In-Line Platform and run the following command to confirm
that Real Time Monitoring is enabled.
go config view data_collect
If Real Time Reporting has been enabled the output will include the
following values:
RT CONV Bucket enable enable
Unsolic Line ID enable enable
Unsolic Pipe ID enable enable
Unsolic VC ID enable enable
Unsolic Service ID enable enable
Unsolic Next hop ASN enable enable
Unsolic Mon srv group enable enable
Unsolic rt conv interval 15
Solicited rt conv interval 15
7. To enable Real Time Reporting (CONV-RTS bucket collection) enter the
following CLI command:
go config data_collect -rt_conv_bucket enable

Data Mediator Installation and Administration Guide 4-9

 Enabling CDR Types

4.6 Enabling SDRs per SMP

Session Detail Records (SDRs) must be generated on the SMP server to extract the
relevant information from the RADIUS message. This is enabled via the NetXplorer
GUI.
1. In the Navigation pane of the NetXplorer User interface, right-click the
Network and select Configuration from the popup menu

OR

Select Network in the navigation pane, and then choose Configuration

from the Actions menu.

The network tabs will be displayed in the Applications pane.

2. Select the SMP tab. In the SMP preferences section, 4 additional tabs will
appear.
3. Select “Session Parameters”. From the Session Parameters tab, determine
the extra RADIUS parameters which you wish to be included in the SDRs.
Check whichever fields you wish to include, as shown in Figure 4-6 below.

Data Mediator Installation and Administration Guide 4-10

 Enabling CDR Types

Figure 4-6: Adding Additional RADIUS Parameters to the SDR

Note: You can also set a default value for some fields. This default value will be
included in each SDR for the particular field specified, should no actual value
be received.

The AVPs which you select in this dialog to be included in the SDR must have
first been configured to be extracted from the RADIUS message in the
radius_params.conf file.
4. Select the “Policy and Charging” tab. Set the SDR Collection Status field to
“On”

Data Mediator Installation and Administration Guide 4-11

 Enabling CDR Types
Note: Unless SDR Collection Status is defined as On in the Policy and Charging Tab,
no SDRs will be generated even if SDR Export is enabled in other areas of the
GUI.

Figure 4-7: Policy and Charging Tab

5. You will be prompted to reboot the SMP server, whereupon SDR collection
will commence.
6. To confirm that SDRs are being generated, open the pccm.log and confirm
that Start SDRManager appears in the log after the SMP has been
rebooted.

4.7 Enabling HTTP Reports per NetXplorer

1. In the Navigation pane of the NetXplorer, right-click the Network in the
Navigation tree and select Configuration from the popup menu.
OR
Select the Network in the Navigation tree and then select Configuration
from the View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
2. Open the Integrated Service tab.

Data Mediator Installation and Administration Guide 4-12

 Enabling CDR Types

Figure 4-8: Integrated Service Tab

3. In the HTTP Monitoring Service Specific Configurations area, set the
Default Policy Action to Always Enabled.

4. Click or select Save from the File menu to save the changes to the
configuration.
Note: It is possible to set the Default Policy Action to Enforcement Policy Based. This
means that HTTP Reports will only be available for Lines, Pipes or VCs for
which the service has been activated in the Enforcement Policy.

Data Mediator Installation and Administration Guide 4-13

5 Smart Data Source
Allot’s Data Mediator features configurable Output Records and Output Profiles,
allowing custom export of records which include only the relevant attributes and
metrics by making use of exclude and include commands. This feature, called Allot
Smart Data Source, collects and exports rich network usage records to your BI
systems, enabling you gain deeper insight about subscribers and the way they use
your network services.
Attributes which have multiple format options, such as Date and Time, can be set
for export in any of the available formats. The records are exportable in CSV format
and are volume, time and event capable.

Any type of record can be exported to multiple systems and also to the Allot
ClearSee network analytics tool. A separate bucket will be generated for each
record with the ability to include or exclude attributes and metrics.
After setting up and connecting the Data Mediators and enabling the required
CDRs, the data record fields to be exported must be selected and configured using
Output Records and Output Profiles. This consists of the following steps, all of
which are taken from the NetXplorer interface:
• Creating Output Records or Selecting/Editing Default Output Records
• Creating Output Profiles and assigning Output Records to them
• Assigning Data Mediators to Output Profiles
Upon installation, a Default Output Profile and several Default Output Records are
created that set output values for the suggested fields of each Data Type to
suggested levels. When the system is upgraded, a new Default Output Profile is
created and an Output Profile called Base is created using your current values.
For full listings of the available Output Fields in each Output Record, see the Allot
Data Dictionary Guide.
 Smart Data Source
An Output Profile is made up of one or more Output Records, which defines the
fields that will be included in the Output Profile. Output Records must be selected
or created and assigned to an Output Profile, and the Output Profile is then
assigned to one or more Data Mediators.
Note: If Allot ClearSee is installed on the Network, the Default Output Profile will be
called Allot Clearsee Profile.

5.1 Output Records

Output Records may be added or edited in the Data Mediation tab of the Network
Configuration window by selecting the Output Records tab. Click the Add button or
select an existing Output Record and click Edit to open the Output Records
Properties.

Figure 5-1: Output Records tab

For full listings of the available Output Fields in each Output Record, see the Allot
Data Dictionary Guide.

Input Record Filtering Rules

In addition you may select fields to be included or excluded globally (from all
Output Records) from the Input Records Filtering Rules section of the Output
Records tab. In this area you must select an Operation (AND indicates that all rules
must be true for all of the fields to be included/excluded and OR indicates that any
of the rules may be true and therefore included/excluded individually).

To add a filtering rule on a global level:

In the Filtering Rules area of the Output Records tab, click Add to add a rule to the
Filtering Rules list. The Filter Rule dialog box appears:

Data Mediator Installation and Administration Guide 5-2

 Smart Data Source

Figure 5-2: Filtering Rule dialog box

1. Enter a name for the filtering rule and select an Operation (AND indicates
that all conditions must be true for the rule to fire and OR indicates that
any of the conditions may be true for the rule to fire).
2. Click Add to add conditions to the rule. The Import Record Filtering Rule
Condition dialog box appears.

Data Mediator Installation and Administration Guide 5-3

 Smart Data Source

Figure 5-3: Import Record Filtering Rule Condition dialog box

3. In the Import Record Filtering Rule Condition dialog, the following
parameters may be configured.
• Name – Enter a name for the condition.
• Bucket Type – This defines the bucket type that is the source of the field.
Select from HDR, CMCS, VDR, CMDR, CMBM, VC, CONV, UDR, SDR or
MOU. For more information on any of these bucket types, see the relevant
chapter in this document.
• Field Per Bucket Type – This defines the specific field you are
including/excluding. The available fields will change depending upon the
bucket type selected. For a full list of available fields for each bucket type
and definitions, see the relevant Bucket Type chapter.
• Operator – This defines the operation that will be performed on the value
listed below. Different options are available depending on the field
selected. The possible options are:
⧫ Equal
⧫ Not Equal
⧫ Greater Than
⧫ Less Than
⧫ Value – The value that will be used to determine if an input record is
included/excluded, using the Operator above.
For example, if you wish a specific HDR client IP to be excluded, you should select
HDR in the Bucket Type Field, ClientIP in the Field Per Bucket Type Field, Equal in

Data Mediator Installation and Administration Guide 5-4

 Smart Data Source
the Operator field and the IP to be excluded in the Value field. In the same way you
can exclude all IP’s other than that specific client by changing the Operator field to
Not Equal.
4. Click OK to add the condition to the Input Records Filtering Rule. The
values that you have set for the condition will be listed in the tab.

Creating/Editing Output Records

To create an Output Record, click the Add button in the Output Records List area,
or to Edit an existing Output Record select the Output Record in the list and click
Edit.
The Output Record Properties dialog appears.

Figure 5-4: Output Record Properties dialog, Output Fields tab

In the Output Record Properties dialog there are four tabs:
• Output Fields – Configures the fields that will appear in the Output Record

Data Mediator Installation and Administration Guide 5-5

 Smart Data Source
• Input Records Filtering Rules – Configures those fields that will be included
or excluded from the Output Record under certain conditions
• Output Keys – Configures keys that connect the Output Records to the
source
• Output Triggers – Configures the trigger that causes the Output Record to
send data to the Data Mediator

Output Fields
In the Output Fields tab you can select and configure the interface fields which will
be included in the Output Record.
For full listings of the available Output Fields in each Bucket, see the Allot Data
Dictionary Guide.
1. Open the Output Fields tab.
2. Select a field that has already been added to the Output Record and click
Edit

OR

Click Add to add a new field to the Output Record.

The Output Record Field dialog will appear.

Data Mediator Installation and Administration Guide 5-6

 Smart Data Source

Figure 5-5: Output Record Field dialog

3. In the Output Record Field dialog, the following parameters may be
configured:
⧫ Output Field Type – This indicates if the field will have No Source
(consisting of a pre-defined constant value, like a comment), Single
Source (coming from a single In-line Platform) or Multi Source
(coming from multiple In-line Platforms).
⧫ Name – Enter a name for the field. By default the name is the name of
the field that you select, but this may be edited.
⧫ Bucket Type – This defines the bucket type that is the source of the
field. Select from HDR, CMCS, VDR, CMDR, CMBM, VC, CONV, UDR,
SDR or MOU. For more information on any of these bucket types, see
the relevant chapter in this document.
⧫ Field Per Bucket Type – This defines the specific field you are adding
from the selected Bucket Type. The available fields will change
depending upon the bucket type selected. For a full list of available
fields for each bucket type and definitions, see the relevant Bucket
Type chapter.

Data Mediator Installation and Administration Guide 5-7

 Smart Data Source
⧫ Operator – This defines the operation that will be performed on the
field. Different options are available depending on the field selected.
The possible options are:
▪ Copy – Replaces the previous value
▪ Min - Calculates the minimum value of an input record’s field
▪ Max - Calculates the maximum value of an input record’s field
▪ Sum - Aggregation of an input record’s field
▪ Average - Calculates the average of an input record’s field
⧫ Format – This selects the display format (if relevant) to be used on the
field. The available options are different for each field and could
include numeric, script, etc.
⧫ Obfuscation - This selects the obfuscation method (if relevant) to be
used on the field. The available options are:
▪ None – No obfuscation
▪ Encrypt – reversible, using configurable key. For more
information, contact Allot Customer Support.
▪ Hash – irreversible, using SHA-1 algorithm
⧫ Export Enable – Select this checkbox to allow the field to be exported.
⧫ Persistent – Select this checkbox to make the field Persistent, which
means that the field will be copied to a new output record with the
same keys.
⧫ Extension Mandatory – Select this checkbox if you wish the field to
require a Dynamic Subscriber ID to be linked to the field’s value for
enhanced obfuscation. If no such ID exists but Extension Mandatory is
selected, then the value will be linked to an empty field.
⧫ Text Decoding – Select this checkbox if you wish all HTTP encoding in
URIs to be properly rendered. For example, check this box to have
character spaces appear properly rather than as %20.
4. Click OK to add the field to the Output Record. The values that were set
for the field in the Output Record Field dialog will be listed in the Output
Fields tab.

Data Mediator Installation and Administration Guide 5-8

 Smart Data Source
Input Records Filtering Rules
In the Input Records Filtering Rules tab you can select and configure the interface
fields which will be included/excluded in this Output Record only due to their
value.

Figure 5-6: Output Record Properties tab –Input Records Filtering Rules tab
1. Open the Input Records Filtering Rules tab.
2. In the Operation area, select one of two radio buttons:
⧫ AND – the selected filtering rules all must be true for the fields to be
included/excluded
⧫ OR – one or more of the rules can be true, each applied individually.
3. In the Action Per All Rules area, select if you wish the rules to all be
included, or all be excluded.
4. Select a field that has already been added and click Edit

Data Mediator Installation and Administration Guide 5-9

 Smart Data Source
OR
Click Add to add a new field.
The Filtering Rule dialog will appear.

Figure 5-7: Filtering Rule dialog box

5. Enter a name for the filtering rule and select an Operation (AND indicates
that all conditions must be true for the rule to fire and OR indicates that
any of the conditions may be true for the rule to fire).
6. Click Add to add conditions to the rule. The Import Record Filtering Rule
Condition dialog box appears.

Data Mediator Installation and Administration Guide 5-10

 Smart Data Source

Figure 5-8: Import Record Filtering Rule Condition dialog box

7. In the Import Record Filtering Rule Condition dialog, the following
parameters may be configured.
⧫ Name – Enter a name for the condition.
⧫ Bucket Type – This defines the bucket type that is the source of the
field. Select from HDR, CMCS, VDR, CMDR, CMBM, VC, CONV, UDR,
SDR or MOU. For more information on any of these bucket types, see
the relevant chapter in this document.
⧫ Field Per Bucket Type – This defines the specific field you are
including/excluding. The available fields will change depending upon
the bucket type selected. For a full list of available fields for each
bucket type and definitions, see the relevant Bucket Type chapter.
⧫ Operator – This defines the operation that will be performed on the
value listed below. Different options are available depending on the
field selected. The possible options are:
▪ Equal
▪ Not Equal
▪ Greater Than
▪ Less Than
⧫ Value – The value that will be used to determine if an input record is
included/excluded, using the Operator above.
8. Click OK to add the condition to the Input Records Filtering Rule. The
values that you have set for the condition will be listed in the tab.

Data Mediator Installation and Administration Guide 5-11

 Smart Data Source
Output Keys
Data can be pre-aggregated using Output Keysto define which records should be
aggregated and the triggers on any customizable record. The aggregation can be
time capable in which case the available time resolutions are 5 min, 10 min, 15 min,
20 min, 30 min, 1 hour or after a certain time out. Alternatively, the aggregation
can be event capable based on an input attribute value and a rule such as 'equals
to', 'less than', 'greater than', etc. One can use this capability to aggregate, for
example, watched videos containing several sub-watches (and thus represented by
separate records) into one record representing the video watch with the
aggregated information. Multiple options are available to define which functions to
apply to aggregate statistics information, i.e. sum of total, max, min etc.
In the Output Keys tab you can define these keys which each represent a different
connection between the source records and the Output Record.

Figure 5-9: Output Record Properties tab – Output Keys tab

1. Open the Output Keys tab.
2. Select a field that has already been added and click Edit

Data Mediator Installation and Administration Guide 5-12

 Smart Data Source
OR
Click Add to add a new field.
The Output Key dialog will appear.

Figure 5-10: Output Key dialog

3. In the Output Key dialog, the following parameters may be configured.
⧫ Name – A user-defined name for the output key.
⧫ Description – A brief description of the key.
⧫ Input Bucket Type – This defines the bucket type that is the source of
the key, which must already have at least one field added to the
Output Record. For more information on any of bucket types, see the
relevant chapter in this document.
⧫ All Available Fields– This defines the specific fields you may use as
part of the key. The available fields will change depending upon the
bucket type selected in the Input Bucket Type field. For a full list of
available fields for each bucket type and definitions, see the relevant
Bucket Type chapter.
⧫ Selected Fields – Those fields which you have added to the key. Use
the arrow keys to add or remove fields from this list.

Data Mediator Installation and Administration Guide 5-13

 Smart Data Source
4. Click OK to add the key to the Output Keys tab. The name and description
that you have set for the key will be listed in the tab.

Output Triggers
In the Output Triggers tab you can define triggers to cause certain actions to be
performed on the Output Record.

Figure 5-11: Output Record Properties tab – Output Triggers tab

1. Open the Output Triggers tab.
2. Select a field that has already been added and click Edit
OR

Click Add to add a new field.

The Output Triggers dialog will appear.

Data Mediator Installation and Administration Guide 5-14

 Smart Data Source

Figure 5-12: Output Triggers dialog

3. In the Output Triggers dialog, the following parameters may be
configured.
⧫ Trigger Name – A user-defined name for the trigger.
⧫ Description – A brief description of the trigger.
⧫ Trigger Type – This defines the nature of the trigger. Options include
Immediate (which means that the trigger fires as soon as the Output
Record receives data), Data Driven (which means that you can select
the field and condition that will act as the trigger), or Time Driven (the
trigger is based on time only).
⧫ Data Driven Configuration (only appears when Data Driven chosen as
Trigger Type) – The parameters in this area allow you to select the
field and situation that will act as the trigger. You need to select the
Field (the field must have already been added to the Output Record),
the Operator which is the condition of the field that triggers the
action (Exist, Equal, Not Equal, Less Than, Less Than or Equal, Greater
Than, Greater Than or Equal), and the Value (if relevant).

For example if you wish the trigger to be when the Octets In drops
below 144000000, then select Field: OctetsIn, Operator: Less Than
and Value: 144000000.

Data Mediator Installation and Administration Guide 5-15

 Smart Data Source
⧫ Time Driven Configuration (only appears when Time Driven chosen as
Trigger Type) – The parameters in this area allow you to select the
time that will act as the trigger. The two options are On Round Time
(every hour) and On Timeout (after a set number of minutes). If you
select On Timeout, you may set the number of minutes in the Timeout
(min) field.
⧫ Actions – The parameters in this area allow you to indicate what will
happen when the trigger occurs. You may select the Export Record on
Trigger Occurrence which exports the record when the trigger fires,
and/or the Reset Persisted Fields which causes all Persisted fields to
be repopulated when the trigger fires.
4. Click OK to add the key to the Output Triggers tab. The name, description
and trigger type that you have set will be listed in the tab.
When a trigger fires, a new output record with the same keys is opened and the
persisted fields are populated if Reset Persisted Fields is checked for this trigger
The record is then exported to a file if Export Record on Trigger Occurrence is
checked in the trigger. The record is then deleted.

5.2 Output Profiles

Output Profiles may be added or edited in the Data Mediation tab of the Network
Configuration window by selecting the Output Profiles tab. Click the Add button or
select an existing profile and click Edit to open the Data Output Profile window
where values for all Data Types are set.
In order to be configured, each Data Mediator must be assigned to one or more
Output Profile. An Output Profile sets the output parameters for each Data Type
and may be assigned to one or more Data Mediators. An Output Profile is assigned
to a Data Mediator when the Data Mediator is added to the system. The Output
Profile assigned to a Data Mediator can be changed at any time from the Properties
window of the Data Mediator.
By Default, four Profiles are created automatically: One which is used with
NetXplorer Reporting (Allot Default Profile) and three profiles for use with ClearSee
(Allot Default ClearSee Metrics Profile, Allot Default Analytics Professional Profile
and Allot Default ClearSee Real Time Profile).

Data Mediator Installation and Administration Guide 5-16

 Smart Data Source

Figure 5-13: Output Profiles tab

In the Output Profiles tab you can select and configure the Output Records and
CDRs that will be included in the Profile.

Creating/Editing/Duplicating Output Profiles

1. Open the Output Profiles tab.
2. Select an existing Output Profile and click Edit
OR
Click Add to add a new Output Profile.
The Output Record Field dialog will appear.

Data Mediator Installation and Administration Guide 5-17

 Smart Data Source

Figure 5-14: Data Output Profile dialog

3. In the Data Output Profile dialog, the following parameters may be
configured.
⧫ Name – Enter a name for the profile.
⧫ Description – Add a description of the Profile, if desired.
⧫ Associated Output Records – This defines those predefined Output
Records that will be included in the Profile.
⧫ File Attributes and Push Properties – These are defined for each
Output Record. Select an Output Record in the Associated Output
Records list in order to define the following parameters
▪ File Size Limit (in bytes) – This defines the maximum possible size
of each exported file before compression. The default is 100,000
bytes (100 MB) and the maximum possible value is 1,000,000,000
bytes (1 GB).

Data Mediator Installation and Administration Guide 5-18

 Smart Data Source
▪ File Closing Interval (in seconds) – This defines how long after
being created the export file is closed. The default is 300 seconds
(5 minutes) and the maximum is 3600 seconds (1 hour).
▪ File Compression method – This sets the file compression method
to be used when exporting the file. The possible options are GZIP
and no compression. GZIP is selected by default.
▪ Delete files older than (in minutes) – This defines the amount of
time the system retains files. The default is 1440 minutes (24
hours) and the maximum is 4320 minutes (72 hours).
▪ Push Properties – Push properties, once enabled (default), define
how the export files are sent to their proper locations for analysis
and storage. This can be set per bucket type. The four push
method options are SFTP, FTP, SCP and Copy. If SFTP or SCP are
selected, the user must select the Security Method, either
Password or SSH Key File.
Note: SSH Key File is of use when pushing data to a 3rd party Data Warehouse.
▪ If SFTP Password is selected, the Username, Password,
Server and Path must be defined.
▪ If SCP Password is selected, the username and password
must be defined.
▪ If SFTP or SCP SSH Key File is slected, click the Upload File
button to locate and upload the appropriate key file,
which will then be indicated in the field below.
▪ For FTP and Copy, only the path must be defined.
Note: If special characters are used in the Push Properties password (for example
"E69{Gs[V%y" ) the data will not be pushed and you will see a "invalid URI
syntax" error in datamediation.log.
To delete an Output Profile, select the Output Profile in the list and click Delete.
Only user-generated Output Profiles that are not associated to a Data Mediator can
be deleted.
To Duplicate an Output Profile select the desired profile and click Duplicate. An
exact duplicate of the selected Output Profile will be created, indicated by a
number following the Profile name, for example, Allot Default ClearSee Metrics
Profile(1).

Data Mediator Installation and Administration Guide 5-19

 Smart Data Source

5.3 Default Profiles

Allot Default Profile

“Allot default profile” includes all the fields that appear in the DataDictionary file
for the following buckets with immediate trigger (no keys and no excluding rules):
• VC
• Conv
• Conv_RTS
• Conv_RTU
• UDR
• SDR
• HDR
• VDR
• CMDR
• CMCS

Data Mediator Installation and Administration Guide 5-20

 Smart Data Source
• CMBS
• WSP Buckets
Note: This Profile may be edited.

Allot Default ClearSee Metrics Profile

Allot Default ClearSee Light profile includes all the fields of all the records that
appear in the following AOS buckets with immediate trigger (no keys and no
excluding rules):
• HTTP
• Conv
• SDR
• CMDR
• WSP Buckets

Data Mediator Installation and Administration Guide 5-21

 Smart Data Source
Note: This Profile may not be edited and is only available with a ClearSee license.

Allot Default ClearSee Analytics Profile

Allot Default ClearSee Analytics Profile includes all the fields that appear in the
following AOS buckets with immediate trigger (no keys and no excluding rules):
• VC
• Conv
• UDR
• SDR
• HDR
• VDR
• CMDR
• MOU
• HTTP
• WSP Buckets
Note: This Profile may not be edited and is only available with a ClearSee license.

Data Mediator Installation and Administration Guide 5-22

 Smart Data Source
Allot Default ClearSee Real Time Profile

Allot Default ClearSee Real Time Profile includes all the fields that appear in the
following AOS buckets with immediate trigger (no keys and no excluding rules):
• Conv-RTS
• Conv-RTU
Note: This Profile may not be edited and is only available with a Real Time
Monitoring license.

5.4 Global Output Trigger

It is possible to configure a global time based trigger for all output records in all
profiles. This trigger’s goal is to make sure no temporary statistics record will be
left in the DM for more than 24. The default time is 1440 min (24 hours) and range
will be 180 to 45000. Accuracy will be 1 hour which means that the DM will purge
records which were unchanged for more than timeout ± 1 hour).
To set the Global Output Trigger, open the Data Mediation tab of the Network
Configuration window and select the General tab. In the Global Output Trigger
area change the value to the desired number of minutes.

Data Mediator Installation and Administration Guide 5-23

 Smart Data Source

Figure 5-15: General tab

Data Mediator Installation and Administration Guide 5-24

 Smart Data Source

5.5 Input Definitions

Configuring HDR Filtering Rules
To set Filtering Rules for HDR Records open the Data Mediation tab in the Network
Configuration window, select Input Definitions and open the HDR sub-tab.

Figure 5-16: Filtering Rules Tab

Using the Filtering Rules area, you may elect to Include All Data in your HDRs or to
Exclude All Data by selecting the appropriate radio button. You may then add
exceptions to the list below by clicking on the Add button.

Figure 5-17: Add Rule dialog

When the Add Rule dialog appears, you may create an exclusion rule by entering a
name and clicking Add to add attributes to the rule.

Data Mediator Installation and Administration Guide 5-25

 Smart Data Source

Figure 5-18: Add Attribute dialog

Clicking the Add button on the Add Rule dialog allows you to select to add a
ContentType, Path, Method or Domain and then enter the appropriate value. Click
OK to save your selections.

KEY DESCRIPTION EXAMPLES

ContentType The MIME type of this text/html
content (Multipurpose
image/gif
Internet Mail
Extensions) image/jpeg
Path The URI /index.path
/news
Method The desired action to GET, CONNECT, POST
be performed on the
resource.
Domain The domain name of a www.cnn.com
server.
www.ynetnews.com
In the Override Attribute Length area, you may define the actual length of each
string type HDR attribute. The length is defined in characters and the range for
each field is between 1 and 1024.
In the Override Attribute Length area, click Add to add a new field. The Override
Attribute Length dialog box appears.

Figure 5-19: Add Attribute dialog

In Attribute Name you can select the field whose length you wish to set using the
drop-down menu. Only relevant fields will be listed in the menu.

Data Mediator Installation and Administration Guide 5-26

 Smart Data Source
In Attribute Length, enter a value between 1 and 1024 to set the maximum length
in characters of the field.
Click OK to add the Attribute to the Override Attribute Length in the HDR tab.

Configuring VDR Filtering Rules

To set VDR Filtering Rules, open the Data Mediation tab in the Network
Configuration window, select Input Definitions and open the VDR sub-tab.

Figure 5-20: VDR Filtering Rules dialog

Using the Filtering Rules area, you may elect to Include All Data in your HDRs or to
Exclude All Data from your HDRs by selecting the appropriate radio button. You
may then add exceptions to the list below by clicking on the Add button.

Figure 5-21: Add Rule dialog

Data Mediator Installation and Administration Guide 5-27

 Smart Data Source
When the Add Rule dialog appears, you may create an exclusion rule by entering a
name and clicking Add to add attributes to the rule.

Figure 5-22: Add Attribute dialog

Clicking the Add button on the Add Rule dialog allows you to select to add a
ContentType, Path, Method or Domain and then enter the appropriate value. Click
OK to save your selections.

KEY DESCRIPTION EXAMPLES

ContentType The MIME type of this text/html
content (Multipurpose
image/gif
Internet Mail
Extensions) image/jpeg
Path The URI /index.path
/news
Method The desired action to GET, CONNECT, POST
be performed on the
resource.
Domain The domain name of a www.cnn.com
server.
www.ynetnews.com
In the Override Attribute Length area, you may define the actual length of each
string type VDR attribute. The length is defined in characters and the range for
each field is between 1 and 1024.
In the Override Attribute Length area, click Add to add a new field. The Override
Attribute Length dialog box appears.

Data Mediator Installation and Administration Guide 5-28

 Smart Data Source

Figure 5-23: Add Attribute dialog

In Attribute Name you can select the field whose length you wish to set using the
drop-down menu. Only relevant fields will be listed in the menu.
In Attribute Length, enter a value between 1 and 1024 to set the maximum length
in characters of the field.
Click OK to add the Attribute to the Override Attribute Length in the VDR tab.

Configuring the SDR SubSession Lifetime

The SubSession Lifetime for SDR entries is also set from the Data Mediation tab in
the NetXplorer GUI. Each subsession indicates the use of a specific application or
connection in a subscribers overall session, or any action which may change the
session data. The SubSession Lifetime value indicates how long a subsession will be
kept open.
1. To configure the SubSession Lifetime, select the Input Definitions tab and
open the SDR sub-tab.

Data Mediator Installation and Administration Guide 5-29

 Smart Data Source

Figure 5-24: Detail Record Config Sub-tab

2. In the SDR are, change the SubSession Lifetime value to define in hours
how long subsessions will be kept open.
Note: When using Allot’s ClearSee Analytics system, this value must be set to 1
Hour.

Configuring UDR Monitored Applications

The Monitored Applications list is where applications-based UDRs are configured.
The applications to be included in the UDR must be selected. In order for an
application to be selected, it must have already been added to the Charging
Applications catalog.
To set Monitored Applications, open the Data Mediation tab in the Network
Configuration window, select Input Definitions and open the UDR sub-tab.

Data Mediator Installation and Administration Guide 5-30

 Smart Data Source

Figure 5-25: Input Definitions – UDR sub-tab

You may add Monitored Applications to the list by clicking on the Edit button.

Figure 5-26: Change Monitored Applications dialog

When the Change Monitored Applications dialog appears, use the Arrow buttons to
move applications from the Available Applications list to the Selected Applications
list. Only those applications that have previously been added to the Charging
Applications catalog will appear in the Available Applications list.
Click OK to save and return to the UDR tab. Those applications you have selected
now appear in the Monitored Applications list.
Modification of the Selected Applications list while the system is running will affect
only new sessions. Statistics for sessions that started prior to the modification will
continue reporting according to the previous Selected Applications list.

Data Mediator Installation and Administration Guide 5-31

 Smart Data Source
Note: A UDR is generated for each monitored application configured, in addition to a
UDR with total usage for that session. The Session application is created
automatically and cannot be removed.
For more information on Charging Applications, see the NetXplorer Operations
Guide.

Configuring CMDR Time Intervals

1. Open the Input Definitions subtab.

Figure 5-27: Input Definitions - CMDR Sub-tab

2. In the CMDR area, enter values for the Time Interval Before Closure in
minutes (default is 20) and the Number of Records Before Closure (default
is 250,000). This defines how long a CMDR session remains open.

Data Mediator Installation and Administration Guide 5-32

6 Appendices
6.1 Appendix A – Binary Data Records (BDR)
Enabling and Configuring BDRs on the In-Line Platform
To enable BDR identification and data records generation, use the following
procedure:
1. Connect to the in-line platform, as sysadmin
2. Create a directory somewhere called WAConfig
3. Copy the configuration files into that directory. i.e.
cp /opt/allot/conf/WebAnalytics/* ./WAConfig
4. Edit your LOCAL ./WAConfig/server.cfg
5. Change the relevant parameters as in the following example:
#
# Pilot Packet configuration file
#
####################################################
# Connection Tuple
# protocol list: 0-TCP / 1-UDP / 2-ANY
# port: 0-65535 (0 for ANY)
# IP: a valid ipv6 address (::ffff:0.0.0.0 for ANY)
####################################################
intIp="::ffff:127.0.0.1"
intPort=0
extIp="::ffff:255.255.255.255"
extPort=8080
protocol=1
####################################################
# DPI Parameters
####################################################
# establish the connection: 0-Parsed / 1-Established
shouldEstablish=0
# DPI connection timeout optimization ( 0 - disabled / 1 - enabled )
special_timeout=1
# predefined servers mode ( 0 - disabled / 1 - enabled )
predefined_servers_enable=1
###################################################
# special tuple parameters
###################################################
 Appendices
behaviour=0
###################################################
# WebAnalytics
###################################################
hdr_enable=1
gcdr_enable=1

Legend
• gcdr_enable – Enable(1) or disable(0) BDRs.
Note: The default configuration is for BDR to be disabled. For customers using the
BDR feature, this flag HAS to be changed to 1.
• operation_mode – 1 is enabled, 0 is disabled. This disables the entire list
of servers.
• extIp / intIp – One of these fields is mandatory. The other is optional.
Therefore, there should be at least one IP address provided here.
• intPort , extPort – Optional.
• protocol – Indicates whether it's TCP (0) or UDP (1) or ANY (2). For BDRs
you should use UDP, 1 , as in the example above.
• shouldEstablish – Must be 0 for BDRs.
• appId – Must be 407 , which is the AppID for "Other IM Applications"
which was altered on the previous instructions.
• hdr_enable – Not relevant, this feature is controlled via the NetXplorer
GUI.
• special_timeout – 1 = Enabled, 0 = disabled. This enables the use of HDR
duration and a real byte count for timeout purposes. After editing this
parameter, restart the DataSrv (using keeperMgr -SW -R DataSrv) or
reboot the host blade.
Note: Use 0 if HDR duration or a real byte count fields are not required.
6. After changing the configuration, please apply using the following
command:
/opt/allot/bin/wa-update.sh <WACONFIG DIRECTORY>
Output Example
# /opt/allot/bin/wa-update.sh /home/sysadmin/WAConfig
Creating a tar file...
genericcdr.xml

Data Mediator Installation and Administration Guide 6-2

 Appendices
webanalytics.xml
webanalytics.xsd
Updating CCs...
Done!
7. This will copy the files in the directory into /opt/allot/conf/WebAnalytics
and apply the configuration.

Configuring BDR Export on the NetXplorer

1. To enable BDR configuration from the NetXplorer GUI, contact Allot
Customer Support.
2. Once BDRs have been enabled in the NetXplorer GUI, select and right-click
the Network in the Navigation tree and select Configuration from the
popup menu.
OR
Select the Network in the Navigation tree and then select Configuration
from the View menu.
OR
Select the Network in the Navigation tree and then click the Configuration
icon on the toolbar.
3. The Configuration window for the Network is displayed.
4. Select the Data Definitions subtab and open Output Profiles.
5. Select your current Output Profile and click Edit
OR
Click Add to create a new Output Profile. Please note that this new profile
will affect all Data Types.
The Data Output Profile dialog will appear.
6. To configure BDRs, select the BDR sub-tab.
7. In the BDR sub-tab, the following fields may be configured.
⧫ File Size Limit (in bytes) – This defines the maximum possible size of
each exported file before compression. The default is 100,000 bytes
(100 MB) and the maximum possible value is 1,000,000,000 bytes (1
GB).
⧫ File Closing Interval (in seconds) – This defines how long after being
created the export file is closed. The default is 300 seconds (5
minutes) and the maximum is 3600 seconds (1 hour).

Data Mediator Installation and Administration Guide 6-3

 Appendices
⧫ File Compression method – This sets the file compression method to
be used when exporting the file. The possible options are GZIP and no
compression. GZIP is selected by default.
⧫ Delete files older than (in minutes) – This defines the amount of time
the system retains files. The default is 1440 minutes (24 hours) and
the maximum is 4320 minutes (72 hours).
⧫ Push Properties – Push properties, once enabled, define how the
export files are sent to their proper locations for analysis and storage.
The four push method options are SFTP, FTP, SCP and Copy. If SFTP is
selected, the Username, Password, Server and Path must be defined.
If SCP is selected, the username and password must be defined. For
FTP and Copy, only the path must be defined. The default is Enabled.
Note: If special characters are used in the Push Properties password (for example
"E69{Gs[V%y" ) the data will not be pushed and you will see a "invalid URI
syntax" error in datamediation.log.

Naming Conventions
BDR files are generated according to the following naming conventions:
<Source unit name>_<Source file IP>_<Mediation IP>_<Seq>_<Date>_< Data
type>_<Version>.[<FE>]

FIELD NAME DESCRIPTION NOTES

< Source unit name > The name of the source unit that Each name must be unique
generated the file
<Source file IP> The IP Address of the file origin.
<Mediation IP> The IP address of the Mediation unit that
created the file
<Seq> A sequencer that is never reset.
<Date> Creation date of the file in the form
yyyymmddhhmmss
<Data type> Represent the file content:
“B40” – for files containing BDR records
<Version> Version of the entry format.
Format: V<format number>
The current version is V1

Data Mediator Installation and Administration Guide 6-4

 Appendices

FIELD NAME DESCRIPTION NOTES

<FE> File extension.
The file extension is bin.gz for BDR files.
While the file is being copied a “.tmp”
suffix will be added. This suffix will be
removed after the copy has been
completed successfully.

Data Mediator Installation and Administration Guide 6-5

 Appendices

6.2 Appendix B - Using the BMC

The Baseboard Management Controller (BMC) is provided with every Allot
appliance based on Lenovo Gen 6 hardware that is shipped from Allot.
The module enables an administrator to connect remotely as if connecting locally
via a console connection.
The BMC interface on the rear of the appliance is detailed below

Figure 6-1: Connection to the BMC on the rear of the SR630 Server

Home

Figure 6-2: BMC, System Status Screen

Open the Home screen to view the status of the server that you are accessing.
From the Home pages, you can:

Data Mediator Installation and Administration Guide 6-6

 Appendices
• Monitor the power status of the server and view the state of the operating
system
• View the server temperature readings, voltage thresholds, and fan speeds
• View the latest server operating-system-failure screen capture
From the Event Log page, you can:
• View certain events that are recorded in the event log of the IMM
• View the severity of events

BMC Configuration
Use the links under BMC Configuration in the toolbar to configure the BMC.
From the Users page, you can:
• Set login profiles to control access to the IMM
• Configure global login settings, such as the lockout period after
unsuccessful login attempts
• Configure the account security level
From the Network page, you can set up the Ethernet connection for the IMM and
you can configure:
• SNMP setup
• DNS setup
• SSH protocol
• SMTP setup
• LDAP setup
• Service location protocol
From the Security page, you can install and configure the Secure Sockets Layer
(SSL) settings.
From the Backup and Restore page, you can back up, modify, and restore the
configuration of the BMC.
Note: Restarting the BMC or changing BMC configuration does not involve the server
operating system or functionality. The server continues to run as is.

6.3 Appendix C – Using the IMM

The Integrated Management Module (IMM) is provided with every Allot appliance
based on IBM/Lenovo Gen 5 hardware that is shipped from Allot.

Data Mediator Installation and Administration Guide 6-7

 Appendices
The module enables an administrator to connect remotely as if connecting locally
via a console connection.
The IMM interface on the rear of the appliance is detailed in Figure 6-3 below

Figure 6-3: Connection to the IMM on the rear of the server

Once Configured, IMM functions are divided into five groups; System Status,
Events, Service and Support, Server Management and IMM Management. These
groups can be accessed from the Menu bar of the IMM Interface.

Data Mediator Installation and Administration Guide 6-8

 Appendices
System Status

Figure 6-4: IMM, System Status Screen

Open the System Status screen to view the status of the server that you are
accessing.
From the System Status pages, you can:
• Monitor the power status of the server and view the state of the operating
system
• View the server temperature readings, voltage thresholds, and fan speeds
• View the latest server operating-system-failure screen capture
From the Event Log page, you can:
• View certain events that are recorded in the event log of the IMM
• View the severity of events

Data Mediator Installation and Administration Guide 6-9

 Appendices
Server Management
Use the functions under the Server Management menu item to directly control the
actions of the IMM and your server. The tasks that you can perform depend on the
server in which the IMM is installed.
You can perform the following tasks:
• View server power and restart activity
• Remotely control the power status of the server
• Remotely access the server console, as if you are located near the server
itself with console cable connected to it
• Remotely attach a disk or disk image to the server, via the remote
connection
Note: When you are performing an action that requires large amount of bandwidth
between the IMM and the server please verify that you have enough
bandwidth available.

IMM Management
Use the links under IMM Management in the toolbar to configure the IMM.
From the Users page, you can:
• Set login profiles to control access to the IMM
• Configure global login settings, such as the lockout period after
unsuccessful login attempts
• Configure the account security level
From the Alerts page, you can:
• Configure remote alert recipients
• Set the number of remote alert attempts
• Select the delay between alerts
• Select which alerts are sent and how they are forwarded
From the IMM Properties page, you can:
• Configure the baud rate of serial port 2 (COM2) for serial redirection
• Specify the keystroke sequence that is used to switch between the serial
redirection and the command-line interface (CLI)
• Change the port numbers of IMM services.

Data Mediator Installation and Administration Guide 6-10

 Appendices
From the Network page, you can set up the Ethernet connection for the IMM and
you can configure:
• SNMP setup
• DNS setup
• SMTP setup
• LDAP setup
• Service location protocol
From the Security page, you can install and configure the Secure Sockets Layer
(SSL) settings.
From the IMM Configuration page, you can back up, modify, and restore the
configuration of the IMM.
From the Reset IMM to Factory Defaults page, you can reset the IMM configuration
to the factory defaults.
From the Restart IMM page, you can restart the IMM.
Note: Restarting the IMM or changing IMM configuration does not involve the
server operating system or functionality. The server continues to run as is.

6.4 Appendix D – Server Memory Upgrades

The Servers provided by Allot may be upgraded to the following memory levels
using the indicated memory modules available from Allot:
For GEN4 Servers:
• 8GB Module – UPG-CSRV-MEM-8G-GEN4
• 16GB Module - UPG-CSRV-MEM-16G-GEN4
For GEN5 Servers:
• 8GB Module – UPG-CSRV-MEM-8G-GEN5
• 32GB Module - UPG-CSRV-MEM-32G-GEN5

PN 64GB 128GB 256GB

PA-MED-SRV- DEFAULT Default plus 8 x 8GB 16 x 16GB
GEN4
8 x 8GB DIMM
PA-MED-SRV-HAP- DEFAULT Default plus 8 x 8GB 16 x 16GB
GEN4
8 x 8GB DIMM (Per Server) (Per Server)

Data Mediator Installation and Administration Guide 6-11

 Appendices

PA-MED-SRV- DEFAULT Default plus 8 x 8GB 8 x 32GB

GEN5
8 x 8GB DIMM
PA-MED-SRV-HAP- DEFAULT Default plus 8 x 8GB 8 x 32GB
GEN5
8 x 8GB DIMM (Per Server) (Per Server)

6.5 Appendix E - Data Mediator Logs

• Datamediation.log – a log providing all system messages at two
configurable levels, info and debug (debug mode will print additional
information in the log).
• App-datamediation.log – a user oriented log, prints messages at 3 levels:
⧫ failure (logging all failures to push to target or pull from source)
⧫ push (logging those files successfully pushed)
⧫ all (logging all files successfully pushed, pulled or created on the Data
Mediator)
An entry in the App-datamediation.log contains the following information:
• Name or address of the source unit
• Name or address of the target unit
• File name
• Transfer start date and time
• Transfer end date and time
• File size in Mbytes
• Number of records in file - for stored or pushed files
• Status – success or failure
• Reason – description of cause of failure

Data Mediator Installation and Administration Guide 6-12

 Appendices

6.6 Appendix F – RTT Working Modes

It is possible to measure RTT traffic in one of two different modes which control the
way RttEstimate fields are reported in CONV, HTTP, CONV-RTS and CONV-RTU
buckets:
• Pure network (default) – In this mode RTT measurements include only the
network RTT (internal side/external side) without the latency that is
introduced by AOS functionality (Such as QoS or Steering).
• Overall network – In this mode RTT measurements include both network
RTT and latency that is introduced by AOS functionality. External RTT
includes delays on outbound traffic due to AOS functionality while Internal
RTT includes delay to inbound traffic.
RTT Measurement starts and stops at different point in the device according to the
Working Mode as follows:

WORKING NETWORK TCP MEASURE START POINT MEASURE STOP POINT

MODE DIRECTION INITIATING
SIDE
Pure Internal Internal Internal port Tx (SYNACK) Internal port Rx (ACK)
network
Internal External Internal port Tx (SYN) Internal port Rx (SYNACK)
External Internal External port Tx (SYN) External port Rx (SYNACK)
External External External port Tx (SYNACK) External port Rx (ACK)
Overall Internal Internal External port Rx (SYNACK) Internal port Rx (ACK)
network
Internal External External port Rx (SYN) Internal port Rx (SYNACK)
External Internal Internal port Rx (SYN) External port Rx (SYNACK)
External External Internal port Rx (SYNACK) External port Rx (ACK)

Setting the Working Mode

To change the RTT Working Mode from Pure Network (the default) to Overall
Network Mode, use the following CLI Command:
go config xml -node dp_rtt_measurement_working_mode -value 1
To change the RTT Working Mode back to Pure Network Mode, enter the following
CLI command:
go config xml -node dp_rtt_measurement_working_mode -value 0

Data Mediator Installation and Administration Guide 6-13

 Appendices

6.7 Appendix G - File Locations

XDRs are generated into the following folders on the Data Mediator.

PATH CONTENT FILES/ FOLDERS

/opt/allot/datamediation/data Dictionary files ChargingApplications.txt

from NX DataDictionary.xml
DeviceIDs.txt
HostGroups.txt
Tubes.txt
ConfigurationDB.txt
DataExportConfig.xsd
DeviceIPs.txt
Services.txt

/opt/allot/datamediation/export Folders with files Binary_CDR

for export
CONV
HTTP_CDR_A
Metadata
MOU
SDR
UDR
VC
VDR
HTTP_CDR

Data Mediator Installation and Administration Guide 6-14

 Appendices

6.8 Appendix H - Server Hardware Specifications

The Servers provided by Allot with purchase of the Data Mediator adhere to the
following specifications:

P/N PA-MED-SRV
Hardware specifications for a standalone DM Server.

Power Supply IBM System x 750W Hotswap Platinum AC Power Supply

Network Connections RJ45

CPU 1 x Intel Xeon Silver 4214

SSDs 2 x 480GB 2.5" 5200 SATA 6GB Hotswap

Memory 2 x 16GB TruDDR4 Memory (2Rx8, 1.2V) 2933MHz RDIMM

P/N PA-MED-SRV-DC
Hardware specifications for a standalone DM Server.

Power Supply IBM System x 48VDC Power Supply

Network Connections RJ45

CPU 1 x Intel Xeon Silver 4214

SSDs 2 x 480GB 2.5" 5200 SATA 6GB Hotswap

Memory 2 x 16GB TruDDR4 Memory (2Rx8, 1.2V) 2933MHz RDIMM

P/N PA-MED-SRV-HAP
Hardware specifications for a DM-HAP Server.

Power Supply IBM System x 750W Hotswap Platinum AC Power Supply

Network Connections RJ45

CPU 2 x Intel Xeon Silver 4214

Data Mediator Installation and Administration Guide 6-15

 Appendices

SSDs 2 x 480GB 2.5" 5200 SATA 6GB Hotswap

Memory 4 x 16GB TruDDR4 Memory (2Rx8, 1.2V) 2933MHz RDIMM

P/N PA-MED-SRV-GEN5
Hardware specifications for a standalone DM Server.

Power Supply IBM System x 750W High Efficiency Platinum AC Power Supply

Minimum Power Input: 0.14 kVA

Maximum Power Input: 0.994 kVA
Network Connections RJ45

Storage 2 x IBM 300GB 15K 6Gbps SAS 2.5" G3HS 512e HDD

Memory 8 x 8GB (64GB total) TruDDR4 Memory (2Rx8, 1.2V) PC4-17000

CL15 2133MHz LP RDIMM
Dimensions Height: 43 mm (1.7 in), width: 429 mm (16.9 in), depth: 734 mm
(28.9 in)
Heat Dissipation Minimum configuration: 461 Btu/hr (135 watts)
Maximum configuration: 4043 Btu/hr (1185 watts)

P/N PA-MED-SRV-HAP-GEN5
Hardware specifications for a DM-HAP Server.

Power Supply IBM System x 750W High Efficiency Platinum AC Power Supply
Minimum Power Input: 0.14 kVA
Maximum Power Input: 0.994 kVA
Network Connections RJ45

Storage 2 x IBM 300GB 15K 6Gbps SAS 2.5" G3HS 512e HDD

Memory 8 x 8GB (64GB total) TruDDR4 Memory (2Rx8, 1.2V) PC4-17000

CL15 2133MHz LP RDIMM
Dimensions Server: Height: 43 mm (1.7 in), width: 429 mm (16.9 in), depth: 734
mm (28.9 in)
Storage: Height: 3.47 in./88.07 mm, width: 18.98 in./482.10 mm,
depth: 19.60 in./497.93 mm

Data Mediator Installation and Administration Guide 6-16

 Appendices

Heat Dissipation Server:

Minimum configuration: 461 Btu/hr (135 watts)
Maximum configuration: 4043 Btu/hr (1185 watts)
Storage: 1,127 (BTU per hour)

Data Mediator Installation and Administration Guide 6-17

 Appendices

Data Mediator Installation and Administration Guide 6-18