AlienVault Unified Security Management Solution

Complete. Simple. Affordable

How to create a data source plugin

Copyright 2014 AlienVault. All rights reserved.

AlienVault, AlienVault Unified Security Management, AlienVault USM, AlienVault Open Threat Exchange, AlienVault OTX, Open Threat Exchange, AlienVault OTX Reputation
Monitor, AlienVault OTX Reputation Monitor Alert, AlienVault OSSIM and OSSIM are trademarks or service marks of AlienVault.

AlienVault Unified Security Management Solution

How to create a data source plugin

CONTENTS
1.

INTRODUCTION ..................................................................................................... 4

2.

TYPES OF DATA SOURCE PLUGINS .................................................................. 4

2.1. Detector Plugins ....................................................................................................... 5
2.2. Monitor Plugins ...................................................................................................... 23

3.

HOW TO CREATE A CUSTOM DATA SOURCE PLUGIN .................................. 23

3.1. Exchange Web SMTP server logs .......................................................................... 24
3.2. Creation of the plugin configuration file exchangews.cfg ........................................ 24
3.3. Create the database file exchangews.sql ............................................................... 26
3.4. Activate data source plugins................................................................................... 26
3.5. Files .local .............................................................................................................. 32

5.

HOW TO USE CUSTOM FUNCTION IN DATA SOURCE PLUGINS .................. 33

APPENDIX A - RECOMMENDATIONS BEFORE CREATING A NEW PLUGIN ......... 35

APPENDIX B - LIST OF DATA SOURCE PLUGINS ................................................... 37

B.1. Database Plugins ................................................................................................... 37
B.2. Log Plugins ............................................................................................................ 37
B.3. Monitor Plugins ...................................................................................................... 38
B.4. Remote Plugins ...................................................................................................... 39
B.5. SDEE Plugins ........................................................................................................ 39
B.6. WMI Plugins ........................................................................................................... 39

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 3 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

1.

INTRODUCTION
The objective of this document is to explain how to create plugins supported by AlienVault
USM.
A plugin is a software component that adds a specific feature to AlienVault USM. Plugins are
used to improve the collection capabilities of the AlienVault Sensors and to indicate to the
system, how to understand and to collect events generated by each application and device.
Sensors receive events from remote hosts using the Syslog, WMI or any other protocols. The
sensors use the Collection Plugins (also called Data Source connectors) in order to support the
maximum possible number of applications and devices.
For any system that consumes logs, it is needed a parser to read those logs and extract
information from them into standard information fields (username, IP addresses, etc.).
AlienVault does this via Agent plugin that defines how to collect events from the application or
device as well as how events should be normalized before sending them to the AlienVault
USM central Server. Log Normalization is essentially breaking down a log message into
common fields.
It is necessary to enable a plugin in order to indicate to the system that must collect events
generated by an application or device. Plugins may be pre-configured by AlienVault or defined
by users.
AlienVault plugins are text configuration files and have the extension *.cfg. These files are
located in /etc/ossim/agent/plugins in the Sensors file system.

2.

TYPES OF DATA SOURCE PLUGINS

There are 2 types: monitor and detector:
Detector. These plugins receive logs, information and extract events from them. They
process text log information from log files created by RSyslog collection system; and from
log data retrieved from remote systems via one of the remote collection protocols such as
SDEE and SFTP. These plugins can be:
Database. They monitor a file in external databases.
Logs. They monitor a file, usually receiving data through syslog.
Remote Logs. They monitor a file in a remote appliance.
SDEE (Security Device Event Exchange). CISCO device logs.
WMI (Windows Management Instrumentation). They collect remotely Microsoft
Windows events and data in an agent-less way.
Monitor. These plugins request information from systems, checking the status of the things
they monitored at the time of the request. They generate text logs that are fed into the

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 4 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

syslog like normal logs and they are often used to correlate log events into alarms by
matching events against the current status of systems.

2.1.

Detector Plugins
DATABASE PLUGINS

2.1.1.

It is easier to understand how this type of plugin works by means of an example:

;; PCI Trace

[DEFAULT]
plugin_id=1698

[config]
type=detector
enable=yes

source=database
source_type=mssql
source_ip=
source_port=3306
user=
password=
db=
sleep=60

process=
start=no
stop=no

[start_query]

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 5 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

query="select TOP 1 pci.RowNumber from pcitrace as pci ORDER BY

pci.RowNumber desc"
regexp=
userdata1={$3}
log= Virus {$3} detected on {$2}, path: {$4} {$5}

[query]
query="select pci.RowNumber, pci.EventClass, pci.TextData,
pci.ApplicationName, pci.NTUserName, pci.LoginName, pci.CPU, pci.Reads
from pcitrace as pci ORDER BY pci.RowNumber"
regexp=
ref=0
plugin_sid=1
username={$5}
userdata1={$2}
userdata2={$3}
userdata3={$4}
userdata4={$6}
userdata5={$7}
userdata6={$8}
log={$1},{$2},{$3},{$4},{$5},{$6},{$7},{$8}

The fields related to database fields are an example for mssql. In case of having mysql, it must
be indicated.
Indicate the point to start to capture. It must be a query to obtain the last event identified by a
sequence number. In this case will be:
select TOP 1 pci.RowNumber from pcitrace as pci ORDER BY pci.RowNumber
desc

The last RowNumer is obtained from a table.

A query for getting all values is needed. The same field used in the start query must be
selected as first element.
query="select pci.RowNumber, pci.EventClass, pci.TextData,
pci.ApplicationName, pci.NTUserName, pci.LoginName, pci.CPU, pci.Reads

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 6 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

from pcitrace as pci ORDER BY pci.RowNumber"

Regexp field must be empty:

regexp=
ref=0
plugin_sid=1

$2 is the second element in the query. In this example is the value of pci.EventClass
username={$5}
userdata1={$2}
userdata2={$3}
userdata3={$4}
userdata4={$6}
userdata5={$7}
userdata6={$8}
log={$1},{$2},{$3},{$4},{$5},{$6},{$7},{$8}

2.1.2.

LOGS
This is an example of a log plugin:
[DEFAULT]
plugin_id=1563

[config]
enable=yes
type=detector

source=log
location=/var/log/optenet.log

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 7 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

create_file=false

process=
start=no

; launch plugin process when agent starts

stop=no

; shutdown plugin process when agent stops

startup=
shutdown=

[optenet - spam detected]

regexp="^(?P<domain>\S+)\t(?P<SRC_IP>\IPV4)\t*\[\S+\]*\t\[(?P<src_mail>\S+)\]
\t\[(?P<dst_mail>\S+)\].*"
event_type=event
plugin_sid=1
device={resolv($SRC_IP)}
src_ip={resolv($SRC_IP)}
userdata1={$domain}
userdata2={$src_mail}
userdata3={$dst_mail}

Plugins extract events (SIDs) from logs by matching each line in the log according to a
regular expression, and then normalizing out data fields from the text. So when the
following log message arrives:
Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from
192.168.6.69 port 33992 ssh2

It matches the following SID from the SSH plugin.

[01 - Failed password]

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 8 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

The information in a log entry to be normalized into field is specified in the regular
expression1:
regexp="(?P<date>\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<dst>\S+).*ssh.*Failed
(?P<type>publickey|password|none) for\s+(?P<info>invalid
user)?\s*(?P<user>\S+)\s.*from\s+(?P<src>\S+)\s.*port\s+(?P<sport>\d{1,5})"

And these values are normalized out of it:

Date = Feb 8 10:09:06
src_ip =192.168.6.69
Username = dgil

The level of information that can be extracted from a log source is dependent on the level
of detail in the plugin. The more SIDs defined, the greater the ability to extract meaning
from processed logs.

REMOTE LOGS

2.1.3.

This is an example of a remote log plugin:

# Alienvault plugin
# Author: Alienvault Team at [email protected]
# Plugin ssh-remote id:4003 version: 0.0.1
# Last modification: 2013-06-05 11:43
#
# Accepted products:
# openbsd - openssh 5.4
# openbsd - openssh 5.5
# openbsd - openssh 5.6

The bolded fields in the regexp indicate that the matching text will be mapped to information fields during
normalization.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 9 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

# openbsd - openssh 5.7

# openbsd - openssh 5.8
# openbsd - openssh 5.8p2
# openbsd - openssh 5.9
# Description:
#
# Ssh (Secure Shell) is a program for logging into a remote machine
# and for executing commands on a remote machine.
# URL: http://www.openssh.com
#
#

$Id: ssh.cfg,v 1.12 2010/03/23 16:42:18 juanmals Exp $

#
#

[DEFAULT]
plugin_id=4003

dst_ip=\_CFG(plugin-defaults,sensor)
dst_port=22

[config]
type=detector
enable=yes

source=remote-log
location=/var/log/auth.log

create_file=false

process=sshd

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 10 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

start=no
stop=no
startup=/etc/init.d/ssh start
shutdown=/etc/init.d/ssh stop

host=
user=root
passwd=
readAll=false

[ssh - Failed password]

event_type=event
regexp="(\SYSLOG_DATE)\s+(?P<sensor>[^\s]*).*?ssh.*?Failed password for
(?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*?port\s+(?P<sport>\PORT)"
plugin_sid=1
device={resolv($sensor)}
date={normalize_date($1)}
src_ip={$src}
dst_ip={resolv($sensor)}
src_port={$sport}
username={$user}

Entries marked in bold must always appear because they are used for connecting to remote
host:
source=remote-log
host=
user=root
passwd=
readAll=false

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 11 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

SDEE (SECURITY DEVICE EVENT EXCHANGE)

2.1.4.

SDEE 2 is a standard that specifies the format of messages and protocol used to
communicate events generated by security devices. This protocol is used in the Cisco
Systems IPS Sensor 5.0. AlienVault support this type of logs collection. AlienVault USM
captures events from:
Cisco Network Prevention Systems (IPS)
Cisco Network Detection Systems (IPS)
Cisco Switch IDS
Cisco IOS routers with the Inline Intrusion Prevention System (IPS) functions
Cisco IDS modules for routers
Cisco PIX Firewalls
Cisco Catalyst 6500 Series firewall service modules (FWSMs)
Cisco Management Center for Cisco security agents
CiscoWorks Monitoring Center for Security servers
If you have your own update package from your vendor, you can populate the AlienVault
database with the new signatures.
Go to /usr/share/ossim/scripts/ to update the plugin sid information:
python createCiscoIPSSidmap.py IOS-S416-CLI.pkg.xml
DELETE FROM plugin WHERE id = "1597";
DELETE FROM plugin_sid where plugin_id = "1597";
INSERT INTO plugin (id, type, name, description) VALUES (1597, 1, 'CiscoIPS', 'Cisco Intrusion Prevention System');
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 5986, NULL, NULL, 'Cisco-IPS:
Microsoft GDI GIF Parsing Vulnerability', 3, 4);

This protocol is used in the Cisco Systems IPS Sensor 5.0 to replace Remote Data Exchange Protocol
(RDEP).

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 12 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,

priority, reliability) VALUES (1597, 5984, NULL, NULL, 'Cisco-IPS: IE COM
Object Code Execution', 3, 4);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 5985, NULL, NULL, 'Cisco-IPS:
Quicktime RTSP Content-Type Excessive Length', 3, 4);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 19159, NULL, NULL, 'Cisco-IPS: Green
Dam Youth Escort Software Update Check', 1, 4);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 19401, NULL, NULL, 'Cisco-IPS:
Microsoft Publisher File Parsing Vulnerability', 3, 4);

This is script generates the needed SQL information to update AlienVault database. Write
the following to insert information:
python createCiscoIPSSidmap.py IOS-S416-CLI.pkg.xml > sdee.sql
ossim-db < sdee.sql

If you want to update cross-correlation information:

python ciscoIPSOsMap.py IOS-S416-CLI.pkg.xml
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 2156, 3001, 1);
replace into plugin_reference values (1597, 2157, 3001, 3);
replace into plugin_reference values (1597, 2157, 3001, 3);
replace into plugin_reference values (1597, 2157, 3001, 3);
...
...

python ciscoIPSOsMap.py IOS-S416-CLI.pkg.xml > sdee-os.sql

ossim-db < sdee-os.sql

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 13 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

Do not forget to restart ossim-server in order to update AlienVault Server

cache.

Follow the instructions below to configure AlienVault Agent and collect events from SDEE
capable device:
1.

Add SDEE reference to this file: /etc/ossim/agent/config.cfg

2.

Edit this file: /etc/ossim/agent/plugins/cisco-ips.cfg

[DEFAULT]
plugin_id=1597

[config]
type=detector
enable=yes

source=sdee
source_ip=
user=
password=
sleep=5

process=
start=no
stop=no

3.

Insert the credentials: your source_ip, user and password data.

4.

Restart AlienVault Agent for receiving data from SDEE device.

Keep in mind the following points:

Each time a new session begins with a SDEE device, a Subscription ID will be provided. If
the device closes the connection or the connectivity is lost, you have to close the session

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 14 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

in order to continue collecting from the device. The AlienVault Agent closes the session
automatically, but if not, you should do it manually.
The latest Subscription ID can be found here: /etc/ossim/agent/sdee_sid.data
Execute the following:
python /usr/share/ossim/scripts/closeSDEEsession.py SubscriptionID

This closes the last session. If you still have problems, execute the following:
grep subs /var/log/ossim/agent.log

The agent debugging can also turn on, stopping the current agent and starting it manually
on verbose mode:
ossim-agent -v

You should get something like this:

2012-05-07 05:15:40,925 Agent [DEBUG]: <?xml version="1.0" encoding="UTF8"?><env:Envelope xmlns="http://www.cisco.com/cids/2006/08/cidee"
xmlns:env="http://www.w3.org/2003/05/soap-envelope"
xmlns:sd="http://example.org/2003/08/sdee"
xmlns:cid="http://www.cisco.com/cids/2006/08/cidee"><env:Header><sd:oobInfo><
sd:sessionId>373397c2f80a792a4029fbcc0cd027e5</sd:sessionId><sd:remainingevents>0</sd:remainingevents></sd:oobInfo></env:Header><env:Body><sd:events></sd:events></env:Body>
</env:Envelope>

2.1.5.

WMI (WINDOWS MANAGEMENT INSTRUMENTATION)

They collect remotely Microsoft Windows events and data in an agent-less way.
This is an example of a WMI plugin:
[DEFAULT]
plugin_id=1518

[config]
type=detector

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 15 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

enable=yes

source=wmi
credentials_file=/etc/ossim/agent/wmi_credentials.csv
sleep=10

process=
start=no
stop=no

[start_cmd]
cmd=wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select
LogFile,RecordNumber from Win32_NTLogEvent Where Logfile = 'Application'" |
head -n 3 | tail -n 1 | cut -f 2 -d \|
regexp=

[cmd]
cmd = wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select
ComputerName,EventCode,Logfile,Message,RecordNumber,SourceName,TimeWritten,Us
er from Win32_NTLogEvent Where Logfile = 'Application' and RecordNumber >
OSS_COUNTER" | cat
start_regexp=^([^\|]+)\|(\d+)\|([^\|]+)\|
regexp="^(?P<system_name>[^\|]+)\|(?P<plugin_sid>\d+)\|(?P<logfile>[^\|]+)\|(
?P<message>[^\|]+)\|(?P<recordnumber>[^\|]+)\|(?P<sourcename>[^\|]+)\|(?P<tim
ewritten>[^\|]+)\|(?P<username>.*)$"
src_ip={resolv($0)}
plugin_sid={$1}
userdata2={$2}
userdata3={$3}
userdata4={$4}
userdata5={$5}
userdata6={$6}
username={$7}

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 16 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

2.1.5.1.

DC-00138

PREPARING WINDOWS
1.

Create a new limited user for not using an administrator account for remote
connections and make the installation much more secure.

2.

For this example, the user wmiuser and password wmi have been created.

3.

Configure DCOM to allow the user access to the computer remotely.

4.

Grant remote launch to DCOM and activation permissions for our user:
a)

Run Dcomcnfg by selecting Run on the Start menu and typing in Dcomcnfg. Then
click OK.

b)

Open Administrative Tools 3 and expand Component Services. Click the

secondary button of the mouse over My Computer and select Properties.

This option is not always in the same place. It depends on the Windows version.

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 17 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

c)

DC-00138

Edition 03

Click on COM Security tab. Then click on Edit Limits under Access
Permissions:

Copyright 2014 AlienVault. All rights reserved.

Page 18 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

d)

DC-00138

Edition 03

Click on Anonymous Logon and then on Remote Access:

Copyright 2014 AlienVault. All rights reserved.

Page 19 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

DC-00138

e)

Click OK.

f)

Click Apply.

g)

Click OK.

5.

Run Dcomcnfg by selecting Run on the Start menu and typing in Dcomcnfg. Then
click OK.

6.

Open Administrative Tools4 and expand Component Services. Click the secondary
button of the mouse over My Computer and select Properties.

7.

Click on COM Security tab and then click on Edit Limits under Launch and
Activation Permissions.

This option is not always in the same place. It depends on the Windows version.

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 20 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

8.

DC-00138

Click Add button.

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 21 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

9.

Enter the user name and click OK.

10. In the Launch and Activation Permisson screen, click the options: Remote Launch,
Local Activation and Remote Activation. Then click OK.

11. Click OK.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 22 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

12. Click Apply.

13. Click OK.

2.1.5.2.

CONFIGURING ALIENVAULT USM

Before the activation of wmi plug-ins, it is necessary to create a file having Windows IPs
and credentials.
1.

Create a wmi_credentials.csv file.

vim /etc/ossim/agent/wmi_credentials.csv

2.

Add ips, users and password with following formats:

127.0.0.1,user,pass
127.0.0.2,domain/user, pass
127.0.0.3,domain/user ,pass

2.2.

Monitor Plugins
These plugins are used to execute actions in sensors in correlation time through directives. For
instance, the 2005 plugin monitor is used in these 2 files:
ntop-monitor.cfg
session-monitor.cfg

This is used from monitor directives like:

/etc/ossim/server/alienvault-attacks.xml: <rule type="monitor" name="AV
More than 10 secs persistence" reliability="5" from="1:SRC_IP"
to="1:DST_IP" port_from="1:SRC_PORT" port_to="1:DST_PORT"
plugin_id="2005" plugin_sid="248" condition="ge" value="10" interval="15"
time_out="30" absolute="true">

3.

HOW TO CREATE A CUSTOM DATA SOURCE PLUGIN

This section explains how to create a custom plugin to process Exchange Web Server logs
through the SIEM engine.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 23 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

3.1.

EXCHANGE WEB SMTP SERVER LOGS

The log file used for the following hands on exercise can be downloaded from here:
exchangews.log

Once the file has been downloaded, open it to see the logs we are going to parse. Here are
some sample lines:
2011-10-09 05:00:19 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 QUIT
- 36A42160 240 6219 68 4 0 SMTP - - - 1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP"
250 46

3.2.

CREATION OF THE PLUGIN CONFIGURATION FILE EXCHANGEWS.CFG

1.

Global plugin configuration settings:

Copy the file ssh.cfg and name this new one as exchangews.cfg.
Change the plugin id field (use 9001 as it is part of the user range that goes up to
10000).
Change location to point to the log file /var/log/exchangews.log
Delete startup and shutdown fields. These fields are not going to be used (there is no
application associated with this plugin).
Create a new translation table5, as it is shown below. This step is optional:

HELO=1
MAIL=2
RCPT=3
DATA=4
QUIT=5

A translation table is used for translating a string to a number in order to use it as plugin_sid. It is necessary to
include the function {translate($field_to_translate)} for using a translation table.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 24 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

xxxx=6
DEFAULT_=9999

2.

Create new rules, filling up the fields below. Create two regular expressions to parse the
data, because there are two different formats in the log file.

[exchangews - Generic rule]

#2011-10-09 05:00:15 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 HELO +36A42160 250 0 48 13 0 SMTP - - - #2011-10-09 05:00:16 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 MAIL +FROM:+<[email protected]> 250 0 57 45 0 SMTP - - - event_type=event
regexp="(?P<date>\d{4}-\d{2}\d{2}\s\d{2}:\d{2}:\d{2})\s(?P<src_ip>\IPV4)\s(?P<userdata2>\S+)\s(?P<hostnam
e>\S+)\s(?P<userdata3>\S+)\s(?P<dst_IP>\IPV4)\s\d\s(?P<type>\w+)"
date={normalize_date($date)}
plugin_sid={translate($type)}
dst_ip={resolv($dst_ip)}
src_ip={resolv($src_ip)}
hostname={$hostname}
userdata2={$userdata2}
userdata3={$userdata3}
[exchangews = Generic rule 2 NCSA Format]
#1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP" 250
46
#1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:41 -0600] "MAIL ?+FROM:+<[email protected]> SMTP" 250 46
event_type=event
regexp="(?P<src_ip>\IPV4)\s\s(?P<dst_ip>\S+)\s\[(?P<date>\d\d\/\w{3}\/\d{4}:\d\d:\d\d:\d\d)\s\d{4}\]\s\"(?P<type>\w+)"
date={normalize_date($date)}
plugin_sid={translate($type)}
dst_ip={resolv($dst_ip)}

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 25 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

src_ip={resolv($src_ip)}

3.

3.3.

Check regular expressions with logs inside the file /var/log/exchangews.log. There are
several utilities on the Internet to test regular expressions written in Python. It is
recommended to use one of these utilities to check that the created regular expressions
match the logs.

CREATE THE DATABASE FILE EXCHANGEWS.SQL

1.

Create a file using the following examples:

INSERT INTO plugin (id, type, name, description) VALUES (9001, 1,

'exchangews', 'Exchange E-mail Web server');
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (9001, 1, NULL, NULL, 'exchangews: HELO'
,3, 2);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (9001, 9999, NULL, NULL, 'exchangews:
Generic exchange event' ,3, 2);

2.

Insert file values into the database in the server box.

cat exchangews.sql | ossim-db

3.

Apply changes in SIEM.

/etc/init.d/ossim-server restart

3.4.

ACTIVATE DATA SOURCE PLUGINS

Choose one of the following options to activate plugins:
Through command line console
Through web

3.4.1.

ACTIVATE PLUGINS THROUGH COMMAND LINE CONSOLE

To activate plugins by a console, the instructions below should be followed:

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 26 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

1.

Open a console terminal application and connect to the AlienVault System by running
the following command:

ssh root@IP_address

IP_address refers to the default IP of your appliance.

2.

Next, it appears a screen which includes the main menu:

3.

By using the arrow keys on the keyboard, select the option Configure Sensor. Accept
the selection (<OK>) by pressing Enter key.
Use the Tab key on the keyboard to move from <OK> to <Exit> or vice versa:

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 27 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

DC-00138

4.

Select the option Configure Data Source Plugins. Accept the selection (<OK>) by
pressing Enter key.

5.

Select the plugins to activate. To move between them use the arrow keys on the
keyboard and select/deselect it by pressing the Space Bar on the keyboard. Accept
the selection (<OK>) by pressing Enter key. It is possible to select several plugins.
Accept the selection (<OK>) by pressing Enter key.

6.

The Configure Sensor window appears. Move from <OK> to <Back> by using the
Tab key on the keyboard. Back to the AlienVault Setup Screen.

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 28 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

DC-00138

7.

Select the option Apply all changes. Accept the selection (<OK>) by pressing Enter
key.

8.

Apply all changes (<Yes>) by pressing Enter key.

9.

The process can take several minutes depending on the number of plugins to activate:

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 29 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

10. Once the process finishes, the following screen appears:

11. Press Enter key. The AlienVault Setup screen appears.

12. Move from <OK> to <Exit> by using the Tab key on the keyboard. Press Enter key.

3.4.2.

ACTIVATE PLUGINS BY WEB

To activate plugins by the web, the instructions below should be followed:

DC-00138

1.

Use a web browser access to your AlienVault console at https://your_ip/

2.

Write a valid IP address in the navigation bar of a web browser.

3.

Enter a valid user name and password and click on Login.

4.

Navigate to Configuration > Deployment:

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 30 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

DC-00138

5.

The following window appears:

6.

Click on one of the Node Name then, on Sensor Configuration link, and finally on
Collection link. A table appears:

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 31 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

This table displays 2 columns. The left column shows plugins that are enabled and the
right column shows plugins that are available to be enabled.
To pass an item from one side to the other, drag and drop the item or use the links [+]
or [-] which are next to each item.
7.

3.5.

To make all changes take effect, click the APPLY CHANGES button.

FILES .LOCAL
Whenever a plugin file is going to be changed, it is recommended to copy the filename.cfg into
another file named filename.cfg.local. Make all necessary changes in the .local file and keep
the .cfg file without any change. By copying the file, you preserve the original plugin version
over the updated version. The new updates will never overwrite your plugin customization.

Keep in mind that the original plugins can be modified by AlienVault when an
update process has been done.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 32 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

5.

HOW TO USE CUSTOM FUNCTION IN DATA SOURCE PLUGINS

The instructions below should be followed:
1.

Insert a custom function field in the plugin .cfg file:

[config]
custom_functions_file=/etc/ossim/agent/plugin/ssh_custom_functions.cfg

2.

Create a function file having in mind that a function must start with Start Function
<func name> and must end with End function:

Start Function

log_hello

def log_hello(self):
return "Hello log!"
End Function

Start Function

log_hello_data

def log_hello_data(self,data):
return "Hello log: %s" % data
End Function

3.

Edit the plugin rules to use the function by using two points:

[0021 - ssh - Failed password]

# Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from
192.168.6.69 port 33992 ssh2
event_type=event
regexp="(\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<sensor>\S*).*ssh.*Failed
password for
(?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*port\s+(?P<sport>\d{1,5})"
plugin_sid=1
sensor={resolv($sensor)}
date={normalize_date($1)}
src_ip={$src}

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 33 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

dst_ip={resolv($sensor)}
src_port={$sport}
username={$user}
userdata1={:log_hello()}
userdata2={:log_hello_data($user)}

It is not possible to apply a built in function to a custom function (for

instance, translate(:log_hello()) ), as the last function which are executed are
the custom ones.
So translate will receive :log_hello() as a simple chain of chars.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 34 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

APPENDIX A - RECOMMENDATIONS BEFORE CREATING A NEW

PLUGIN
Follow the following recommendations before creating a new plugin:
Copy a log as big as possible.
Extract events from the log using consecutive grep -v, until the command does not return
anything.
Use grep to check individually every event. Try to find different values that a same event
may take.
Discard repeated events.
Look for event patterns in order to group them using some identifier such as the same field
distribution, for instance.
Take into account that your target will be identifying individual events using a plugin_sid,
you may need to think what translates you will be using.
For every event, find out the number of times it is repeated within the log, using the
following command to count the lines.
wc -l

Think if it's worth using a single regex for an event or if several can be grouped together
without making the regex very complex.
Only capture the fields that are going to be used in correlation later on.
Create a rule with a generic regex at the end to capture any remaining event.
Choose the right pre-check, keeping in mind that it applies a first filter to the events.
Make sure the rules are alphabetically ordered, starting with 0001 and finishing with 9999,
creating 0002, 0003... groups, leaving room for future expressions.
The rules are loaded and applied in alphabetical order, so events captured by a rule will
not be processed by the rules loaded after that one. Rule order must be chosen carefully
to avoid event masking due to generic rules being loaded before specific ones.
The SQL does not need the sids to be correlative. Gaps can be left in order to make it
more maintainable.
Lets say from 1000 to 1999 for A event types, from 2000 to 2999 for B event types, etc.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 35 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

Be careful if you add a custom function into a plugin; or if you access to a proprietary
database. This may deteriorate the performance if it is not well designed.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 36 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

APPENDIX B - LIST OF DATA SOURCE PLUGINS

B.1. Database Plugins
drupal-wiki.cfg

eljefe.cfg

forensics-db-1.cfg

mcafee-epo.cfg

moodle.cfg

motion.cfg

oracle-sql.cfg

panda-se.cfg

post_correlation.cfg

vmware-vcenter-sql.cfg

B.2. Log Plugins

airlock.cfg

aix-audit.cfg

aladdin.cfg

allot.cfg

alteonos.cfg

amun-honeypot.cfg

apache.cfg

apache-syslog.cfg

arpalert.cfg

arpwatch.cfg

artemisa.cfg

aruba.cfg

aruba-6.cfg

ascenlink.cfg

avast.cfg

axigen-mail.cfg

bind.cfg

bit9.cfg

bluecoat.cfg

bro-ids.cfg

cisco-3030.cfg

cisco-ace.cfg

cisco-acs.cfg

cisco-acs-idm.cfg

cisco-asa.cfg

cisco-asr.cfg

cisco-fw.cfg

cisco-ids.cfg

cisco-ips-syslog.cfg

cisco-nexus-nx-os.cfg

cisco-pix.cfg

cisco-router.cfg

cisco-vpn.cfg

cisco-wlc.cfg

citrix-netscaler.cfg

clamav.cfg

clurgmgr.cfg

courier.cfg

cyberguard.cfg

dhcp.cfg

dionaea.cfg

dovecot.cfg

dragon.cfg

enterasys-rmatrix.cfg

exchange.cfg

extreme-switch.cfg

extreme-wireless.cfg

f5.cfg

f5-firepass.cfg

fidelis.cfg

fortigate.cfg

fortiguard.cfg

fortimail.cfg

fw1-alt.cfg

fw1ngr60.cfg

gfi.cfg

glastopng.cfg

heartbeat.cfg

honeyd.cfg

hp-eva.cfg

iis.cfg

impervasecuresphere.cfg

intrushield.cfg

ipfw.cfg

iphone.cfg

iptables.cfg

ironport.cfg

isa.cfg

juniper-srx.cfg

juniper-vpn.cfg

kismet.cfg

linuxdhcp.cfg

lucent-brick.cfg

m0n0wall.cfg

mcafee.cfg

mcafee-antispam.cfg

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 37 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

modsecurity.cfg

monit.cfg

motorola-firewall.cfg

mwcollect.cfg

nagios.cfg

nepenthes.cfg

nessus.cfg

nessus-detector.cfg

netgear.cfg

netkeeper-fw.cfg

netkeeper-nids.cfg

netscreen-firewall.cfg

netscreen-igs.cfg

netscreen-manager.cfg

netscreen-nsm.cfg

nfs.cfg

nortel-switch.cfg

ntsyslog.cfg

openldap.cfg

optenet.cfg

oracle-syslog.cfg

osiris.cfg

ossec.cfg

ossec-idm.cfg

ossec-idm-single-line.cfg

ossec-single-line.cfg

ossim-agent.cfg

p0f.cfg

pads.cfg

paloalto.cfg

pam_unix.cfg

panda-as.cfg

pf.cfg

postfix.cfg

prads.cfg

prads_eth0.cfg

proxim-orinoco.cfg

pureftpd.cfg

radiator.cfg

radware-ips.cfg

raslogd.cfg

realsecure.cfg

rrd.cfg

rsa-secureid.cfg

sap.cfg

sendmail.cfg

serviceguard.cfg

shrubbery-tacacs.cfg

sidewinder.cfg

siteprotector.cfg

siteprotector-snmp.cfg

sitescope.cfg

smbd.cfg

snare.cfg

snare-idm.cfg

snare-mssql.cfg

snare-msssis.cfg

snort_syslog.cfg

sonicwall.cfg

sophos.cfg

spamassassin.cfg

squid.cfg

squidGuard.cfg

ssh.cfg

stonegate.cfg

stonegate_ips.cfg

storewize-V7000.cfg

sudo.cfg

suhosin.cfg

suricata-http.cfg

symantec-ams.cfg

symantec-epm.cfg

syslog.cfg

tacacs-plus.cfg

tarantella.cfg

tippingpoint.cfg

token-rsa.cfg

trendmicro.cfg

usbudev.cfg

vandyke-vshell.cfg

vmware-esxi.cfg

vmware-vcenter.cfg

vmware-workstation.cfg

vplus.cfg

vsftpd.cfg

vyatta.cfg

W2003DNS.cfg

watchguard.cfg

webmin.cfg

websense.cfg

wuftp.cfg

B.3. Monitor Plugins

The following plugins are monitor plugins:

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 38 of 39

AlienVault Unified Security Management Solution

How to create a data source plugin

nessus-monitor.cfg

nmap-monitor.cfg

ntop-monitor.cfg

ocs-monitor.cfg

opennms-monitor.cfg

ossim-monitor.cfg

ping-monitor.cfg

session-monitor.cfg

tcptrack-monitor.cfg

whois-monitor.cfg

wmi-monitor.cfg

B.4. Remote Plugins

ssh-remote.cfg

B.5. SDEE Plugins

cisco-ips.cfg

B.6. WMI Plugins

wmi-application-logger.cfg

wmi-security-logger.cfg

wmi-security-logger-srv2008.cfg

wmi-system-logger.cfg

This plugin is used by certain directives to ascertain whether a DOS or DDoS attack was successful. See
directive 34031 for use-case.

DC-00138

Edition 03

Copyright 2014 AlienVault. All rights reserved.

Page 39 of 39