AlienVault Building Collector Plugins

Building Collector Plugins

Admin Guide

Copyright © Alienvault 2010

All rights reserved. No part of this work may be reproduced or transmitted in any form or by

any means, electronic or mechanical, including photocopying, recording, or by any

information storage or retrieval system, without the prior written permission of the

copyright owner and publisher.

Any trademarks referenced herein are the property of their respective holders.

Building Collector Plugins - Admin Guide


Table of Content

1 Overview ..................................................................................................................................................... 4

1.1 OSSIM Agent Role ............................................................................................................................... 4

1.1.1 Event Collection .......................................................................................................................... 4

1.1.2 Event Normalization ................................................................................................................... 4

1.2 OSSIM Server Role .............................................................................................................................. 6

1.2.1 Event Enrichment ....................................................................................................................... 6

1.2.2 Policies and Actions .................................................................................................................... 7

1.3 The Configuration Workflow .............................................................................................................. 8

2 Configuring Detector Plugins .................................................................................................................... 10

2.1 Rsyslog .............................................................................................................................................. 10

2.1.1 Configuration File ..................................................................................................................... 10

2.1.2 Listener Configuration .............................................................................................................. 10

2.1.3 Filters ........................................................................................................................................ 10

2.2 OSSIM Agent Configuration .............................................................................................................. 11

2.2.1 Configuration File ..................................................................................................................... 11

2.2.2 Parameters ............................................................................................................................... 11

2.3 Detector Plugin Configuration .......................................................................................................... 13

2.3.1 Configuration Files .................................................................................................................... 13

2.3.2 Common Event Types ............................................................................................................... 13

2.3.3 Parameters ............................................................................................................................... 13

2.3.4 Using Local (Plugin) Variables ................................................................................................... 15

2.3.5 Using Global (Agent) Variables ................................................................................................. 15

2.4 Aliases ............................................................................................................................................... 16

2.4.1 Path ........................................................................................................................................... 16

2.4.2 Predefined Regular Expressions ............................................................................................... 16

2.5 Functions .......................................................................................................................................... 16

2.5.1 Path ........................................................................................................................................... 16

2.5.2 Conversions .............................................................................................................................. 16

2.5.3 Application Specific Translations .............................................................................................. 17

2.5.4 User Defined Translations ........................................................................................................ 17

2.6 Event Fields ....................................................................................................................................... 18

2.7 Rules ................................................................................................................................................. 19

2.7.1 Evaluation Order....................................................................................................................... 19



2.7.2 Structure ................................................................................................................................... 19

2.8 Loading Plugins ................................................................................................................................. 21

2.8.1 Priority and Reliability values ................................................................................................... 21

2.8.2 SQL Statement .......................................................................................................................... 21

2.9 Plugin Activation ............................................................................................................................... 22

2.9.1 Activate the Plugin on the Server Side ..................................................................................... 22

2.9.2 Activate the Plugin on the Agent Side ...................................................................................... 22

3 Log files ..................................................................................................................................................... 22

4 Debugging ................................................................................................................................................. 22

5 Appendix ................................................................................................................................................... 23

5.1 Regular Expressions .......................................................................................................................... 23

5.2 Configuration Example ..................................................................................................................... 25

5.2.1 Scenario .................................................................................................................................... 25

5.2.2 Write a script to monitor the “last” status ............................................................................... 25

5.2.3 Log sample ................................................................................................................................ 25

5.2.4 Collect the logs in a new log file ............................................................................................... 25

5.2.5 Restart “rsyslog” ....................................................................................................................... 26

5.2.6 Check whether the new entries are written in the new log file ............................................... 26

5.2.7 Create a plugin file .................................................................................................................... 26

5.2.8 Register the Plugin with the OSSIM Agent ............................................................................... 29

5.2.9 Register the Plugin with the OSSIM Server .............................................................................. 30

5.2.10 Check whether the plugin was successfully registered ............................................................ 31

5.2.11 Restart the OSSIM Server ......................................................................................................... 31

5.2.12 Restart the OSSIM Agent .......................................................................................................... 31

5.2.13 Check whether Events and Alarms are received ...................................................................... 32



1 Overview

1.1 OSSIM Agent Role

1.1.1 Event Collection

The collection process involves extracting the data logs from the source systems (Security, OS,

RDBMS, etc.) and allows first steps for event log filtering. At this stage can be decided what is going

to be read by the OSSIM Agent and what is going to be discarded before having an impact on the

system performance.

Before starting to write a plugin some actions to reduce the amount of events could be considered:

- Manage the log level settings at the application and managed device level

- Fix the problem that is generating events

- Use Pcap filters to ignore certain hosts or networks (Snort, Tcpdump...)

- In deployments with a big amount of analysed data, filtering at the application level

should be done whenever possible

o Log Files

Good practice is to use one log file per plugin in order to increase performance. Having just

one generic log file, all the plugins would have to read the same extensive content in order

to catch the few relevant entries.

Using rsyslog it is possible to filter the collected logs based on the syslog tags.

1.1.2 Event Normalization

In the normalization stage a series of rules or functions applies to the data extracted from the source

system in order to transform it in a common OSSIM format.

o Raw Event

The raw event might be a generic syslog message, an application log, an SNMP trap, the

result of an SNMP or SQL Query or some other kind of information in a more or less

structured form that is appended to a log file.

Example:

dmz01:/var/log/auth.log:

May 30 13:15:52 dmz01 sshd[12980]: Accepted password for root from

192.168.178.20 port 4445 ssh2



o Normalized Event

There is a certain set of fields which are required in order to ensure a consistent evaluation

and correlation of the events by the OSSIM server. These fields can be populated with

information from the log message or statically through the plug-in.

Example:

ossim-sensor:/var/log/ossim/agent.log:

2010-05-30 13:15:49,441 Output [INFO]: event type="detector" date="1275239752"

sensor="192.168.178.201" interface="eth0" plugin_id="4003" plugin_sid="7"

src_ip="192.168.178.20" src_port="4445" dst_ip="192.168.178.200" dst_port="22"

username="root" log="May 30 13:15:52 dmz01 sshd[12980]: Accepted password for

root from 192.168.178.20 port 4445 ssh2" fdate="2010-05-30 13:15:52" tzone="0"



1.2 OSSIM Server Role

1.2.1 Event Enrichment

The OSSIM server enriches the received normalized event with the metadata stored in the OSSIM

Database.

o Enriched Event

The OSSIM Server enriches the event with the Priority and Reliability values, which are

specific to the event type (plugin_id) and subtype (plugin_sid), as well as with the Asset

Value which is specific to the Source (asset_src) and the Destination (asset_dst) hosts.

Example:

ossim:/var/log/ossim/server.log:

2010-05-30 06:48:41 OSSIM-Message: Event received: event id="0" alarm="0"

type="detector" fdate="2010-05-30 13:15:52" date="1275239752" tzone="0"

plugin_id="4003" plugin_sid="7" src_ip="192.168.178.20" src_port="4445"

dst_ip="192.168.178.200" dst_port="22" sensor="192.168.178.201" interface="eth0"

protocol="TCP" asset_src="2" asset_dst="2" log="May 30 13:15:52 dmz01

sshd[12980]: Accepted password for root from 192.168.178.20 port 4445 ssh2"

username="root"

o Priority

The priority is related to threats and it reflects the importance of a specific attack, having

nothing to do with a specific host or environment. It only measures the relative importance

of the attack itself.

Range: 0 - 5

Default value: 1

Example:

A Unix server running Samba gets attacked by the Sasser worm .

Apart from the fact that the attack won’t have an impact on the given environment, it

has the potential to exploit a big security hole and for that reason the priority is

considered as being high.

o Reliability

Classical risk-assessment would refer it as "probability ". Since it's quite difficult to

determine how probable it is for a network to be exposed to certain vulnerabilities, the IDS

related “reliability” approach was considered more appropriate.

Range: 0 - 10

Default value : 1.

Example:

If a host connects to 5 different hosts in the same subnet using port 445, could be a

normal behavior, unreliable for IDS purposes. If connecting to 15 hosts would be



suspicious, with 500 connections to different hosts in less than an hour the attack

would get more and more reliable.

o Asset Value

It is assigned to both the Source and the Destination Hosts and represents the importance

the host has to the enterprise.

Range: 0 - 5

Default value: 1 (also used for hosts not being defined in the asset database)

Example:

A database server can have an asset value of 5, a development test server an asset

value of 2 and an unknown host in the Internet causing a portscan event would just

have an asset value of 1.

o Alarm

Based on the Event Priority (0-5), Event Reliability (0-10) and the Asset Value (0-5), a Risk

Value (0-10) is calculated and for values equal or greater than 1 Alerts are generated.

The Risk is calculated based on the following formula:

Risk = (Priority * Reliability * Asset) / 25

1.2.2 Policies and Actions

Policies are defined in order to define what has to be done with the events as they reach the OSSIM

Server:

Correlation (i.e. checked against the correlation directives)

Forwarding (i.e. one copy is sent to the forensic storage)

Actions (i.e. send an email)

Discard - the last filter possibility before saving the event in the database, although it is

recommended to filter the events as close to the source as possible.

Policies can make decisions on which events are going to be filtered based on:

Source and Destination Assets (Hosts, Networks, ANY...)

Ports

Plugin Group

Time Range



1.3 The Configuration Workflow

o Collect a Log Sample

First thing to start with is checking which log messages the application generates and eventually

identify sets of logs having a similar structure. Those logs having a similar structure will be where

possible covered by a single collector rule.

o Create a Plugin File

Best is to copy one existing file and modify its content to match the new application. Should a plugin

exist for a similar application, it is recommended to copy such a file, as there is a good chance that

rules have a similar content and are grouped in a similar way - a generic HTTP-Proxy log will always

contain a URL, a generic Firewall log will contain a Source IP Address and Source Port as well as a

Destination IP Address and Destination Port. Some user defined fields might be defined for a specific

application and the correlation at the server level can be simplified if similar applications use the

same user defined fields.

o Define a Generic Rule

This is the last Rule to evaluate, which catches all the events that cannot be grouped under specific

rules.

o Define Specific Rules

The Specific rules are defined for specific error conditions or categories of events. There might also

be that one single rule is used to generate different types or subtypes of events.

o Discard Noise

Events that are considered noise can be discarded by OSSIM by excluding certain event subtypes

(Plugin_SIDs) in the plugin file, by the way the regular expressions are defined or by using policies.

However, the best way to discard events is by filtering them on the monitored device or at syslog

level on the host running the OSSIM Agent.

o Review the Evaluation Order

The rules are evaluated alphabetically, which means that all it counts is the name of a rule and not

the position in the plug-in file. The Generic Rule might even be on the first position if the name is

properly chosen. Having rules alphabetically placed after the Generic Rule will have as effect that

the corresponding logs will be evaluated as generic events instead of having the proper event type

and subtype assigned.

o Register the Plugin with the OSSIM Agent

In order to have a Plugin activated and sending events to the OSSIM server, the path to the plugin file

has to be specified in the Agent configuration file.



o Register the Plugin with the OSSIM Server

This is required in order to let the server know which events should be expected and which priority

and reliability values the events should get assigned.

o Activate the Plugin on the Server Side

Restart the OSSIM Server process.

o Activate the Plugin on the Agent Side

Restart the OSSIM Agent process.

o Testing

Using the logger command sample logs can be replayed in order to test the operation of the OSSIM

Agent or Server.



2 Configuring Detector Plugins

2.1 Rsyslog

Rsyslog is the Syslog implementation shipped with OSSIM and allows configuring filtering and

forwarding in a really easy way compared to the classical syslog daemon. Syslog is also the common

method to send and receive logs.

Before starting with the plugin configuration it is recommended to check whether the subset of logs

the plugin will normalize are saved in an individual file and whether noise can be filtered before

reaching the plugin rules.

2.1.1 Configuration File

/etc/rsyslog.conf

2.1.2 Listener Configuration

$ModLoad imudp

$UDPServerRun 514

$ModLoad imtcp

$InputTCPServerRun 514

2.1.3 Filters

Forward certain events to a local file

if $msg contains 'error' then /var/log/error

if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME' and ($msg contains 'error1' or

$msg contains 'error0') then /var/log/somelog

Stop processing some events

if $msg contains 'error' then ~

Regex in Rsyslog

http://www.rsyslog.com/user-regex.php



2.2 OSSIM Agent Configuration

2.2.1 Configuration File

/etc/ossim/agent/config.cfg

2.2.2 Parameters

[daemon]

daemon: Daemon mode (True or False)

pid: Path to the PID file (Process identifier)

[event-consolidation]

Enables event consolidation at agent level. It is recommended to use polices instead of this

feature as consolidation at the agent level affects the correlation process.

by_plugin: List of plugins that will be consolidated

enable: Enable or disable (True or False)

time: Wait n seconds to consolidate the events before sending them

Example:

[event-consolidation]

by_plugin=1001-1150,1501-1550,4001-4010

enable=False

time=10

[log]

Configures the verbose level and the path to the different log files

error: File in which the error events will be stored

file: File in which all the agent logs will be stored

stats: File in which the agent stats will be stored (Every 5 minutes)

verbose: Configures the verbose level (Debug, Info, Warning, Error or

Critical)

[output-plain]

Writes in a log file what is being sent to the OSSIM Server (Useful for debugging and

developing purposes)


file: File in which the output-plain will be stored

[output-server]

Configures the server to which events are sent

enable: Enable or disable sending events to the server (True or False)

ip: IP address of the OSSIM Server

port: Listening port of the OSSIM Server



[plugin-defaults]

In this category variables can be defined to be used in the plugins configuration.

Example:

[plugin-defaults]

date_format=%Y-%m-%d %H:%M:%S

interface=eth0

sensor=192.168.178.201

[plugins]

Defines which plugins (detectors and monitors) are enabled

name_of_the_plugin=path_to_the_plugin_config_file

Example:

[plugins]

postfix=/etc/ossim/agent/plugins/postfix.cfg

ssh=/etc/ossim/agent/plugins/ssh.cfg

[watchdog]

Monitor the process associated to each plugin (In case it is running in the same machine)


interval: Wait X seconds between checks

restart_interval: Restart the process every X seconds (This has to be enabled in each

plugin)



2.3 Detector Plugin Configuration

2.3.1 Configuration Files

/etc/ossim/agent/plugins/*.cfg

2.3.2 Common Event Types

Copy and modify the existing plugin files to create plugins of the following types.

a. Log - Reading from files

Plugin statement: source=log

b. Database - Reading from databases

Plugin statement: source=database

− mssql - Microsoft SQL

Plugin statement: source_type=mssql

− mysql - MySQL

Plugin statement: source_type=mysql

c. SDEE - Cisco device logs

Plugin statement: source=sdee

d. SnortLog - Snort logs

Plugin statement: source=snortlog

e. WMI - Windows Management Instrumentation

Plugin statement: source=wmi

2.3.3 Parameters

[DEFAULT]

Any variable defined inside this category will be sent to the OSSIM Server if not modified by

a plugin rule. User reserved range is between 9000 and 10000.

plugin_id: Numerical identifier of the plugin within the OSSIM system

Example:

plugin_id=4003



[config]

type: detector

enable: Enable or Disable the plugin (It must be enabled in config.cfg)

source: Source of the events (log, mssql, mysql, wmi)

location: The file(s) where the logs can be found - can contain multiple

comma-separated files

create_file: Create the log file in case it does not exist

process: Name of the process generating logs (If on the same system)

start: Start the process when the agent starts (yes/no)

stop: Stop the process when the agent stops (yes/no)

startup: Command that starts the process

shutdown: Command that stops the process

exclude_sids=SID List Use this option to exclude SIDs

Example (hp-eva):

process=snmptrapd

start=yes

stop=yes

startup=/etc/init.d/snmpd start

shutdown=/etc/init.d/snmpd stop

exclude_sids=404,200,403

[translation]

string=value Used to map strings to their corresponding values

Example (Postfix):

[translation]

sent=10

bounced=11

[Rule IDs – Specific Rules]

Here are the events collected and normalized.

event_type=event

regexp=Regular Expression

plugin_sid=Plugin SID

Event_Field=Value

Example(ssh):

[01 - Failed password]

event_type=event

regexp="(\SYSLOG_DATE)\s+(?P<sensor>[^\s]*).*?ssh.*?Failed password for inval

user (?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*?port\s+(?P<sport>\PORT)"

plugin_sid=1

date={normalize_date($1)}



src_ip={$src}

dst_ip={resolv($sensor)}

src_port={$sport}

username={$user} [Rule IDs – Specific Rules]

[Rule ID – Generic Rule]

Example (ssh):

[99 - Generic rule]

# Nov 15 11:55:35 11.1.4.9 sshd[1769702]: **********

event_type=event

regexp="(\SYSLOG_DATE)\s+(?P<sensor>[^\s]*).*?ssh.*"

plugin_sid=99


dst_ip={resolv($sensor)}

Note: As rules are ordered alphabetically the Generic Rule has to have the highest Rule ID .

2.3.4 Using Local (Plugin) Variables

The different configuration variables defined in the plugin configuration file can be used with the

following syntax:

%()s

Example:

process=pads

shutdown=killall -9 %(process)s

2.3.5 Using Global (Agent) Variables

\_CFG()

Example:

In the agent configuration file (/etc/ossim/agent/config.cfg):

[watchdog]

restart_interval=3600 ; seconds between plugin process restart

In the plugin configuration file(/etc/ossim/agent/plugins/*.cfg):

restart_interval=\_CFG(watchdog,restart_interval)



2.4 Aliases

2.4.1 Path

/etc/ossim/agent/aliases.cfg

2.4.2 Predefined Regular Expressions

The predefined regular expressions can be used when creating new plugins.

IPV4= \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}

MAC= \w{1,2}:\w{1,2}:\w{1,2}:\w{1,2}:\w{1,2}:\w{1,2}

PORT= \d{1,5}

TIME= \d\d:\d\d:\d\d

SYSLOG_DATE= \w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d

SYSLOG_WY_DATE= \w+\s+\d{1,2}\s\d{4}\s\d\d:\d\d:\d\d

To use an Alias in the regular expression use the \IPV4, \MAC, \SYSLOG_DATE, etc.

2.5 Functions

2.5.1 Path

/usr/share/ossim-agent/ossim_agent/ParserUtil.py

2.5.2 Conversions

resolv(host): translates a host name to an IPv4 address

resolv_ip(addr): translates an IPv4 address to a host name

resolv_port(port): translate a port name into its number

normalize_date(date): convert date strings to isoformat (must tag the regular expressions

with the following: <year>, <month>, <minute>, <hour>, <minute>,

<second> or <timestamp> for timestamps. To define new date

formats add a new regexp to the DATE_REGEXPS array.

normalize_protocol(proto): translates the protocols to the protocol numbers, based on the

PROTO_TABLE

md5sum(datastring): calculates the md5 checksum

upper(string): all upper case

hextoint(string): get the integer value of a hexadecimal number



2.5.3 Application Specific Translations

snort_id(id): adds 1000 to the Snort ID

intrushield_sid(sid,name): all McAfee Intrushield IDs are divisible by 256, and this length

doesn't fit in the OSSIM table ( mcafee_sid =

hextoint(mcafee_sid)/256)

netscreen_idp_sid(msg): translates the Netscreen messages based on the

NETSCREEN_IDP_SID_TRANSLATION_TABLE translation table

(defined in ParserUtil.py)

iss_siteprotector_sid(msg): translates the ISS_SiteProtector messages based on the

ISS_SITEPROTECTOR_SID_TRANSLATION_MAP translation table

(defined in ParserUtil.py)

resolv_iface(iface): normalize interface name to either “ext” or “int”

2.5.4 User Defined Translations

translate(string): translates strings based on the entries defined in the [translation]

section of the plugin.

Example (from the iptables plugin):

# The translation section in the plugin configuration file

[translation]

ACCEPT=1

REJECT=2

DROP=3

DENY=3

Inbound=4

Outbound=5

# Rule ID

[0 - iptables]

# Log sample

# Oct 31 08:59:25 M2600001 kernel: RULE 0 -- ACCEPT IN= OUT=lo SRC=127.0.0.1

DST=127.0.0.1 LEN=60

# TOS=0x00 PREC=0x00 TTL=64 ID=8437 DF PROTO=TCP SPT=57275 DPT=836

SEQ=2806649400

# ACK=0 WINDOW=32767 RES=0x00 SYN URGP=0

# Log Parsing

regexp=(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(\S*) (\S*):.*?(\S+)\s+IN=(\S*) OUT=(\S*) SRC=(\S+)

DST=(\S+) LEN=(\d+) \S+ \S+ TTL=(\d+) .*? PROTO=(\S*) SPT=(\d*) DPT=(\d*)

………

# plugin_sid is set to 1, the translated value for ACCEPT

plugin_sid={translate($4)}



2.6 Event Fields

Mandatory – no default values, have always to be set when creating a new plugin

plugin_id Event Type

plugin_sid Event Subtype

Mandatory – default values are assigned by the OSSIM Agent

date The time the event has been collected from the device

sensor The IP Address of the sensor collecting the event

interface The interface where the event has been collected

protocol IP Protocol (see /etc/protocols)

src_ip The Source IP Address

src_port The Source Port

dst_ip The Destination IP Address

dst_port The Destination Port

Optional

username The User referred in the event

password The Password referred in the event

filename The Filename referred in the event

userdata1 – userdata9 User defined fields that could be used in custom reports,

correlation directives, etc.

Special types of events and the list of fields that can be used in each event type:

Host-os-event Host-mac-event Host-service-event

host host host

os mac sensor

sensor vendor interface

interface sensor port

date interface protocol

date service

application

date



2.7 Rules

The Rules define the format of each event and how they are normalized. It is composed by a regular

expression and the list of fields that the event will include once it is sent to the OSSIM Server.

In some cases only one regular expression will collect every event coming from one application, in

some other cases more than one rule will be required.

2.7.1 Evaluation Order

Rules are loading in alphabetical order based on the name given to each rule (Rule ID).

Once the log matches the regex of one rule the ossim agent stops processing the event, therefore

generic rules must be the last to be evaluated.

2.7.2 Structure

o Name / Rule ID

The name of the rule is mandatory

o Regular Expression

The regexp field contains the regular expression that defines the format of the events, and

extracts the information to normalize the event.

The regular expression has to be written following Python regular expression syntax:

http://docs.python.org/library/re.html

The information extracted by the regular expression from the log can be accessed by:

Position: (\d\d):(\d\d):(\d\d)

hour={$1}

minutes ={$2}

seconds={$3}

Tags: (?P<hour>\d\d):(?P<minutes>\d\d)(?P<seconds>\d\d)

hour={$hour}

minutes ={$minutes}

seconds={$seconds}

o Normalized Fields

As the server must receive normalized events, where IP addresses for instance are using the

IPV4 format and the date uses the format YYYY-MM-DD HH:MM:SS (2010-12-31 22:57:00)

To simplify the process of normalizing events functions are defined (more details on

functions can be found in the “Functions” section of this document):

resolv()

Translates hostnames into IPV4 addresses (DNS queries)



normalize_date()

The normalize_date function translates many date formats into the format

accepted by the OSSIM Server.

o Translations

Used for instance when the Event ID is not numeric, but plugin_sid has to be numeric.

Translations have to be defined inside the [translation] section. The actual translation is

triggered by using the translate() function.

o Exclusions

Some events can be filtered during the collection process editing the configuration file for

each plugin:

- Using the option exclude_sids

- Modifying the regular expressions to avoid matching certain events



2.8 Loading Plugins

2.8.1 Priority and Reliability values

For each Plugin_ID/Plugin_SID pair the Priority and Reliability values will have to be defined while

registering the plugin with the OSSIM Server.

2.8.2 SQL Statement

Similar to copying an existing plugin file and customize it in order to create a new plugin file, an SQL

script can be copied and customized in order to insert the new Plugin information in the database.

The sample SQL script can be found under:

/usr/share/doc/ossim-mysql/contrib/plugins/*.sql

Other than with the Plugin configuration file, the SQL script should be created and executed on the

OSSIM Server and not where the OSSIM Agent runs.

The following is performed by the SQL script:

- Remove the Plugin ID from the ”plugin” table, should such an entry already exist

- Remove the Plugin SIDs from the ”plugin_sid” table, should already exist

- Insert the new Plugin ID information into the “plugin” table

- Insert the new Plugin SIDs into the “plugin_sid” table

To run the script use the following command (please double-check the content of the SQL scripts

and the command line syntax before applying the changes to the database):

ossim-server:/usr/share/doc/ossim-mysql/contrib/plugins# ossim-db < ssh.sql

Example (/usr/share/doc/ossim-mysql/contrib/plugins/ssh.sql):

-- SSHd

-- plugin_id: 4003

DELETE FROM plugin WHERE id = "4003";

DELETE FROM plugin_sid where plugin_id = "4003";

INSERT INTO plugin (id, type, name, description) VALUES (4003, 1, 'sshd', 'SSHd: Secure Shell

daemon');

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES

(4003, 1, NULL, NULL, 'SSHd: Failed password', 3, 2);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES

(4003, 2, NULL, NULL, 'SSHd: Failed publickey', 2, 2);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority,reliability) VALUES

(4003, 99, NULL, NULL, 'SSHd: Generic SSH Event', 1, 1);



2.9 Plugin Activation

2.9.1 Activate the Plugin on the Server Side

Restart the OSSIM Server process:

ossim-server:~#/etc/init.d/ossim-server restart

2.9.2 Activate the Plugin on the Agent Side

Restart the OSSIM Agent process:

ossim-sensor:~#/etc/init.d/ossim-agent restart

3 Log files Generic Syslog

/var/log/syslog (Unix)

/var/adm/messages (Solaris)

To identify where the logs for specific applications or certain logging levels are saved, check the

/etc/syslog.conf or /etc/rsyslog.conf files.

OSSIM Agent

/var/log/ossim/agent.log

OSSIM Server

/var/log/ossim/server.log

4 Debugging Note: Do never leave an application running in Debug mode in a production environment

OSSIM Agent

ossim-agent –vv

OSSIM Server

ossim-server –D6



5 Appendix

5.1 Regular Expressions

Operator Meaning

c A non-special character matches with itself

\c Removes the special meaning of the character c; The RE \$ matches with $

^ Indicates the beginning of the line

$ Indicates the end of the line

. Any individual character

[…] One or any of the characters …; accepts intervals of the type a-z, 0-9, A-Z

[^…] A char different from … ; Accepts intervals of the type a-z, 0-9, A-Z

Regular Expression Matches with

a.b axb aab abb aSb a#b ...

a..b axxb aaab abbb a4$b ...

[abc] a b c (one character srtings)

[aA] a A (one character srtings)

[aA][bB] ab Ab aB AB (two character srtings)

[0123456789] 0 1 2 3 4 5 6 7 8 9

[0-9] 0 1 2 3 4 5 6 7 8 9

[A-Za-z] A B C ... Z a b c ... Z

[0-9][0-9][0-9] 000 001 .. 009 010 .. 019 100 .. 999

[0-9]* empty_chain 0 1 9 00 99 123 456 999 9999 ...

[0-9][0-9]* 0 1 9 00 99 123 456 999 9999 99999 99999999 ...

^.*$ A full line



Operator Meaning

r* 0 or more occurrences of the RE r

r+ 1 or more occurrences of the RE r

r? 0 or an occurrence of the RE r, and no more

r{n} n occurrences of the RE r

r{,m} 0 or at most m occurrences of the RE r

r{n,m} N or more occurrences of the RE r, but at most m

r1|r2 The RE r1 or the RE r2

Regular expression Matches with

[0-9]+ 0 1 9 00 99 123 456 999 9999 99999 99999999 ..

[0-9]? empty_string 0 1 2 .. 9

(ab)* empty_string ab ababab abababababab

([0-9]+ab)* empty_string 1234ab 9ab9ab9ab 9876543210ab 99ab99ab ...

Regular expression Matches with Equals

\d Any decimal character [0-9]

\D Any non decimal character [^0-9]

\s Any space character [ \t\n\r\f\v]

\S Any non space character [^ \t\n\r\f\v]

\w Any alphanumeric character and “_”

[a-zA-Z0-9_]

\W Any non alphanumeric character [^a-zA-Z0-9_]

\Z End of line



5.2 Configuration Example

5.2.1 Scenario

In order to detect user logons on a Unix system, the “last” command output will be used. The “last”

command displays the content of the /var/log/wtmp file, where closed and opened terminal sessions

as well as system restarts are logged.

To just create events on status updates, the last output will be collected periodically and compared

to the similar information saved with the previous loop.

The status updates will be sent by syslog, with the help of the “logger” command.

5.2.2 Write a script to monitor the “last” status

#!/bin/sh

# create the file if does not exist

touch /var/log/last.prev

while true

do

# get last entries

last > /var/log/last.new

# send new entries to syslog

diff /var/log/last.prev /var/log/last.new | grep '^>' | logger -t LOGON_EXAMPLE -p local2.info

# move .new to .prev

mv /var/log/last.new /var/log/last.prev

sleep 5

done

5.2.3 Log sample

dmz01:~# tail -f /var/log/messages

Jul 14 19:21:32 dmz01 LOGON_EXAMPLE: > root pts/3 localhost Wed Jul 14 18:49 - 19:21 (00:31)

Jul 14 19:23:28 dmz01 LOGON_EXAMPLE: > dbadmin pts/3 localhost Wed Jul 14 19:23 still logged in

Jul 14 19:23:59 dmz01 LOGON_EXAMPLE: > root pts/4 localhost Wed Jul 14 19:23 still logged in


Jul 14 19:24:09 dmz01 LOGON_EXAMPLE: > dbadmin pts/3 localhost Wed Jul 14 19:23 - 19:24 (00:00)

Jul 14 19:24:09 dmz01 LOGON_EXAMPLE: > root pts/2 172.22.22.10 Wed Jul 14 18:38 - 19:24 (00:45)

Jul 14 19:24:54 dmz01 LOGON_EXAMPLE: > root pts/2 172.22.22.10 Wed Jul 14 19:24 still logged in

Jul 14 19:26:15 dmz01 LOGON_EXAMPLE: > root pts/2 172.22.22.10 Wed Jul 14 19:24 - 19:26 (00:01)

Jul 14 19:26:20 dmz01 LOGON_EXAMPLE: > ossim pts/2 172.22.22.10 Wed Jul 14 19:26 still logged in

Jul 14 19:26:25 dmz01 LOGON_EXAMPLE: > ossim pts/2 172.22.22.10 Wed Jul 14 19:26 - 19:26 (00:00)

5.2.4 Collect the logs in a new log file

Add the following to the rsyslog.conf on the system running the OSSIM Agent:

#

# LOGON_EXAMPLE

#

local2.info /var/log/last_logon.log



5.2.5 Restart “rsyslog”

opensourcesim:~# /etc/init.d/rsyslogd restart

5.2.6 Check whether the new entries are written in the new log file

opensourcesim:/etc/ossim/agent/plugins# tail -f /var/log/last_logon.log

Jul 14 19:38:49 dmz01 LOGON_EXAMPLE: > root pts/2 localhost Wed Jul 14 19:38 still logged in


Jul 14 19:38:59 dmz01 LOGON_EXAMPLE: > ossim pts/2 localhost Wed Jul 14 19:38 still logged in

Jul 14 19:40:51 dmz01 LOGON_EXAMPLE: > ossim pts/2 localhost Wed Jul 14 19:38 - 19:40 (00:01)

Jul 14 20:15:09 dmz01 LOGON_EXAMPLE: > reboot system boot 2.6.31.6 Wed Jul 14 17:39 - 20:15 (02:35)

5.2.7 Create a plugin file

Copy an existing plugin to build the new one on the existing structure

opensourcesim:/etc/ossim/agent/plugins# cp syslog.cfg example.cfg

Set the new plugin specific parameters

;; Building Plugins Example

;; plugin_id: 9001

;; type: detector

[DEFAULT]

plugin_id=9001

[config]

type=detector

enable=yes

source=log

# Enable syslog to log everything to one file. Add it to log rotation also.

# echo "*.* /var/log/all.log" >> /etc/syslog.conf; killall -HUP syslogd

#location=/var/log/all.log

location=/var/log/last_logon.log

# create log file if it does not exists,

# otherwise stop processing this plugin

create_file=true

process=

start=no

stop=no

startup=

shutdown=

## rules

[Rule 01 - Console Session Open]

# Jul 14 20:36:47 dmz01 LOGON_EXAMPLE: > root tty1 Wed Jul 14 20:36 still logged in

event_type=event

regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE:

>\s+(?P<username>[^\s]+)\s+(?P<tty>tty\d+)\s+(?P<logged_event>.*still logged in.*))$"

sensor=\_CFG(plugin-defaults,sensor)




plugin_sid=1

username={$username}

dst_ip={resolv($host)}

userdata1={$tty}

userdata2={md5sum($logline)}

userdata3={$logline}

userdata4={$logged_event}

[Rule 02 - Console Session Closed]

# Jul 14 20:35:46 dmz01 LOGON_EXAMPLE: > root tty1 Wed Jul 14 20:18 - 20:35 (00:17)

event_type=event


>\s+(?P<username>[^\s]+)\s+(?P<tty>tty\d+)\s+(?P<logged_event>.*))$"



plugin_sid=2



userdata1={$tty}




[Rule 03 - New User Session - IP]

# Jul 14 20:21:49 dmz01 LOGON_EXAMPLE: > root pts/1 172.22.22.10 Wed Jul 14 20:21 still logged in

event_type=event


>\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>\IPV4)\s+(?P<logged_event>.*still logged in.*))$"



plugin_sid=3


src_ip={$source}


userdata1={$tty}




[Rule 04 - New User Session - hostname]

# Jul 14 19:23:28 dmz01 LOGON_EXAMPLE: > dbadmin pts/3 localhost Wed Jul 14 19:23 still logged in

event_type=event


>\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>localhost)\s+(?P<logged_event>.*still logged in.*))$"



plugin_sid=3


src_ip=127.0.0.1


userdata1={$tty}






[Rule 05 - User Session Closed - IP]

# Jul 14 19:26:25 dmz01 LOGON_EXAMPLE: > ossim pts/2 172.22.22.10 Wed Jul 14 19:26 - 19:26 (00:00)

event_type=event


>\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>\IPV4)\s+(?P<logged_event>.*))$"



plugin_sid=4


src_ip={$source}


userdata1={$tty}




[Rule 06 - User Session Closed - hostname]

# Jul 14 19:33:56 dmz01 LOGON_EXAMPLE: > root pts/2 localhost Wed Jul 14 19:33 - 19:33 (00:00)

event_type=event


>\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>localhost)\s+(?P<logged_event>.*))$"



plugin_sid=4


src_ip=127.0.0.1


userdata1={$tty}




[Rule 07 - Reboot Detected]

# Jul 14 20:15:09 dmz01 LOGON_EXAMPLE: > reboot system boot 2.6.31.6 Mon May 24 13:51 - 20:15 (51+06:23)

event_type=event

regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >reboot.*))$"



plugin_sid=5



userdata3={$generator}


[Rule 99 - Catch all]

# Whatever doesn't match the above rules

event_type=event

regexp="^(?P<logline>(?P<date>\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE:.*))$"


date={normalize_date($date)}

plugin_sid=99






5.2.8 Register the Plugin with the OSSIM Agent

Add the example.cfg plugin to the agent configuration file

opensourcesim:~# more /etc/ossim/agent/config.cfg

[plugins]

syslog=/etc/ossim/agent/plugins/syslog.cfg

example=/etc/ossim/agent/plugins/example.cfg

wmi-system-logger=/etc/ossim/agent/plugins/wmi-system-logger.cfg

Alternatively the plugin can be activated with the ossim-setup utility:

opensourcesim:~# ossim-setup

1) Change Sensor Settings

2) Select Detector Plugins

3) Select the “example” Plugin



4) Save & Exit

5.2.9 Register the Plugin with the OSSIM Server

Copy an existing SQL script to build the new one on the existing structure

opensourcesim:/usr/share/doc/ossim-mysql/contrib/plugins# cp syslog.sql example.sql

Get a list of the rules defined in the Plugin configuration file.

opensourcesim:~# grep '^\[' /etc/ossim/agent/plugins/example.cfg

[Rule 01 - Console Session Open]

[Rule 02 - Console Session Closed]

[Rule 03 - New User Session - IP]

[Rule 04 - New User Session - hostname]

[Rule 05 - User Session Closed - IP]

[Rule 06 - User Session Closed - hostname]

[Rule 07 - Reboot Detected]

[Rule 99 - Catch all]

Rules having the same plugin_sid will only require one SQL statement and plugin_sid defined on the

OSSIM server. Different rules where used just because both IP addresses and hostnames are

returned as sources by the “last” command.

-- plugin_id: 9001

DELETE FROM plugin WHERE id = "9001";

DELETE FROM plugin_sid where plugin_id = "9001";

INSERT INTO plugin (id, type, name, description) VALUES (9001, 1, 'Example', 'User logons based on the last output');

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 1, NULL, NULL, 'Login: System

console' , 5, 5);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 2, NULL, NULL, 'Logout: System

console' , 5, 5);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 3, NULL, NULL, 'Login: Pseudo

terminal' , 3, 5);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 4, NULL, NULL, 'Logout: Pseudo

terminal' , 3, 5);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 5, NULL, NULL, 'System reboot:

Restarted' , 5, 5);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 99, NULL, NULL, 'Last: Generic

messages' , 1, 1);

After changing the script to reflect the Plugin IDs and SIDs, load the changes with the command:

opensourcesim:/usr/share/doc/ossim-mysql/contrib/plugins# cat example.sql | ossim-db



5.2.10 Check whether the plugin was successfully registered

Plugin ID

Plugin SIDs

5.2.11 Restart the OSSIM Server

opensourcesim:~# /etc/init.d/ossim-server restart

5.2.12 Restart the OSSIM Agent

opensourcesim:~# /etc/init.d/ossim-agent restart



5.2.13 Check whether Events and Alarms are received

Events

Alarms

AlienVault Building Collector Plugins

Documents

Transcript of AlienVault Building Collector Plugins