Rspamd symbols

SYMBOLS AND RULESRSPAMD

WHAT ARE SYMBOLS AND RULES

DEFINITIONS

RULE SYMBOL

SCOREGROUP

DESCRIPTION

WEIGHT*

true/false

OPTIONSDynamic part

Static part

∑ Results

WHY DO WE NEED SYMBOLS

SYMBOL_ALLOW

SYMBOL_DENY

SYMBOL_UNKNOWN

Either of symbols

Multiple symbols

RULE1 SYMBOL1

Dependency

▸ Rules define what is executed:

▸ Regexps expression

▸ Lua code

▸ Plugin logic

▸ Each rule can be associated with one or many symbols

▸ Rule can depend on other rules identified by associated symbols

▸ Each rule can define the current dynamic weight (usually from 0 to 1)

SYMBOLS

▸ Symbols define meta-information of a rule:

▸ Name

▸ Static score

▸ Other data (description, group, flags, etc)

▸ Symbols can be:

▸ Normal: associated with exactly one rule

▸ Virtual: are not associated with rules but grouped with normal symbol)

▸ Callback: do not have name or score, just define common rule

▸ Special: have special purpose (e.g. composite symbols)

SYMBOLS

SYMBOLS GROUPS

SYMBOL1

SYMBOL2

SYMBOL3

SYMBOL4

Group1

SYMBOL3

SYMBOL4

SYMBOL5

SYMBOL6

Group2GROUP LIMIT

GROUP DESCRIPTION

SYMBOLS

SYMBOLS GROUPS

▸ Groups join common symbols logically

▸ Groups can set joint limit for symbols scores enclosed

▸ Groups can be used in composite rules:

▸ SYMBOL5 && G:GROUP1

▸ SYMBOL5 && (G:GROUP1 || !G:GROUP2)

EXPRESSIONS IN RULES

▸ Expressions are used in:

▸ Regexp rules

▸ Composite symbols

▸ Expressions have common syntax:

▸ Logic operations: AND (&&), OR (||), NOT (!)

▸ Braces

▸ Limit operation: A + B + C > 2

▸ Elements are called atoms

REGEXP EXPRESSIONS

▸ Atoms are regular expressions (/re/flags):

▸ Header: Header=/re/H

▸ Mime (/P): scan text parts

▸ Body (/B): scan full undecoded body

▸ URL (/U): scan URLs found

▸ There is no order of regexps execution within an expression

▸ Same expressions are cached and executed once

COMPOSITE EXPRESSIONS

COMPOSITES STRUCTURE

SYMBOL3

SYMBOL4

SYMBOL5

SYMBOL1

Group2

NOT SYMBOL6AND

SYMBOL1 AND GR:GROUP2 AND !SYMBOL6

COMPOSITES OPERATIONS

SYMBOL3

SYMBOL4

SYMBOL5

SYMBOL1

Group2

NOT SYMBOL6AND AND

SYMBOL1

SYMBOL3

Symbols to remove

Stage 1: Check

Stage 2: Remove symbols

COMPOSITES STRUCTURE

▸ Composite atoms can include:

▸ Other symbols

▸ Groups (gr:)

▸ Other composites (with recursive references check)

▸ Composite operations can be the following:

▸ Remove symbol and weight (SYMBOL)

▸ Remove weight only (~SYMBOL)

▸ Remove symbol but preserve weight (-SYMBOL)

▸ Always remove symbol and weight (^SYMBOL)

COMPOSITES OPERATION

▸ If any composite proposes that a symbol should NOT be removed, then it is NOT removed:

▸ A & ~B and C & B: B will NOT be removed because of the first rule, but its weight will be removed

▸ A & -B and C & ~B: neither weight, nor symbol B will be removed

▸ Removal could be forced by “^” symbol:

▸ A & ^B and C & -B: weight and symbol B are both removed

PRACTICAL EXAMPLES

A SIMPLE REGEXP EXPRESSION

local reconf = config['regexp'] -- Define alias for regexp module

-- Define a single regexp rulereconf['PRECEDENCE_BULK'] = { -- Header regexp that detects bulk email re = 'Precedence=/bulk/Hi', -- Default score score = 0.1, description = "Message marked as bulk", group = 'upstream_spam_filters'}

rspamd.local.lua:

PRACTICAL EXAMPLES

A MORE COMPLEX EXAMPLErspamd.local.lua:

-- Define encodings types-- /X is undecoded headerlocal subject_encoded_b64 = 'Subject=/=\\?\\S+\\?B\\?/iX' local subject_encoded_qp = 'Subject=/=\\?\\S+\\?Q\\?/iX'-- Define whether subject must be encoded (contains non-7bit characters)local subject_needs_mime = 'Subject=/[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f\\x7f-\\xff]/X'

-- Final rulereconf['SUBJECT_NEEDS_ENCODING'] = { -- Combine regexps re = string.format('!(%s) & !(%s) & (%s)', subject_encoded_b64, subject_encoded_qp, subject_needs_mime), score = 3.5, description = "Subject contains non-ASCII chars but it is not encoded", group = 'headers'}

PRACTICAL EXAMPLES

COMPOSITES EXAMPLElocal.d/composites.conf:

# Ignore forged recipients in case of mailing list composite "FORGED_RECIPIENTS_MAILLIST" { # MALLIST symbol is preserved expression = "FORGED_RECIPIENTS & -MAILLIST"; }

# Ignore forged sender if a message has been forwarded composite "FORGED_SENDER_FORWARDING" { # Symbols from `forwarding` group are removed expression = "FORGED_SENDER & g:forwarding"; }

# Ignore forged sender if a message has been from the mailing list composite "FORGED_SENDER_MAILLIST" { # Symbol 'FORGED_SENDER' is forced to be removed expression = "^FORGED_SENDER & -MAILLIST"; }

PRACTICAL EXAMPLES

Rspamd symbols

Engineering

Transcript of Rspamd symbols