Message Processing

The Message Processing tab of Syslog Collector Configuration controls how each incoming syslog message is normalized, filtered, and associated with its originator.

Syslog Collector Configuration — Message Processing tab

The tab has three sections:

  1. Preprocessor — normalizes severity and encoding

  2. Filter — discards unwanted messages early (off by default). See Incoming Filter.

  3. Originator IDs — identifies the device or system that sent each message

Preprocessor

The preprocessor is the first stage applied to each message after it is received from the network. It sets a consistent severity and UTF-8 encoding for later stages.

If message has no severity, assign

Every syslog message stored by Syslog Watcher must have a severity level. Some originators send messages without one, which is non-compliant with the syslog standards.

When the preprocessor receives such a message, it assigns the severity selected here. The default is (7) Debug.

Leave it at Debug when no other value is obviously better. Those messages can be filtered out later, or distinguished in views by severity.

Message body encoding

Syslog Watcher stores all messages internally in UTF-8 format. Most modern syslog originators already send messages in UTF-8, so the default UTF-8 setting requires no conversion.

If originators send messages in a different encoding (for example, Windows-1252 or ISO-8859-1), select that encoding here. The preprocessor converts each incoming message from the specified encoding to UTF-8 before further processing.

Encoding conversion applies to all originators. For mixed encodings, keep similar devices on one collector, or convert on the sender.

Originator IDs

An originator is the entity that initially generates a syslog message — typically a device, server, or workstation. Syslog Watcher uses an originator ID string as the key to organize, filter, and report on syslog data per device.

The Originator IDs section defines how the collector assigns an ID to each incoming message.

Default originator ID rule

Selects the strategy used when no specific rule matches a message. The default is ID = IP address of the message sender, which uses the source IP address of the syslog packet as the originator ID.

Other strategies extract the ID from message fields such as the RFC 5424 HOSTNAME field or the RFC 3164 hostname token.

Resolve name (reverse DNS lookup)

When Yes (the default), the collector performs a reverse DNS lookup to fill the originator name from an ID that came from the sender IP address. The originator ID stays the IP address (or the value from a matching rule). When No, the name is the same string as the ID.

Reverse DNS lookups add a small delay per new originator and can slow startup if the DNS server is slow or unreachable. Disable this option in environments without working reverse DNS.

If originator ID is unknown

Controls what happens when the collector encounters an originator ID it has not seen before:

  • Add new originator (default) — insert a storage record and keep the message

  • Discard messages — drop the message without adding an originator

Add new originator is the usual choice for small or changing networks. Discard messages when only a known set of devices should be accepted.

Originator ID Rules

Custom rules override the default rule for specific IP ranges. Click Add Rule to create one.

IP Range Rule

Match messages by source IP address range and apply specific extraction logic:

Field Description

IP range start

Lower bound of the matched IP address range (inclusive). Leave empty to match any address.

IP range end

Upper bound of the matched IP address range (inclusive). Leave empty to match any address.

Originator ID

Strategy for extracting the originator ID from messages within this range — for example, Extract as RFC5424:HOSTNAME uses the structured-syslog hostname field.

If unable to extract ID

Fallback when the chosen field is empty or missing. Initial: Apply the default rule.

If originator ID is unknown

Same options as the global setting but applied only within this range.

Use IP range rules when different parts of the network identify themselves differently — for example, network appliances that send their hostname in RFC 5424 vs legacy devices that need their source IP used directly.

Memory Buffer

The memory buffer sits between the incoming message stream and the storage writer. It absorbs short bursts of high-volume traffic so that messages are not lost when the storage write speed cannot keep up.

It is configured on the Syslog Storage tab of Syslog Collector Configuration.

Field Initial Description

Buffer size (messages)

100000

Maximum number of messages held in memory at once. Larger buffers handle longer traffic bursts at the cost of more RAM.

Warn if buffer is % full

75

High-water-mark threshold. When the buffer occupancy exceeds this percentage, a warning is logged to Diagnostic Records. Use this as an early signal that the buffer is undersized or that storage throughput is dropping.

A buffer of 100,000 messages typically consumes a few hundred MB of RAM, depending on average message size. If the warning appears often, or the Dropped counter rises in Monitoring, increase the buffer size or fix the storage bottleneck.

For high-volume tuning guidance, see Performance.

Related Topics