Performance

The collector receives messages, processes them, and writes them to syslog storage. Storage write speed is the usual limit. When messages arrive faster than storage can write them, the memory buffer fills; once it is full, new messages are discarded as Buffer overflow.

Storage Write Speed

Syslog storage must be on a local disk; network folders are not accepted.

  • Prefer an SSD (NVMe or SATA) over a spinning disk.

  • Keep storage on a different volume from the operating system and the page file when the traffic is high.

  • Exclude the storage folder from real-time antivirus scanning if the security policy allows it.

To measure what the disk can do, run the storage speed test. It writes test data the same way the collector does and reports messages per second. Compare that rate with the peak rate at which your originators send messages.

Memory Buffer

The memory buffer absorbs bursts while storage catches up. Watch Buffer Usage in the Syslog Collector section of the Server tab.

  • Short peaks that fall back to a low value are normal.

  • Usage that stays high, or Buffer overflow in Discarded Messages, means storage is too slow for the sustained rate. A larger buffer only helps with bursts, not with a sustained rate that storage cannot write.

Size the buffer for the longest burst it must absorb: at 5,000 messages per second, the initial buffer of 100,000 messages covers about 20 seconds. The buffer holds messages in RAM, so a larger value needs more memory.

Incoming Filter

Discarding unwanted messages before storage is the cheapest way to cut load. If much of the traffic is never used (for example Debug from verbose originators), the Incoming Filter removes it before it enters the memory buffer.

The stored messages by severity on the Server tab help to choose a severity threshold.

Character Encoding

Converting messages from another encoding adds a small cost per message; messages with only ASCII characters are not converted. Keep UTF-8 when originators already send UTF-8, as most do today. See Preprocessor.

Transport

UDP has the lowest cost per message. TCP adds connection handling, and TLS adds encryption on top of that. Use TLS where messages must be encrypted; its CPU cost matters only with very high message rates or many connections.

For bursts on UDP, increase Socket buffer size in Advanced Options only after the storage and buffer checks above, and preferably with technical support.

High-Volume Checklist

  1. Run the storage speed test and move storage to a faster local disk if needed.

  2. Drop unneeded traffic with the Incoming Filter.

  3. Size the memory buffer for bursts.

  4. If one server still cannot keep up, split the load: install Syslog Watcher on a second server and point some of the originators to it.

Related Topics