Forwarding Issues
Issue:
- A few events are missing in Splunk Cloud. Most of the data is being ingested as expected. Missing events that should be forwarded by the forwarder, UF or HF. This issue happens intermittently.
Root Cause:
- Splunk forwarder queues are saturated. When it was released Splunk tailreader went to read the file and the file wasn't available anymore (rolled or deleted).
WARN TailReader [3360 tailreader0] - Access error while handling path: failed to open for checksum: '/path/to/the/log/file.text' (No such file or directory)
Solution:
- Increase the number of parallel ingestion pipeline as possible, based on CPU cores in the instance.(parallelIngestionPipelines)
File name: /opt/splunk/etc/system/local/server.conf
[general]
parallelIngestionPipelines = 2