Splunk Crashing Issues
Issue:
Splunk software is crashing too often
Splunk software is crashing too often
Root cause:
The host machine was having a low Ulimit setting (default)
The host machine was having a low Ulimit setting (default)
Note: The ulimits control the resources available to a *nix shell and any processes that the shell starts. A *nix host running Splunk software often needs a higher ulimit setting than the defaults.
Query:
index=_internal source=*splunkd.log ulimit
index=_internal source=*splunkd.log ulimit
03-03-2011 21:50:09.027 ERROR ulimit - Splunk may not work due to low file size limit
03-03-2011 21:50:09.027 INFO ulimit - Limit: open files: 1024
03-03-2011 21:50:09.027 INFO ulimit - Limit: cpu time: unlimited
Solution:
Increase the ulimit values to a higher number
File name: /etc/security/limits.conf
* hard nofile 64000
* hard nproc 16000
* hard fsize -1
(or)
File name: /etc/systemd/system/Splunkd.service
LimitNOFILE=64000
LimitNPROC=16000
LimitDATA=8589934592
LimitFSIZE=infinity
TasksMax=8192