After seeing apache using all it’s threads, and connections not timing out as they should, I looked at the apache error_log and found the following error.
Message: collection_store: Failed to write to DBM file "/var/cache/modsecurity/ip": Invalid argument
I not only saw this on cPanel servers, but on Plesk and plain LAMP (with mod_security and comodo waf installed).
It looks like Comodo somehow released a broken update, that caused the /var/cache/modsecurity/ip.pag to corrupt (that’s my guess).
The fix is rather simple. Update your Comodo WAF rules to version 1.142 or higher, and reset the /var/cache/modsecurity/ip.pag file.
Check the rules.dat file for the version number.
$ echo "" > /var/cache/modsecurity/ip.pag
Then restart apache, and this should fix you issue. You’ll also notice apache threads timing out faster.
Comodo WAF: https://waf.comodo.com/