Otdfbt

so due some spamming attacks and stuff I decided to enable modsecurity on my webserver.
I installed Fail2Ban and configured it like that:

excerpt from jail.local:

#
# Custom modsecurity
#

[modsec]
enabled = true
filter = modsec
action = iptables-multiport[name=ModSec, port="http,https"]
 sendmail-buffered[name=ModSec, lines=5, dest=user@googlemail.com]
logpath = /var/log/apache2/modsec_audit.log
bantime = 86400
maxretry = 1

And the modsec filter:

# Fail2Ban configuration file
#
# Author: Florian Roth

[Definition]
failregex = [.*?]s[w-]*s<HOST>s
ignoreregex =

Modsecurity itself is configured like this (pretty standard except it should also block access if a rule hits):

#SecRuleEngine DetectionOnly
SecRuleEngine On


Include /usr/share/modsecurity-crs/*.conf
Include /usr/share/modsecurity-crs/base_rules/*.conf
Include /usr/share/modsecurity-crs/optional_rules/*.conf


SecRequestBodyAccess On


SecRule REQUEST_HEADERS:Content-Type "text/xml" 
 "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"

SecRule REQUEST_HEADERS:Content-Type "application/json" 
 "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"


#SecRequestBodyLimit 13107200
#SecRequestBodyNoFilesLimit 131072
#SecRequestBodyInMemoryLimit 131072
SecRequestBodyLimitAction Reject

#
SecRule REQBODY_ERROR "!@eq 0" 
"id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%reqbody_error_msg',severity:2"

#
SecRule MULTIPART_STRICT_ERROR "!@eq 0" 
"id:'200003',phase:2,t:none,log,deny,status:400, 
msg:'Multipart request body failed strict validation: 
PE %REQBODY_PROCESSOR_ERROR, 
BQ %MULTIPART_BOUNDARY_QUOTED, 
BW %MULTIPART_BOUNDARY_WHITESPACE, 
DB %MULTIPART_DATA_BEFORE, 
DA %MULTIPART_DATA_AFTER, 
HF %MULTIPART_HEADER_FOLDING, 
LF %MULTIPART_LF_LINE, 
SM %MULTIPART_MISSING_SEMICOLON, 
IQ %MULTIPART_INVALID_QUOTING, 
IP %MULTIPART_INVALID_PART, 
IH %MULTIPART_INVALID_HEADER_FOLDING, 
FL %MULTIPART_FILE_LIMIT_EXCEEDED'"

SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" 
"id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"

SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000

SecRule TX:/^MSC_/ "!@streq 0" 
 "id:'200005',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %MATCHED_VAR_NAME'"


SecResponseBodyAccess off

SecResponseBodyMimeType text/plain text/html text/xml

SecResponseBodyLimit 524288

SecResponseBodyLimitAction ProcessPartial

SecTmpDir /tmp/

SecDataDir /tmp/

#SecUploadDir /opt/modsecurity/var/upload/

#SecUploadKeepFiles RelevantOnly

#SecUploadFileMode 0600

#SecDebugLog /opt/modsecurity/var/log/debug.log
#SecDebugLogLevel 3

SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"

SecAuditLogParts ABIJDEFHZ

SecAuditLogType Serial
SecAuditLog /var/log/apache2/modsec_audit.log

#SecAuditLogStorageDir /opt/modsecurity/var/audit/

SecArgumentSeparator &

SecCookieFormat 0

SecUnicodeMapFile unicode.mapping 20127

SecStatusEngine On

Now the problem is, that fail2ban never ban an IP which caused a modsecurity violation. Fail2ban log is also not saying anything interesseting, just startup messages and jail creations. If a modsecurity violation happens, nothing appears in the fail2ban.log

Modsecurity hits well,.. any directory traversal or SQL injection stuff gets an 403 error. But no jailing happens (banning the IP).

What did I do wrong?

TL;DR. The log format has changed and you need an additional [client [d.:]+])?:

failregex = ^%(_apache_error_client)s(?: [client [d.:]+])?
ModSecurity:s+(?:[(?:w+ "[^"]*"|[^]]*)]s*)*Access denied with
code [45]dd (?:.*)$

The log format for libapache2-mod-security2 has changed to have the client two times.

[client 198.51.100.10:62741] [client 198.51.100.10] ModSecurity: Access denied...

The %(_apache_error_client)s in filter.d/apache-common.conf gets the IP address from the first one.

[] [(:?error|S+:S+)]( [pid d+(:S+ d+)?])? [client <HOST>(:d1,5)?]

Like Sean Owen notices, the second one needs to be added to the failregex regular expression, but simple a .* might be exploitable as it matches any number of characters. [client [d.:]+])? is more specific.

Detailed solution: Update Fail2Ban apache-security.conf filter for security2 module by Kazimer Corp. This article adds a new custom filter.d/apache-security2.conf to avoid it from getting overwritten by the distribution.

I have been having this problem as well and I found one error in apache-modsecurity.conf that fixed it for me. Maybe this will help you.

The default Fail2Ban apache-modsecurity.conf failregex looks like this:

failregex = ^%(_apache_error_client)s ModSecurity: etc...

My apache error log looks like this:

[...time stamp...] [:error] [pid #####] [client ###.###.###.###:#####] [client ###.###.###.###] ModSecurity: etc...

Notice the 2 client statements in the apache log. I believe Fail2Ban is only looking for 1 client statement. So adding .* in front of ModSecurity allows any number of any character to precede ModSecurity after the first client statement. I'm curious if there's a better solution to the problem but that quick fix solved it for me.

New Fail2Ban regex:

 failregex = ^%(_apache_error_client)s .*ModSecurity: etc...

Fail2Ban now finds the modsecurity log entries in apache's logs and bans them appropriately.

Hope this helps you.