What is a web application firewall?

A web application firewall is a process that monitors all requests sent to the web server, which are then passed on to the web scripts, for violations of certain rules. It is similar to a firewall or an intrusion detection system (IDS), or some may say an intrusion prevention system (IPS), and sits between the web server (like Apache) and web applications (e.g phpBB [HREF13], Horde [HREF14], Mambo [HREF15], etc.).
One such web application firewall is mod_security [HREF16], an open source (GPL2) Apache module written by Ivan Ristic of Thinking Stone Ltd [HREF17] and author of Apache Security [HREF18].

Why use a web application firewall?

Isn't my network firewall/IDS enough?

While having a firewall and/or an IDS is very important for the security of a web server, they cannot provide full protection. Firewall/IDS is blind to many details of the requests sent to web applications, particularly if they are encrypted, for example, with SSL. A web application firewall running as a module of the web server has full access to all requests and can scan all web streams for known attacks and suspicious requests.

Isn't it better to write more secure code for web applications?

Improving the code to make it more secure is always most desirable, but sometimes this is not always practical. Web applications aren't always the easiest to keep up to date as often there is no automatic update method, and newer updates more often than not break backward compatibilities and need more care than conventional software applications. They could also come from third parties that may take a long time to acknowledge and release patches. It is also not uncommon to find old and forgotten web applications installed somewhere in the web tree, that is no longer used but not removed completely, apart from the links from the main website. They are the perfect backdoors for would-be attackers.
A layer of automatic protection, such as a web application firewall, is therefore a very important and handy tool for web administrators.

The basics

Installation

Mod_security is available from many repositories for Unix operating systems. It can be installed via a package manager command such as apt-get, yum or portinstall. Installing from source is also fairly trivial. Mod_security requires the usual development packages (for compilers and development libraries) and in particular, the development package for Apache. Then all that is required to compile and load mod_security as an Apache module is to download [HREF19] and unpack the source tarball, change to the directory that matches your Apache version and run this:

apxs2 -cia mod_security.c

Enabling mod_security

To enable mod_security, the following needs to be set in the server-wide configuration file for Apache:

SecFilterEngine On
SecFilterDefaultAction "deny,log,status:500"

The first directive turns mod_security on by default for all websites hosted by this web server. The second directive specifies the default action for requests that violate mod_security rules, which here is:

1. Deny access
2. Log request
3. Return an error status code, which is 500 Internal Server Error in this case.

The error code can be anything you like. It is fine to specify an unique error code to separate mod_security related errors from others, but it is also a good idea to confuse the attackers by returning an error code that hides the presence of mod_security.

Logging

Mod_security can log relevant requests (that violate its rules) for further analysis. Logging can be done universally or per virtual host. To do so, these directives need to be set in either server-config or virtual-host config:

SecAuditEngine RelevantOnly
SecAuditLog /var/log/apache2/audit.log

Mod_security log is very detailed by default. Here is an example of a mod_security log entry with default status code 403 (Forbidden) and access deny, where some bot is trying to exploit horde, a webmail application:

==837bd26a==============================
Request: www.example.com 123.123.123.123 - - [24/Apr/2006:15:04:16 +1000] "GET /horde/services/help/
?show=about&module=;%22.passthru(%22uname%20-a;id%22); 
HTTP/1.0" 403 1033 "-" "DataCha0s/2.0" - "-"
Handler: type-map
----------------------------------------
GET /horde/services/help/?show=about&module=;%22.passthru(%22uname%20-a;id%22); HTTP/1.0
Connection: close
User-Agent: DataCha0s/2.0
Host: www.example.com
mod_security-message: Access denied with code 403. Pattern match "(chr|fwrite|fopen|system|echr|
passthru|popen|proc_open|shell_exec|exec|proc_nice|proc_terminate|proc_get_status|proc_close|
pfsockopen|leak|apache_child_terminate|posix_kill|posix_mkfifo|posix_setpgid|posix_setsid|
posix_setuid|phpinfo)\\(.*\\)\\;" at REQUEST_URI [id "300008"] [rev "1"] 
[msg "Generic PHP exploit pattern denied"] [severity "2"]
mod_security-action: 403

HTTP/1.0 403 Forbidden
Vary: accept-language,accept-charset
Accept-Ranges: bytes
Connection: close
Content-Type: text/html; charset=iso-8859-1
Content-Language: en
Expires: Mon, 24 Apr 2006 05:04:16 GMT
--837bd26a--

Server identity masking

One technique that is often employed to make life a little harder for attackers is to change the identity of the web server, which by default is sent with every HTTP response in the Server header. Apache is particularly friendly towards attackers by not only honestly telling them everything about itself, including version number, but also allowing child modules to append their identities to that string too.
The usual way to change this behavior is to modify the Apache source code and recompile the server. Mod_security provides a much easier method to achieve the same thing:

SecServerSignature " " # Blank signature
# Or:
SecServerSignature "Microsoft-IIS/5.0"

This also has an advantage over the other method in that it only shows this fake signature to the public, but when writing to internal logs, it uses the real signature.

Internal chrooted Apache

Chroot, an operation which changes the working directory on Unix operating systems, can be used on web servers to create a jail or an artificial environment for Apache. It is a pre-emptive way to contain a security breach of Apache, through web applications, by preventing any damage to the actual system. The process is highly complicated and in practice, it is mostly used on operating systems that come with Apache chrooted by default, such as OpenBSD [HREF20].
Mod_security has a built-in chroot functionality that makes the whole process less complicated via the SecChrootDir directive. A detailed instruction and fact list can be found in the mod_security documentation [HREF21]. There are two main advantages to mod_security approach to chrooting:
Chroot can be enabled or disabled easily without the need for reconfiguration. All that is required is a restart of Apache (and mod_security).
Only data files are required in the chroot jail. Because mod_security makes the chroot call after web server initialisation but before forking, all shared libraries are already loaded, all web server modules are initialised, and log files are opened.
Nevertheless, chroot is an advanced operation that requires profound understanding of Unix operating systems, Apache as well as every web application that runs under it. Therefore, it should always be considered somewhat experimental, particularly for complex web setups, and used with precautions.

Integration with firewall/IDS and anti-virus scanner

As previously mentioned, mod_security complements firewall and intrusion detection/prevention systems by providing an important security layer at web application level for web servers. It also provides integration with them via utility scripts and daemons for more thorough protection.

The guardian daemon

Ivan Ristic, the author of mod_security and apachesecurity.net, also writes a set of tools for Apache [HREF22]. Although they can work independently to mod_security, they have support for it and can act as a link between mod_security and the firewall/IDS/IPS. Among those tools, httpd-guardian is a daemon script written in Perl that can monitor requests that violate mod_security rules. Once a threshold is reached, httpd-guardian can either issue a warning, send notifications to administrators or run a script to block the offending IP address. The script execution feature allows httpd-guardian and mod_security to integrate with firewall, IDS and IPS easily.
The daemon also specifies a default script called blacklist, which is in the same set of tools and maintains a dynamic list of IP addresses not allowed through IP tables for a configurable period of time. This blacklist also allows unblocking for false positives.
For httpd-guardian and its related scripts to be effective, they need to see all offending requests that is logged by mod_security. This is done by specifying a single directive in mod_security main configuration:

SecGuardianLog "|/path/to/httpd-guardian"

Anti-virus scanner

ModSecurity is capable of intercepting files uploaded through POST requests and multipart/form-data encoding or (as of 1.9) through PUT requests. Verification scripts can be run through files before they are delivered to web applications. Files that fail verification can be kept for further analysis. Applying this mechanism is very simple:

SecUploadDir /tmp/webfiles
SecUploadApproveScript /full/path/to/the/script
SecUploadKeepFiles RelevantOnly

Mod_security comes with an utility script named modsec-clamscan.pl that provides integration with ClamAV, the popular Unix antivirus scanner [HREF23]. It can call a running ClamAV daemon to scan the uploaded files through the above SecUploadApproveScript directive. For this to work, the /tmp/webfiles directory in the above example and files created in it (which are owned by the apache group) need to be accessible by the user that ClamAV daemon runs as.

Rules and exclusions

Rule directives

Mod_security's most basic rule directive is of the form:

SecFilter KEYWORD [ACTIONS]
# Example:
# The following provides a rudimentary protection against shell script executions:
SecFilter /bin/sh

KEYWORD is a regular expression of the request. ACTIONS (optional) is a list of actions that can be performed on requests that violate this rule. They could be log/nolog, allow/deny, or to give this rule an unique id, revision, severity rating and a message (which are very useful for monitoring).
The more advanced directive, which is recommended, is of the form:

SecFilterSelective LOCATION KEYWORD [ACTIONS]

The new parameter, LOCATION, is a list of location identifiers separated with a pipe. These identifiers consist of all Server variables, special mod_security only variables and variables that are passed to web applications. The full list can be found in the mod_security documentation [HREF21].

Rule examples

Blocking a known troublemaker:

SecFilterSelective REMOTE_ADDR 123\.123\.123\.123

Blocking web leeches:

SecFilterSelective HTTP_USER_AGENT "Web Downloader"

Blocking XSS in the UserAgent string:

SecFilterSelective HTTP_USER_AGENT "<(.|\s|\n)?(script|about|applet|
  activex|chrome|object)(.|\s|\n)?>.*<(.|\s|\n)?(script|about|applet|activex|
  chrome|object)"

Blocking generic PHP exploit signatures:

SecFilterSelective THE_REQUEST "\;(chr|fwrite|fopen|system|echr|passthru|
   popen|proc_open|shell_exec|exec|proc_nice|proc_terminate|proc_get_status|
   proc_close|pfsockopen|leak|apache_child_terminate|posix_kill|posix_mkfifo|
   posix_setpgid|posix_setsid|posix_setuid|phpinfo)\(.*\)\;"
   "id:300007,rev:1,severity:2,msg:'Generic PHP exploit pattern denied'"

Blocking generic PHP remote file inclusion attack signature, also an example of how rules can be chained:

SecFilterSelective REQUEST_URI "\.php\?" chain
SecFilter "(http|https|ftp)\:/" chain
SecFilter "(cmd|command)=(cd|\;|perl |python |rpm |yum |apt-get |emerge |lynx |
   links |mkdir |elinks |cmd|pwd|wget |id|uname|cvs |svn |(s|r)(cp|sh) |net(stat|cat) |
   rexec |smbclient |t?ftp |ncftp |curl |telnet |gcc |cc |g\+\+ |whoami|\./|killall |rm \-
   [a-z|A-Z])"
SecFilterSelective REQUEST_URI "\.php\?" chain
SecFilter "(http|https|ftp)\:/" chain
SecFilter "(cmd|command)=.*(cd|\;|perl |python |rpm |yum |apt-get |emerge |lynx |
   links |mkdir |elinks |cmd|pwd|wget |id|uname|cvs |svn |(s|r)(cp|sh) |net(stat|cat) |
   rexec |smbclient |t?ftp |ncftp |curl |telnet |gcc |cc |g\+\+ |whoami|\./|killall |rm \-
   [a-z|A-Z])"

Blocking SQL injection in cookies:

SecFilterSelective COOKIE_VALUES "((select|grant|delete|insert|drop|alter|
   replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]+
   [[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]|
   UNION SELECT.*\'.*\'.*,[0-9].*INTO.*FROM)"
   "id:300011,rev:1,severity:2,msg:'Generic SQL injection in cookie'"

Exclusions

Sometimes mod_security rules can cause false positives and block innocent requests. Most common problems are with SQL injection rules, since SQL syntax is very close to the English language. In practice, it is usually certain areas of the website that trigger these false positives. For example, a WYSIWYG text editor in the administration area could be the source for false positives on SQL injection or XSS. Fortunately, mod_security provides ways to exclude certain rules.
If an area of the site is only ever used by trusted users (meaning that if a malicious user has access to it then mod_security has no use anyway) then it is OK to exclude that area from mod_security scans:

<Location "/myadmins">
    SecFilterEngine Off
</Location>

If you know that certain rules will trigger false positives in an area, then it is possible to remove those particular rules from being applied there:

<Location "/foobar">
    SecFilterRemove 1001 1002 3003 # rule IDs
</Location>

Signatures, update services and support

Mod_security is similar to an IDS/IPS or an antivirus service in that it is more effective if its rules get updated regularly. You can write rules yourself if you know your web applications well and there is a limited number of them, but it can be overwhelming if there are too many of them. Fortunately, there are a few services providing mod_security rule updates which can be of great help for managing mod_security.

Converting from snort rules

Snort [HREF24] is a very popular open source IDS which has many web-related rules. Mod_security comes with an utility script named snort2modsec.pl that can convert snort rules to mod_security rules.

Getting updates from GotRoot

GotRoot [HREF25] is a website owned by a security consulting group, Prometheus Group [HREF26], that provides updated modsecurity rules of many categories:

• Web application protection
• Just-in-time patches for vulnerable applications
• Bad UserAgents blocking
• Comment spam blacklist
• Compromised/Hacker boxes blacklist
• Anti-Proxy protection
• Apache2 protection
• Known rootkits and worms
• Rule Exclusions
• "Google Hacks" signatures
• Blacklist of known attackers and spammers

They also provide a utility script to automatically download and update mod_security rules as new ones are released. However, it is recommended that you should not use all of them, for the list of rules is massive and every request is to be matched with every single rule, which can seriously affect performance. The author of mod_security recommends that mod_security should be used with a few hundred rules at most. The blacklists provided by GotRoot.com are also temporary as they wait for mod_security 2.0 which has support for querying DNS servers instead of looking up a flat text file, which will make blacklisting much faster and more feasible in practice.

Update and support services from ThinkingStone Ltd

Mod_security is also offered as a Pro version, which consists of a subscription service that provides updated and well-tested rules as well as support. Details, including costs, can be obtained from the company behind mod_security: ThinkingStone.com. ThinkingStone also offers ModSecurity Console, a network-based console designed to collect logs and alerts from remote ModSecurity sensors in real-time, providing security analysts with the support they need to keep their web systems secure. Mod_security will still be available as an open source and free module for Apache.
Community support is also available on the mod_security mailing list. Details can be found on modsecurity.org contact page [HREF27].