On May 16, 2005 eWeek published a review of Web Application Firewalls . To test the firewalls, they deployed the firewall in front of a typical application, let the firewall "learn" from the traffic and then ran a series of tests--automated scans, forceful browsing and manual buffer overflows attempts.
Why? Testing a firewall by verifying its ability to protect against canned attacks is like test driving a car by just starting the ignition.
Here is how a web application firewall could be put through its paces:
First, perform a penetration test of the application without the firewall in front.
Next, deploy the firewall in its default configuration and verify if the attacks still succeed.
Take the above results and see if the firewall can be configured to block those attacks.
Verify if the attacks still go through after the firewall has been configured to block them.
Why is this better? One, it checks how an application actually benefits by adding a firewall in front. Second, it gives the firewall a chance to be configured to protect against specific attacks on the application. Third, and most importantly, we do not limit ourselves to canned attacks from scanners, but actually see its effectiveness against logical attacks on the application.