How frequently should an Application be tested?

By Paladion

April 22, 2009

We are often asked how frequently an application should be tested for security. In this post, I'd like to discuss the criteria for determining the frequency of tests.

First, let's review the benefits of doing periodic penetration tests:

  1. New attacks are invented regularly. Jeremiah Grossman compiles a list of new attacks invented each year. He counted 70 new techniques in 2008, 83 in 2007 and 65 in 2006. That's 15-20 new attack ideas each quarter. A periodic test keeps you current on all the latest attacks too.
  2. New features (and bugs) are added regularly If your application adds new features regularly, then any of those new features could also introduce security holes. In our periodic tests, we've noticed that new holes are added almost every time new features are added. Periodic tests are useful to spot them.
  3. There's more focus on the residual holes This not-so-scientific graph shows the pattern of open vulnerabilities after repeated tests. This is what we've observed after our periodic tests, and suggests that developers fix tougher, residual holes after the easier ones are fixed.


Based on these observations, here're the criteria we recommend for you to determine the ideal frequency for your security tests:

  • Sensitivity of the data: If your application handles sensitive data like credit cards, you're a more likely target for new attacks, so test the app more frequently.
  • Criticality of the Application: If your application is business critical, it's better to test it more frequently and reduces your risk.
  • Frequency of changes: If your application adds new features or undergoes changes regularly, test it more frequently.

Most of the sensitive applications under our care are tested quarterly. The less sensitive ones are tested once in six months. The less sensitive ones with no changes are tested only annually.

Tags: Uncategorized