Catch'em Young - How to discover vulnerabilities early

Paladion
By Paladion

November 15, 2004

Bugs are introduced at every stage in the development lifecycle. Some of them are caught quickly in the same stage itself. However, many are caught only much later. Here’re the systems we find to be most effective to address security bugs

Bugs are introduced at every stage in the development lifecycle. Some of them are caught quickly in the same stage itself. However, many are caught only much later. A 2002 study by NIST found that only 45% of all bugs are discovered in the same stage they are introduced in1. Anecdotal evidence from our application security assessments indicate that 80% of the vulnerabilities have their origins in design and coding stages. These also turn out to be the ones that have the highest impact on the business compared to errors in deployment.

The NIST study further found that it would cost 30 times less to fix a bug than it does today were it discovered in the stage it was introduced in. Different studies from IBM and others have estimated this number at different levels - at times as high as 100. While the exact number will vary for each development team and project, it’s clear that it costs much less to fix a bug that’s discovered
early. What should software executives do to reduce this cost?

Many development teams we work with have been putting systems in place to find and fix security bugs early. Here’re the systems we find to be most effective to address security bugs:

Training: Architects and developers are not always aware of the threats their software is exposed to. Unfamiliarity with the techniques used by adversaries
result in applications that trust the environment and user inputs. Training programs bridge this gap. Trained designers address security issues in the design explicitly and trained developers take extra care when writing code. Both translate directly to software with less flaws.

Peer Reviews: Disciplined peer reviews at the design and coding phases yield positive results. Security architecture
reviews have been especially effective as they are checklist-driven, quick and direct.

Automated Code Scanning: A smaller number of teams have integrated static source code scanning into their regular build processes. These search for potentially insecure function calls and flag the developer
early. Static analysis for security bugs is not yet a mature technology, but it is still a useful technique to catch many simple bugs early at a low cost.

Higher Standards for Unit Testing: Many code level vulnerabilities are difficult to discover at the QA stage. So software organizations are experimenting with higher standards for unit testing. Investing additional time in unit testing -- specifically for security vulnerabilities -- is expected to reduce the total cost of testing and also catch bugs early.

Pre-launch Security Testing: Gray box testing of applications
just before they are launched has been very popular to discover security holes. This consistently reduces the number of holes discovered after deployment.
If your software security program is taking off now, you could consider the above notes. And as always, we would like to hear from you about your experience on what works and what doesn’t.

1NIST REPORT, "The Economic Impacts of Inadequate Infrastructure For Software Testing", 2002.


Tags: Best Practices

About

Paladion