Interview: What works in Training Security Testers

Paladion
By Paladion

June 15, 2005

As software organizations figure out how to integrate security testing into the QA process, Palisade talked to Firosh Ummer to learn how he set up the internal training program for security testers at Paladion. With participants from the training program going on to test over 300 applications in the last 3 years, Firosh has been continuously refining the program to make it more effective.

As software organizations figure out how to integrate security testing into the QA process, Palisade talked to Firosh Ummer to learn how he set up the internal training program for security testers at Paladion. With participants from the training program going on to test over 300 applications in the last 3 years, Firosh has been continuously refining the program to make it more effective.

Palisade: Tell us about the training program you have internally for security testers.

Firosh: We have a 5-day training program that introduces the participants to security testing. It includes 3 days of theory and 2 days of hands-on testing. The participants learn the common vulnerabilities in applications, how they are exploited, and how to discover them. They study our internal security testing checklist, and learn how to develop security test cases using threat models.

Remember our participants already have some exposure to security issues--not much, but they have already gone through the basics during their first few months at Paladion. At the end of this program, they will be working alongside seniors for two months. That's when they gain the confidence to develop an entire test plan on their own. So, the focus of the program is to lay a foundation for security testing.

Palisade: Let's go a bit deeper… how exactly do you lay the foundation?

Firosh Ummer

Firosh: Application security testing is really about systematically developing test cases to discover potential vulnerabilities. When the tester gets an application, he will have to think through security risks and develop test cases to verify if the application is secure. Firstly, that requires an understanding of different vulnerabilities and how to exploit them. Secondly, it requires a framework for conceptualizing threats--that's where threat modeling and checklists come in. Once a tester has thought through potential lines of attack, then developing the test cases becomes easier.

Palisade: How similar is this to functional testing? Is it easy for a functional tester to do security testing?

Firosh: Both security and functional testing require discipline in defining test cases. Both try to detect failure conditions for the software. The difference is in the mindset: a functional tester checks compliance against a specification, a security tester has to envision lines of attack to gain an advantage. Is it easy? (laughs) Given adequate training and hands-on experience, this is a skill that can be learnt like any other.

Palisade: What role do tools play in security testing? Is there a lot of automation that can be achieved?

Firosh: That's an interesting question. Many of these tests require the help of tools--to inject data into traffic, to search memory, track API calls, etc. A good tester needs a thorough understanding of the tools to be able to utilize them fully. However, automation is a different ball game. Most test cases are designed for the specific application's variables, and they are refined as the testing progresses; so, it's inefficient to try and automate the testing because a lot of human intervention, especially thinking, is involved during the testing.

Palisade: How do you assure quality in the testing itself? Wouldn't quality differ based on the expertise of the tester?

Firosh: Yes, that's a key challenge. Experience and expertise do play a role in the quality of testing. We use checklists to aid the tester define test cases. That ensures a minimum standard. Less experienced testers are paired with seniors, and all test plans are reviewed a second time before the testing is closed. These are some of the systems we put in place to ensure standards.

Palisade: Extending what you said about expertise, isn't this a fast-changing area where a lot of techniques are getting invented even as we speak?

Firosh: Absolutely! So our testers go "back-to-school" every 6 months for a week to learn the advances in the field and "sharpen the saw". And every fortnight, they share their experiences from the field through informal sessions - these help them to refine their thinking and design new test cases. Like I said, expertise makes a difference, and it's essential that they are thorough with new techniques attackers use to exploit systems.

Palisade: Thank you, Firosh. Could you recommend some good resources for our readers on application security testing?

Firosh: Sure, here're a few sites and books that security testers will find of value.

OWASP Testing Checklist is a useful starting point if you are setting up a security QA program.

Exploiting Software : How to Break Code, by Greg Hoglund and Gary McGraw discusses several techniques to discover vulnerabilities.

Threat Modeling by Frank Swiderski is a good place to learn a structured approach to envisioning threats.


Tags: Features

About

Paladion