Selecting Application Security Vendors – Part II

balaji
By balaji

February 13, 2009

In March 2005, Jose Varghese outlined the best practices for selecting application security vendors in Palisade. That article gave pointers to mid size and large enterprises who are leveraging external application expertise or intending to leverage external resources. Fours year later, we review the themes in that article. Have those criteria changed over these years when application security has moved from back-stage to center-stage? As we reviewed the criteria, we observed that the core principles Jose laid out in 2005 still hold true.

evaluating-appsec-vendors.jpg

In March 2005, Jose Varghese outlined the best practices for selecting application security vendors in Palisade. That article gave pointers to mid size and large enterprises who are leveraging external application expertise or intending to leverage external resources.

Fours year later, we review the themes in that article. Have those criteria changed over these years when application security has moved from back-stage to center-stage? As we reviewed the criteria, we observed that the core principles Jose laid out in 2005 still hold true.

The two primary areas to evaluate continue to be Technical Skills and Delivery Methodology. The improvements the industry has achieved in the last 4 years raise the bar for evaluating Delivery Methodology. This article suggests how you can measure the improvements in Delivery Methodology. Further, globalization has enabled greater reduction in cost and risk - this should now be a factor in your evaluation of vendors.

March 2005

Feb 2009

Evaluate Technical Skills

  • Security Domain Knowledge
  • Software Development Experience
  • Business Domain Expertise

Evaluate Delivery Methodology

  • Testing Methodology
  • Reporting Capabilities
  • Solution Support

Evaluate Technical Skills

  • Security Domain Knowledge
  • Software Development Experience
  • Business Domain Expertise

Evaluate Delivery Methodology

  • Testing Methodology ++
  • Reporting Capabilities ++
  • Solution Support ++

Evaluate Cost and Risk Reduction

  • Global delivery centers
  • On-demand offerings
  • Data security practices

Over the last 4 years, the application security industry has been influenced by several trends that has led to improvements in Delivery Methodologies:

  • Compliance has become a major driver for application security. PCI DSS and SOX are good examples.
  • High profile breaches have led to demand for highly secure solutions from highly secure providers
  • Application security is getting integrated into overall vulnerability & risk management frameworks.
  • Application security vulnerability scanners have improved significantly
  • There is greater choice from globalization
  • On-Demand technologies are maturing, SaaS solutions are available

Keeping these trends in mind, here's the revised criteria we'd like to recommend:

Evaluating Technical Skills

  • Security Domain Knowledge
  • Software Development Experience
  • Business Domain Expertise

Evaluating Delivery Methodology

  • Testing Methodology
    • Additionally, check for the integration of best in class software tools that have matured in recent years
  • Reporting Capabilities
    • Additionally, look at integration capabilities to collaborative solutions, GRC solutions, dashboard solutions, QA solutions & ticketing systems.
    • Additionally, expect strong trending and aggregated vulnerability data that is benchmarked to industry. Have the ability to know exactly where you stand and how effective your programs are.
  • Solution Support
    • Expect far greater clarity and specificity in recommended solutions
    • Are the solutions specific to your platform?
    • Can developers apply the solutions immediately without further research?
  • Compliance
    • Additionally expect compliance specific solutions. (eg. PCI DSS 1.2 compliance)

Evaluate both Cost & Risk reduction

Beyond the regular criteria for vendor risk management, specifically look at the following:

  • Evaluate on demand technologies and global service providers that can dramatically reduce cost
  • Evaluate security practices, standards and technologies and find vendors who have best in class internal security systems. At a minimum, review the following:
    • Data security practices
    • Security certifications for the organization
    • Background checks of engineers
    • Controls to prevent co-mingling of vulnerability data with other clients

Tags: Best Practices

About

balaji