Thursday, December 17, 2009

Vulnerability Assessment or Penetration Test: How do I decide

Many organizations may be reeling from the decision on whether to perform a vulnerability assessment or a comprehensive penetration test, and rightfully so. These are difficult decisions and often require knowledge of specific industry regulations and mandates. Maybe an organization feels that they have too much at risk and would like to leverage an unbiased outsider's expertise regarding how to provide the necessary level of infosec protection, or may have unfortunately been a victim of compromise. Let us provide some some assistance with this decision.

Requirements gathering for manual/automated penetration should be based on a multitude of items. This is extremely important as many businesses are different and their requirements may not necessitate a complete penetration assessment. Some of these requirements may be:
  • Sensitivity classification associated with the resident system/app data
  • What environment does the system/app currently reside
  • What are the requirements for service availability
  • Security reviews as an integral step of the SDLC process
  • Impact to the business (reputation/revenue) if compromised
These are just a small number of the questions that should be asked when performing the initial determination of the organization's next steps. As a result, answers should lead to questions, which should lead to hypothetical scenarios that may be documented via threat modeling exercises; however, this may be performed as a precursory step to a complete penetration test.

The important item to note is that vulnerability assessments and penetration tests are very different but are still very dependent on each other. For instance, before a penetration test can be executed, some level of information and vulnerability gathering is necessary. This typically, comes in the way of a vulnerability assessment. The goal is to document/map application and system vulnerabilities back to industry security standards that may be used for overall remediation. The vulnerability assessment can stop there and the outcome may be for an organization to remediate the findings ranked by severity. It is worth noting that my personal opinion, and that is all it is, leans towards the fact that any mention of an automated penetration test is simply a vulnerability assessment as much of the human intervention, observation, and extrapolation is removed from the process. Although for quick hit items like vulnerable services at a network level (eg. exposed SNMP, mgmt interfaces, defaults, etc...), this may be an adequate solution. The security issues identified within this process, along with manually derived vulnerabilities, will be investigated further during the penetration testing phase if authorized by the business.

The Penetration test should, if performed correctly, identify items that a vulnerability assessment (aka. 1-click pen test) typically cannot find. This is most prevalent within web application testing, for instance. Items like concurrent session management, advanced cookie/token vulnerabilities, vertical/horizontal privilege escalation, decompilation issues, and second order injection will not be caught by automated solutions. Additionally, many of the more severe vulnerabilities should be documented to illustrate proof of concepts used to provide exploit root cause and potential impact scenarios to management.

In conclusion, it is my opinion that a penetration test should be used when the data or system imposes a level of risk that the business is not willing to accept and needs to ensure detective, preventative, and corrective controls are in place to safeguard the asset. This may be performed on both test and production systems and typically on an annual basis, but may be driven by regulatory requirements. The vulnerability assessment should be used as more of a detection method to provide insight into the effectiveness of simple organizational patch and configuration management standards and procedures. By no means, am I discounting the necessity of both a vulnerability assessment and a penetration test. Obviously, both have their place and should be an integral part of organizational policies.

--Chris Patten

1 comment:

  1. I especially like your comment about "unbiased outsider's expertise regarding how to provide the necessary level of infosec protection". If they do an assessment of their information security they'll have a better understanding of priorities and be able to decide on which to do first: i.e. Vulnerability Assessment or Penetration Test. An infosec assessment is at a higher level then the vulnerability assessment.