Requirements gathering for manual/automated penetration should be based on a multitude of items. This is extremely important as many businesses are different and their requirements may not necessitate a complete penetration assessment. Some of these requirements may be:
- Sensitivity classification associated with the resident system/app data
- What environment does the system/app currently reside
- What are the requirements for service availability
- Security reviews as an integral step of the SDLC process
- Impact to the business (reputation/revenue) if compromised
The important item to note is that vulnerability assessments and penetration tests are very different but are still very dependent on each other. For instance, before a penetration test can be executed, some level of information and vulnerability gathering is necessary. This typically, comes in the way of a vulnerability assessment. The goal is to document/map application and system vulnerabilities back to industry security standards that may be used for overall remediation. The vulnerability assessment can stop there and the outcome may be for an organization to remediate the findings ranked by severity. It is worth noting that my personal opinion, and that is all it is, leans towards the fact that any mention of an automated penetration test is simply a vulnerability assessment as much of the human intervention, observation, and extrapolation is removed from the process. Although for quick hit items like vulnerable services at a network level (eg. exposed SNMP, mgmt interfaces, defaults, etc...), this may be an adequate solution. The security issues identified within this process, along with manually derived vulnerabilities, will be investigated further during the penetration testing phase if authorized by the business.
The Penetration test should, if performed correctly, identify items that a vulnerability assessment (aka. 1-click pen test) typically cannot find. This is most prevalent within web application testing, for instance. Items like concurrent session management, advanced cookie/token vulnerabilities, vertical/horizontal privilege escalation, decompilation issues, and second order injection will not be caught by automated solutions. Additionally, many of the more severe vulnerabilities should be documented to illustrate proof of concepts used to provide exploit root cause and potential impact scenarios to management.
In conclusion, it is my opinion that a penetration test should be used when the data or system imposes a level of risk that the business is not willing to accept and needs to ensure detective, preventative, and corrective controls are in place to safeguard the asset. This may be performed on both test and production systems and typically on an annual basis, but may be driven by regulatory requirements. The vulnerability assessment should be used as more of a detection method to provide insight into the effectiveness of simple organizational patch and configuration management standards and procedures. By no means, am I discounting the necessity of both a vulnerability assessment and a penetration test. Obviously, both have their place and should be an integral part of organizational policies.