Pentests are useless if the preconditions are not right
Pentests are a good way of finding overlooked security gaps. But the conditions must also be right, comments Janis König.
(Image: peterschreiber.media/Shutterstock.com)
- Janis König
We want to do a pentest. This is how many of my customer conversations began for some time - sometimes with the variation "have to" instead of "want to". But why do we pentest at all?
Like other advanced security mechanisms, pentests are based on previously implemented security measures. Without them, their usefulness is significantly reduced. This is because pentests are a tool for verifying the effectiveness of such measures – if there are no measures, they will of course work. But what exactly do pentests verify? I don't need a pentest to prove the presence of security vulnerabilities – they exist, I guarantee it.
Step one: Fix known problems
Pentests are limited in their scope and will only find problems selectively, on a random basis. However this also means that if I want to "become secure", it is not enough to fix the issues found. Pentests are too expensive for that. A lot has to happen beforehand: static code analysis in the CI during development, vulnerability scanners during deployment, review processes, staff training. If this is not in place, there is no need to start with a test: It is cheaper and quicker to go to the development department or the admins. They often know which corners are full of dirt.
If the security processes are in place (and being used), a pentest is a good way to have third parties check the code or the network for points that have been overlooked. And then it's on to the next process: increasing coverage so that this type of gap cannot occur in the future.
Fulfilling the basics instead of overwhelming with checklists
Of course, you can also commission pentests before introducing all these processes to improve the visibility of problems and gain an overview. But all too often you then run into Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." Attempts are made to overwhelm the abundance of findings with checklists instead of tackling the basics. You run from one immediate measure to the next - and in the end the Excel spreadsheet may be green, but after the next pentest everything will be red again. Often even with the same gaps: With the next feature/roll-out, the same mistake has been made, just in a different flavor. There is no lasting learning effect.
Last but not least: pentests are not attack simulations. The latter tests the effectiveness of defensive measures such as a Blue Team, a SOC or SIEM, EDR/XDR, etc. – all mechanisms that aim to identify attack methods and suspicious behavior. A simulation that is as realistic as possible is of course essential for this.
Pentest should find your gaps!
However, a pentest should find vulnerabilities, and as many ones as possible. This also means that a supposedly "realistic" black box test, in which the pentesters – like an attacker – know nothing about the infrastructure to be tested, only generates effort for the testers. And due to less accurate results, it also causes additional work for those being tested. And then the results are also less comprehensive, as a large part of the paid time was wasted on orientation and searching for information. It is therefore better to reduce the scope, but to carry out the test as a white box with the necessary prior knowledge. Then you can at least do something with the results.
(pst)