Penetration Testing - Compliance, value and security

Written by Jason Charalambous on 01/09/2017

What and why?

Penetration testing, ethical hacking and white-hat hacking are just a few names people use to refer to a very interesting technical process that aims to test your security. It does this by simulating real-world attack scenarios, using all the tricks and tips available to cybercriminals at large. Organisations adopt it either because they have to (compliance to a security standard), or because they want to prove to senior management/stakeholders that their infrastructure and applications are secure, perhaps as a result of a recent change. Most often it’s the former, with standards such as PCI DSS mandating penetration tests.

Compliance is NOT security

People tend to think of pen testing in different ways, but surprisingly few organisations do it right. As you’ve gathered from my previous blogs, to make it right you must look, think, and understand. It’s not just something that can be booked in and ticked off. It’s unlikely an organisation will be able to test their whole infrastructure, usually due to budgetary or resource constraints. Therefore, to get it right you must look (assess) the risks of your business to see what’s vital to your organisation (the scope), and the test must be based around this. Next you must think about the perils and pitfalls the pen test reveals: what impact do they have? Lastly, you must understand that compliance has limitations: just because you’re compliant with Standard X does not mean you’re impenetrable. Compliance is not security.

What I mean by this is that the hacking1 process comes with limitations on the scope, the time available, and the techniques and reporting skills the pen tester has. There are also different types of tests available: White Box testing is where the tester knows the nitty gritty of the environment in question, Black Box testing is where they don’t, and Grey Box is somewhere in between. This is important to note as in real life, an attacker might find vulnerabilities in places that aren’t in your pen testing scope – plus they’ll have more time. A real-world hack isn’t a blitzkrieg; it’s a slow, methodical, deliberate process that doesn’t raise suspicion. It can take weeks or even months.

Getting the value

Not all penetration tests give the value you might think. Whilst most will identify flaws in the network infrastructure, web, standalone/mobile applications (etc), they don’t always check things like physical security. This is an oft-overlooked flavour of social engineering that checks how strong your physical security is – could someone con their way in and get access to your secure systems? Social engineering in the wider sense should be included as part of a pen test: threats are shifting from infrastructure to application. Infrastructure must be hardened, but staff must also be trained.

The skills of the person performing the penetration test matters. I have met some awesome pen-testers, malware authors and ex-blackhats over the years, especially during my CHECK training. I’ve also met pen-testers who just don’t have what it takes to do the job. I suppose I’m really talking about an X-Factor here: a good pen-tester has to think laterally as well as logically. It requires innovation, and continuous personal development in learning and programming. You’d be amazed at how many pen-testers don’t know how to program. Scriptkiddies using available tools and exploits are one thing, and they’ll give your defences a run for their money, but true hackers who know how to program can code their way in to your systems. Time is another factor: a pen-tester might not have the time to write a custom exploit to test something, but a real-world hacker will have the time to do it.


My advice is to always get a penetration tester with the right qualifications and experience to do the job. Qualifications like CREST and Tigerscheme mean a lot: they’re not just theoretical accreditations, but tests that demand pen testers perform a practical hack. To become qualified in these standards you need innovative thinking and a variety of skills in security, networks, sysadmin, programming and more.

Test over, game over?

So you set the scope, got a qualified pen-tester in and had it done. Nice! Now what? You’ll get a report outlining the outcome, but what value you get from this? Is the result something that can be interpreted, actioned and have an overall positive impact on your security, or is it a standard, non-pragmatic outcome from a template the pen-tester ran? Without addressing what needs to be achieved and what is critical to you, the pen-tester will deem what is critical based on their experience. A good pen-tester will outline in your report the root cause of the problems found in a pragmatic fashion. Not all organisations are tech savvies so the people you will employ to do this job must be capable of prioritising the problems found. They should also assist you in the whole remediation process and not just leave it to you to figure the right way to fix the flaws.


To summarise, to have a successful penetration test you must first assess the risks and find out what is vital to you. Then select the appropriate the scope based on your risk assessment, identify any limitations and have the right people to do the job for you by helping you throughout all stages of the process until your flaws are remediated and the risk is minimised. And remember: true security is a more holistic, overall approach that goes far beyond technical measures. It’s a culture.

1 It’s a scary word, isn’t it? But remember that’s what penetration testing really is: authorised, controlled hacking.

  • Bulletproof are CREST approved

    CREST approved

  • Bulletproof are ISO 27001 and 9001 certified

    ISO 27001 and 9001 certified

  • Bulletproof are Tigerscheme qualified testers

    Tigerscheme qualified testers

  • Bulletproof are a PCI DSS v3.2 Level 1 service provider

    PCI DSS v3.2 Level 1
    service provider

  • Bulletproof have 24/7 on-site Security Operations Centre

    24/7 on-site Security
    Operations Centre

Get a quote today

If you’re interested in our services, get a free, no obligation quote today by filling out the form below.

By submitting this form, I agree to the Bulletproof privacy policy.