What matters most: vulnerability scans or pen tests?

Security scanning as a strategy

All sources agree that cyber crime is increasing year on year, putting businesses small and large at increasing risk. Attacks jumped by 31% during the height of the 2020 pandemic alone, and is predicted to cost the global economy over $10 trillion by 2025. In order to stay ahead of the hackers, savvy enterprises are stepping up their security scanning regimes by using vulnerability scanning and penetration tests to uncover security flaws. But what’s the real difference between these two technologies, and are both really needed?

Some businesses are confused by vulnerability scanning (also called VA scans) and penetration testing (commonly known as pen tests), sometimes believing them to be the same thing. This might be through no fault of their own, as there are deceitful IT security companies out there that will sell a VA scan as a penetration test, and/or muddy the waters about what each one aims to achieve. Not only does this prove expensive to a business that has fallen foul of this unscrupulous behaviour, but it also puts their cyber security in jeopardy.

While vulnerability assessments and pen tests are related in that they help organisations find security flaws in security before a cyber criminal can exploit them, they each have their own approach to how this is achieved. Understanding this can help mitigate security risks.

The question is also often asked ‘vulnerability scans or penetration testing, which matters most?’ As you’ll discover from reading this article, they are both important: one can’t replace the other. A penetration test can’t substitute for a vulnerability scan and vice versa.

We will take a closer look at what they do and their differences, but in brief: a vulnerability assessment is an automated scan that checks your systems for known security vulnerabilities. A penetration test, on the other hand, is a more elaborate and manual process with the aim of getting past your security defences.

What is a vulnerability assessment?

A vulnerability assessment scans the organisation’s networks and applications to identify potential known security weaknesses. The scan uses automated tools and once completed, generates a report of the vulnerabilities found. These assessments can vary depending on what you’re scanning and who your provider is, with high-quality scans searching for fifty thousand vulnerabilities or more. VA scans typically use the CVE database – an international list of all known security vulnerabilities.

As we’ve briefly touched on, not all vulnerability scans are created equal. The outcome will depend on the scanning software, its configuration, and how up-to-date it is. The best vulnerability assessments will be as up-to-date as possible and generate detailed, useful reports that sort the vulnerabilities into severity ratings and give remediation advice. In this way, you can prioritise the security weaknesses that present the most significant risks.

Why vulnerability assessments are important

If you want to keep ahead of the bad guys who are looking to steal money, data or just do damage to your business, regular vulnerability scanning is a security measure that shouldn’t be overlooked. You need to be one step ahead of the hackers and discover and fix the vulnerabilities that they can exploit. Vulnerability scanning is uninstructive and straightforward to run without any disruption to your business. It can be scheduled to be run regularly, allowing you to stay on top of any newly discovered security flaws.

VA scanning is so useful that it’s mandated and encouraged by a range of best practices and security standards. For example, PCI DSS, which is (in)famous for its stringent requirements, demands regular VA scans. Similarly, to be certified for ISO 27001, organisations are required to have robust vulnerability management practices in place. And whilst vulnerability scanning as such is not technically an explicit requirement of the GDPR, they do require organisations that process personal data to implement security measures to identify vulnerabilities. And the best and most cost-effective way to do that is through a VA scan.

Having regular vulnerability assessments shows that your business has integrity and takes security seriously. Your clients, employees, partners and other people who interact with your business will feel safer if you have robust security measures in place. Regular VA scanning helps retain existing clients and gain new ones, by proving that you are proactive on the issue of cyber security.

However, vulnerability scanning isn't 100% effective. Similar to antivirus scans, they depend on a database of known security vulnerabilities. If the software hasn’t been updated or is low quality, weaknesses will go undetected. Plus even the best vulnerability scanning software won’t identify complex vulnerabilities that can’t be detected by using automated tools, which brings us nicely on to penetration testing.

What is penetration testing

A penetration test is a sanctioned and simulated cyber-attack that attempts to exploit vulnerabilities in a business’s systems just as a hacker would do in a real-world threat. Experienced security analysts, sometimes known as ethical hackers, but more universally called simply ‘pen testers’, will aim to identify and exploit security weaknesses in an organisation. The penetration testers will use various tools similar to those of a hacker, but also rely on creativity and experience to play out a comprehensive simulated attack.

This human factor is what makes pen-testing so effective, and it requires expert third-party services who have the expertise and impartiality to achieve the best results. No automated solution comes close to a skilled and knowledgeable penetration tester, who will have the nose to sniff out weaknesses that software alone would never find.

The scope of the penetration test will depend on the business and its requirements. Some might want to penetration test mobile apps or network infrastructure, whereas others might want to go full-on and have the penetration testers carry out social engineering attacks and red team testing to test the security posture of an organisation to the max.

Once the penetration testing has been carried out, the organisation will be provided with a detailed report that will highlight how the penetration testers managed to compromise their system and what they need to do to prevent real-life cybercriminals from doing likewise.

There are typically five steps in a penetration test:

1. Information regarding your business will be gathered.
2. Your network and systems will be scanned for vulnerabilities, often this is a VA scan!
3. Weak points found may attempt to be exploited to access your systems.
4. Once your system is accessed, it will be explored for new vulnerabilities to exploit and escalate the attack.
5. An in-depth report will be generated, detailing the controlled attack and how the organisation's systems were exploited.

After the report is delivered, it’s up to you to take the actions it advises in order to ensure the vulnerabilities can’t be further exploited in a real-world attack. Or to put it another way: fix the stuff the pen testers found.

Choosing the right company to perform a penetration test for your organisation is an important consideration. As we mentioned at the start of this article, there are some companies online that will only perform a vulnerability assessment marketed as a pen test, leaving you out of pocket and vulnerable to cyber threats. Other companies may perform actual penetration tests, but have a poor track record. It's essential that the security service provider you hire has a proven track record of previously conducted penetration tests in industries similar to yours. For more help with procuring a penetration test, read our free penetration testing white paper.

Why penetration testing is important

Undertaking pen testing enables you to evaluate your security system's weak points and robustness under controlled conditions. This will allow you to remedy any shortcomings before any hackers can attacks your system with malicious intent. Discovering the weaknesses in your existing security posture can also help you budget IT spending and know where resources are best employed.

Just as with a VA scan, regular penetration testing will build trust with those who engage with your organisation and ensure that it is ready to meet specific compliance and certification requirements. As we mentioned in our ‘ 5 reasons you need a pen test’ blog, compliance is a driver for penetration testing. Yearly penetration tests are mandated as part of PCI DSS, and most ISO 27001 assessors will encourage you to get one to help pass the ISO 27001 certification.

Vulnerability assessment vs penetration testing

The difference between a penetration test and a vulnerability scan is that a vulnerability scan is trying to scan all your systems against a list of known vulnerabilities, with no attempt or exploiting or checking them. They simply scan everything and output a report that will give you a list of patches in order of priority. A pen test on the other hand has got a human in it, an ethical hacker (or penetration tester) using their insight, ingenuity, and experience to leverage one security floor to get to another to see what damage they could potentially do.

Our expert says:

Penetration tests are much more in-depth, and they'll tell you things that a vulnerability scan never would. It's not a case of ones better than the other, they both have their place. you should do a pen test once a year, and as with any kind of test, pay attention to what the report tells you, and fix things in the report. To keep on top of your security in between your pen tests run vulnerability scans because patches are coming out all the time for newly discovered security flaws.

Purpose

The goal of a vulnerability scan is to discover as many known vulnerabilities as possible with the goal of producing a list. A pen test goes much deeper and will look to identify vulnerabilities that are more complex in nature and that automated scanning tools wouldn't detect. Pen testing will also go a step further by attempting to not just identify, but exploit those vulnerabilities.

Scope

Vulnerability scanning is broad in scope and scans all devices with an IP address including desktop computers, laptops, mobile devices, printers, routers, firewalls and servers and more. Though you can tailor your scan to look at whatever particular assets you’d like. Pen tests are more narrow in breadth and will take a more bespoke approach. The area of scope and what needs testing should be defined from the start. Penetration tests are a more active test, so care needs to be taken that the penetration test won’t affect live or critical systems.

Skills required

Vulnerability scanning is something that can be conducted by an in-house IT department with the required technological knowledge and access to a scanning engine. A typical in-house IT department could both procure and perform the scan, as well as understand the fixes that need to be made. In contrast, a pen test is a complex procedure that needs a high-level of expertise and experience. A reputable third-party cyber security service will be required to undertake the task. Most reputable providers will be able to give you peace of mind that they can be trusted. For example, here’s a blog from Bulletproof about how we hacked a leading information security expo.

Duration of vulnerability scanning compared to pen testing

How long it takes to conduct a vulnerability scan will depend on how many assets you wish to scan, but generally takes a few hours for a large business. The time it takes to conduct a pen test will also depend on several factors, primarily the depth of the testing. As we saw in our penetration testing infographic, a pen test engagement typically takes from 2-5 days to complete, with a report following a few days after.

Duration of vulnerability scanning compared to pen testing

The cost of a VA or a pen test will vary greatly depending on factors such as your current infrastructure, systems, size and scope. Vulnerability scanning can be performed in house, but it’s generally more cost-effective to outsource to a reputable third party. Still, the automated scanning software should be up to date and an IT professional with the requisite skills on hand to carry out the scan. For many small to medium-sized enterprises, it makes more sense to use an experienced third-party vendor for their vulnerability scanning requirements.

Also, an accredited security professional is often required to perform the vulnerability scan to meet compliance regulations for some businesses. Penetration testing is more expensive due to its greater scope, the duration of the test and the expert security professionals required to perform the test. If a cybersecurity services provider tells you that a pen test and a vulnerability scan are the same prices, this should raise an immediate red flag.

Frequency of vulnerability scanning and penetration testing

While a pen test is much more expensive than a VA, it is recommended that it is only undertaken every six months to a year. A vulnerability scan meanwhile is recommended on a regular monthly basis. However, if you are making any significant changes in infrastructure in your organisation, then it would be best to follow up with a pen test to mitigate any new risks.

In Conclusion

So to answer the question of vulnerability scanning vs penetration testing, which is the most important, we can see that both strategies are necessary. They achieve different goals and are enacted differently, so they’re complementary, not competing, services. As part of a comprehensive and proactive security testing process, a monthly vulnerability scan and a bi-annual penetration test are recommended. Regular pen tests help you scour for hard-to-find security flaws, and between tests you can guard against any newly-disclosed security vulnerabilities with more regular VA scans.