Bots: how worried should we be?

Written by Joseph Poppy on 19/10/2018

Prove you are a human. If you think about that sentence for too long, you realise it’s actually incredibly complex and can bring about a sense of existential angst. Yet, it’s something that is demanded of us on a near daily basis, sometimes more. It turns out, proving our humanity doesn’t require showing the capacity to love, or even passing Blade Runner’s Voight-Kampff test. Rather, we just need to be able to click on pictures of cars or shop fronts.

Why do we have to do this? Well, because the bots are here and they’re here in huge numbers. In fact, a recent study suggests that of all Tweets that share links (usually news related), 66% are suspected to be from bots. In 2017, 21.8% of website traffic was reportedly from ‘bad bots’, with another 20.4% of website traffic from ‘good’ bots. Those of you good at maths will have realised that this means close to a half of all website traffic is not from a human. Is this something we should be worried about?



The good, the bad and the ugly

What do we mean by good bots and bad bots? There are numerous good bots out there, doing repetitive and useful things efficiently and with minimal complaints. Their use is somewhat varied. For example, copyright bots are drifting through the varied and often treacherous waters of the internet, flagging up plagiarised content or illegal uploads (think torrenting). Their effectiveness could well be called into question as there’s still a lot of that going on, but they are there nonetheless.

There are also data bots that provide up-to-the-minute information on a wide variety of subjects (news, weather, trending topics etc.). Some of these bots are even deployed by business marketing teams. Google, Amazon and Apple all make extensive use of these.

Then there are spider bots or “crawlers” as they are sometimes known. These crawl (makes sense now) the web. These are mainly used in search engines for SEO purposes, feeding data to various algorithms.

On the other hand, you have your bad bots. You are perhaps more familiar with these. There are many examples I could name, but here are the main offenders:

  • Click bots that generate advertisement clicks, skewing data for marketers (and wasting money if you’re running PPC campaigns).
  • Spam bots that have been around forever and bombard sites with nonsense comments or send out junk email en masse.
  • Scraper bots that scrape up content to post elsewhere. This can have negative consequences on your site’s SEO if Google deems this as duplicate content.

I got 99 problems... they’re all bots

A group of connected bots is a botnet. You’ve probably heard of them. These can be particularly dangerous. Their recent prevalence is, in part, due to the sheer number of devices that now come complete with network connectivity. These Internet of Things (IoT) devices can become unwitting hosts for bots just as any server or computer can.

A DDoS attack taking your services offline through a flood of traffic is arguably the most obvious problem with botnets, but there are other, more complex issues too. Returning to the statistics, if 66% of link sharing Tweets are from bots, then logically speaking, 66% of the links we see are going to be pushed by them. Whoever can influence the most bots (whoever controls the largest botnets), can influence what an audience of millions see on a regular basis. Suddenly fake news goes from an irritation to a significant influencer of global politics. And I’m sure we can all agree, bots influencing democratic processes is a scary idea.

A scraper bot, taking content from your site and posting it elsewhere could do more than just damage your SEO rankings. If your content ends up on a less than reputable site, then your reputation could be negatively affected.


Downright dirty

Bots can even be designed to drop various forms of malware onto computers, servers and all manner of internet devices. The possibilities here are almost endless. Bots could (and do) end up creating complex cryptomining networks, dropping spyware, installing ransomware, or syphoning off data. As well as the delivery vector, botnets are also used as decentralised command-and-control systems. The bots are now telling other bots what to do.


There is hope

Fortunately, we are more aware of the prevalence of bots, and big companies are working hard trying to get rid of them (the bad ones). Twitter has recently gone through a purge, reportedly deleting tens of millions of bots. Facebook are also reportedly taking steps to tackle the problem (though as recent news has proven, what Facebook says and what they do are often two different things). Google has launched an initiative to combat the prevalence of fake news, which is often pushed or heavily promoted by bots.


But there is also fear

AI and machine learning are both becoming big things in the world of cyber security, for both good and bad. Bots can and will start getting more advanced as time goes on. It is already getting harder to tell a sophisticated bot and the average internet using human apart. Increasing numbers of website live chats are actually bots masquerading as humans. It’s likely that we’re going to have to come up with other ways of separating them. Proving you're human might become a lot more difficult.

CAPTCHAs which made use of words used to be a common way of doing this. These words would usually be distorted or edited in such a way that only a human should be able to interpret them. These protected accounts from being overtaken or brute forced, or protected a website from spam and the like. However, machine learning algorithms began to recognise these words, which is why it’s far more common to be tested by images now. CAPTCHAs (or reCAPTCHAs) tend to use images and instruct users to pick out those linked by a theme (like how many have cats). The idea here is that humans can analyse pictures much better than robots can.

...for now.

Because we will never learn, no matter how many science fiction films we see, humans have been working with AI to improve their ability to recognise and produce convincing images. Theoretically, this could lead to machines that can analyse images as well as any human, meaning these defences we have in place will become all but useless.


Bots are bots

Of course, bots are ultimately tools and are only really a problem because people use them to achieve nefarious ends. As bots become more sophisticated, so will the ways in which we detect and defend against them. At Bulletproof we’re already building machine learning systems to create ‘good bots’ to help us detect bad bots.

In terms of how worried we should be, well... that’s a difficult one. Trends come and go. We know bots can do a great deal of damage We know they can cause a DDoS, in which case we can have DDoS mitigation strategies in place. We know they push news stories, spam comment boards or push a political agenda. It could be argued that the more aware we become of this, the less effective these campaigns are. The more we know about bots, the less likely we are to take a tweet seriously. What we should be worried about the most, is the next creative way bots are used by hackers or malicious parties. This is an unknown parameter. We could well be taken by surprise.

For the time being however, if you’re vigilant and your site is well protected and up to date, you can avoid most of the current bot related shenanigans. Anti-brute force mechanisms and reCAPTCHAs are a must and a quality SIEM system should be able to detect if your network is being scanned by suspicious entities (which can be and often are bots).


  • Bulletproof are CREST approved

    CREST approved

  • Bulletproof are ISO 27001 and 9001 certified

    ISO 27001 and 9001 certified

  • Bulletproof are Tigerscheme qualified testers

    Tigerscheme qualified testers

  • Bulletproof are a PCI DSS v3.2 Level 1 service provider

    PCI DSS v3.2 Level 1
    service provider

  • Bulletproof have 24/7 on-site Security Operations Centre

    24/7 on-site Security
    Operations Centre

Get a quote today

If you’re interested in our services, get a free, no obligation quote today by filling out the form below.