Subject: Apache Log4j2 Remote Code Execution (RCE) Vulnerability - CVE-2021-44228 ESA-2021-31

​​Note — We will update this announcement with new details as they emerge from our analysis. Please check back periodically.

Last Update: Dec 15, 2021 - 22:30 UTC

Summary

A high severity vulnerability (CVE-2021-44228) impacting multiple versions of the Apache Log4j 2 utility was disclosed publicly via the project’s GitHub on December 9, 2021. The vulnerability impacts Apache Log4j 2 versions 2.0 to 2.14.1.

This announcement summarizes the currently known potential impacts to Elastic products and related announcements for mitigations of the issue. Elastic engineering and security teams continue to actively work on the analysis and any actions our users should perform, alongside detection signatures that may be used to identify exploitation of the vulnerability.

[Update 15 December]

A further vulnerability (CVE-2021-45046) was disclosed on December 14th after it was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations. Our guidance for Elasticsearch, APM Java Agent, and Logstash are unchanged by this new vulnerability.

Elasticsearch

Supported versions of Elasticsearch (6.8.9+, 7.8+) used with recent versions of the JDK (JDK9+) are not susceptible to either remote code execution or information leakage. This is due to Elasticsearch’s usage of the Java Security Manager. Most other versions (5.6.11+, 6.4.0+ and 7.0.0+) can be protected via a simple JVM property change. The information leak vulnerability does not permit access to data within the Elasticsearch cluster. We have released Elasticsearch 7.16.1 and 6.8.21 which contain the JVM property by default and remove certain components of Log4j out of an abundance of caution. This is applicable to both CVE-2021-44228 and CVE-2021-45046. Additional details below.

Elastic Cloud

Our security testing has not identified any exploitable RCEs against any Elastic Cloud products. Our investigation continues and we will provide updates of any new findings. As a normal practice we will update components with the latest version of Log4j as they become available. We recommend that users who are on versions before 7.2 restart their deployments to pick up an updated setting. Additional details below.

APM Java Agent

APM Java Agent is only known to be exploitable when the Agent is configured with log_level=trace and at the same time using a PLAIN_TEXT log format. Additional details below

Elastic Cloud Enterprise

Our security testing has not identified any exploitable vulnerabilities related to this issue. We are continuing to analyse the issue and will advise with any updates. As a normal practice we will update any components which include the vulnerable Log4j versions in the next release. Mitigation details for an Elasticsearch cluster managed by ECE are below.

Elastic Cloud on Kubernetes

While the ECK Operator is not impacted by this vulnerability, mitigation details for an Elasticsearch cluster managed by ECK are below.

Logstash

Exposure to remote code execution exists on JDKs prior to 8u191. On newer versions of JDKs there is exposure to Denial of Service and information leakage, but no known remote code execution exposure. Mitigation requires removal of the JndiLookup Class or updating to Logstash version 6.8.21 or 7.16.1, which have been released on December 13th. Additional details below.

Swiftype

Our security testing has not identified any exploitable RCEs against Swiftype products. Our investigation continues and we will provide updates of any new findings. We have mitigations in place as a precaution and will update components with the latest version of Log4j as they become available.

Not Impacted

We have validated that the vulnerability does not exist in the following Elastic products:

• APM Server
• Beats
• Cmd
• Elastic Agent
• Elastic Cloud on Kubernetes
• Elastic Endgame
• Elastic Maps Service
• Endpoint Security
• Enterprise Search
• Fleet Server
• Kibana
• Machine Learning

---

APM Java Agent Code Execution issue (ESA-2021-31)

In versions 1.17.0-1.28.0, the only known way the Log4j vulnerability may be exploited is when the APM Java Agent is configured with log_level=trace and at the same time using a PLAIN_TEXT log format (log_format_stdout/log_format_file=PLAIN_TEXT).

Affected Versions:

Versions 1.17.0-1.28.0

Solutions and Mitigations:
Users running affected versions should upgrade to the latest version (1.28.1 or newer).

In versions 1.17.0-1.28.0, the issue can be mitigated manually by setting system property -Dlog4j2.formatMsgNoLookups=true.

[Update December 15th]

Version 1.28.1 of the Elastic APM Java agent also mitigates CVE-2021-45046, as we have excluded the vulnerable JndiLookup class. This version still uses log4j 2.12.1 but the vulnerable code is not present.
We will release 1.28.2 soon, which ships with a patched log4j version (2.12.2) that should not trigger false positive alerts from vulnerability scanners.

---

Elasticsearch announcement (ESA-2021-31)

Elasticsearch 6 and 7 are not susceptible to remote code execution with this vulnerability due to our use of the Java Security Manager. Elasticsearch running on JDK8 or below is susceptible to an information leak via DNS which is fixable by the JVM option identified below. This option is effective for Elasticsearch versions 5.6.11+, 6.4+, and 7.0+.

As of December 13, 2021, we have released Elasticsearch 6.8.21 and 7.16.1 which set the JVM option identified below and remove the vulnerable JndiLookup class from Log4j out of an abundance of caution. If you are on a 6.x version prior to 6.4.0 and upgrading is not possible, you can follow the instructions here.

Elasticsearch 5 is susceptible to both remote code execution and an information leak via DNS. For versions 5.6.11 - 5.6.16, this can be mitigated by setting the JVM option. Users on an earlier version of 5.x, are recommended to upgrade to 5.6.16. If you are on a 5.x version prior to 5.6.11 and upgrading is not possible, you can follow the instructions here. Please note that while we provide these remediations, Elasticsearch 5 is not a supported version, and we always recommend updating to the latest release.

Elasticsearch 2 and earlier used a Log4j version that is not vulnerable to the newly discovered flaw. Please note that Elasticsearch 2 is not a supported version, and we always recommend updating to the latest release.

For users running on Elastic Cloud, versions 7.2+ have never been susceptible to either the RCE or the information leakage as these versions already run on JDK11 or higher. We recommend users running any version of Elasticsearch earlier than 7.2 restart their clusters as soon as possible - the JVM option identified below will automatically be applied and fully protect clusters on restart. Any new clusters will be deployed with the JVM option included. See Elastic Cloud announcement for more details

Affected Versions:

Elasticsearch versions 5.0.0+ contain a vulnerable version of Log4j. Elasticsearch 5 is susceptible to both remote code execution and an information leak via DNS. We’ve confirmed that the Security Manager mitigates the remote code execution attack in Elasticsearch 6 and 7.

Solutions and Mitigations:

The simplest remediation is to set the JVM option -Dlog4j2.formatMsgNoLookups=true and restart each node of the cluster.
For Elasticsearch 5.6.11+, 6.4+, and 7.0+, this provides full protection against the RCE and information leak attacks.

Users may upgrade to Elasticsearch 7.16.1 or 6.8.21, which were released on December 13, 2021. These releases do not upgrade the Log4j package, but mitigate the vulnerability by setting the JVM option -Dlog4j2.formatMsgNoLookups=true and remove the vulnerable JndiLookup class from the Log4j package.

Note: In both of these scenarios, some vulnerability scanners may continue to flag Elasticsearch in association with this vulnerability based on the Log4j version alone. However, any of the above mitigations sufficiently protect both remote code execution and information leakage.

If Elasticsearch is managed by ECK, set the JVM option in the Elasticsearch custom resource podTemplate specification

If Elasticsearch is managed by ECE, for versions 6.x and <7.2, we recommend reinstalling stackpacks, which have been patched to include the JVM option mitigation. After re-installing relevant stackpacks, we recommend restarting deployments. For the 5.x series, we recommend overriding the JVM options to add the property that will mitigate the vulnerability, and restart the cluster to pick up the change: For details and guidance, please reach out to Elastic support.

[Update Dec 14th]

Log4j 2.16.0 has been released to address CVE-2021-45046. This does not change the mitigation guidance for Elasticsearch described above, that does not require an update to Log4j 2.16.0. Elastic guidance remains to either apply the JVM option described above and restart all nodes, or upgrade Elasticsearch to 7.16.1 or 6.8.21.

Details on Elasticsearch information leakage

The information leakage vulnerability in Log4j enables an attacker to exfiltrate certain environmental data via DNS - it does not permit access to data within the Elasticsearch cluster. The data that can be leaked is limited to those available via Log4j “lookups”, which includes system environment variables and a limited set of environmental data from other sources. For a complete list, see the Log4j Lookups documentation.

Notes of PoCs expanding RCEs to recent Java versions**

We are actively monitoring developments in the security community, such as this one, which seek to expand the JDKs and scenarios where this exploit will apply. Our implementation of the Java Security Manager in Elasticsearch 6 and 7, in combination with JDK9 or greater, continues to protect against all known PoC’s. While these efforts seek to provide a viable RCE even when com.sun.jndi.ldap.object.trustURLCodebase=false (as in recent JDKs), our Security Manager cuts off the attack earlier in the process, preventing both remote and local (on the class path) variants of the attack.

---

Logstash announcement (ESA-2021-31)

A high severity vulnerability (CVE-2021-44228) impacting multiple versions of the Apache Log4j 2 utility was disclosed publicly via the project’s GitHub on December 9, 2021. Logstash uses Log4j as its logging subsystem and may be vulnerable.

When running Logstash on JDKs older than 8u191 and 11.0.1, an attacker is able to inject and execute a remote Java class. On recent JDKs the attack is limited to DoS - causing data ingestion to temporarily stop - and information leakage, but no remote code execution attack vectors are known.

Affected Versions:

Logstash versions 5.0.0+ up to and including 7.16.0 contain a vulnerable version of Log4j. The severity depends on the JDK used as stated above.

Docker images below version 6.4.3 include a JDK older than 8u191, which means they are open to Remote Code Execution. Images 6.4.3+ don't have known RCE attacks but are still susceptible to Denial of Service and information leaks.

Solutions and Mitigations:

Users should upgrade to Logstash 7.16.1 or 6.8.21, which were released on December 13, 2021. These releases replace vulnerable versions of Log4j with Log4j 2.15.0.

The widespread flag -Dlog4j2.formatMsgNoLookups=true is NOT sufficient to mitigate the vulnerability in Logstash in all cases, as Logstash uses Log4j in a way where the flag has no effect. If the user cannot upgrade to Logstash 7.16.1 or 6.8.21, it is necessary to remove the JndiLookup class from the log4j2 core jar, with the following command (which is applicable for 5.x, 6.x, and 7.x):

zip -q -d <LOGSTASH_HOME>/logstash-core/**/*/log4j-core-2.* org/apache/logging/log4j/core/lookup/JndiLookup.class

Please note that a restart of the Logstash process is necessary for the change to take effect.

[Update Dec 14th]

Log4j 2.16.0 has been released to address CVE-2021-45046. Logstash is not impacted by the vulnerability disclosed in CVE-2021-45046 because Logstash does not ship with logging layouts that can be exploited to trigger JNDI lookups through Thread Context references. Elastic guidance remains to either remove the JndiLookup.class or upgrade Logstash to 7.16.1 or 6.8.21.

[Update Dec 15th]

Users have noticed that the newly released Logstash 6.8.21 contains a jar that bundles a vulnerable version of log4j-core. Logstash core will load the Log4j 2.15.0 from core instead of any other log4j-core jar in plugins, so no Jndi Lookups will happen.

The fact that the logj4-core 2.15.0 from core is the only one loaded has been confirmed through code analysis and experimentation, but if users are required to remove the class, the command is:

zip -q -d <LOGSTASH_HOME>/vendor/**/*/logstash*tcp*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

---

Elastic Cloud announcement (ESA-2021-31)

Our security testing has not identified any exploitable RCEs against any Elastic Cloud products. Our investigation continues and we will provide updates of any new findings. As a normal practice we will update components with the latest version of Log4j as they become available. Users runningElastic Cloud versions 7.2+ have never been susceptible to either the RCE or the information leakage as these versions already run on JDK 11 or higher.

Solutions and Mitigations:

Deployments hosted in Elastic Cloud have been updated to leverage the JVM Option -Dlog4j2.formatMsgNoLookups=true which will take effect on restart of the deployment and on any config change to the Elasticsearch deployment.

For users running a cluster on a minor version older than 7.2, we recommend a restart.

The simplest way to restart a deployment is to do the following:

1. Log in to the Cloud UI. Navigate to the “deployments” section in Elasticsearch Service.
2. Select the deployment you’d like to restart.
3. In the “manage” menu, select “restart”. Any kind of restart will work: “no downtime” restarts are fine.

---