While amazee.io has rolled out an infrastructure wide mitigation against CVE-2021-44228 on Solr 7 & 8 via the `log4j2.formatMsgNoLookups` system property, we know today that this mitigation does not cover every possible exploit against log4j. see https://logging.apache.org/log4j/2.x/security.html.
Therefore we suggest all customers to upgrade their Solr containers/pods as soon as possible: - Customers that reference `amazeeio/solr` or `uselagoon/solr` images (without specific tagged versions) in their `docker-compose.yml` file just need to trigger a redeployment, Lagoon will automatically pick up the newest Solr Base Image. - Customers that specifically define a version number of the Lagoon Base Images they need to use the tag `21.12.1` (like `uselagoon/solr-7:21.12.1`) to specifically use the newest images in their source code and trigger a deployment.
Posted Dec 16, 2021 - 02:46 UTC
Update
The log4j security team has released a new version of log4j 2.16.0 which addresses CVE-2021-45046 https://logging.apache.org/log4j/2.x/security.html This new vulnerability is classified as moderate and therefore usually does not warrant an emergency maintenance or release of Lagoon images. Based on the recent release of a highly critical release of log4j just a couple of days ago, the amazee.io security team decided to review and publish information about CVE-2021-45046. Additionally the log4j security team announced that previous mitigations against CVE-2021-44228 via the system property `log4j2.formatMsgNoLookups` are not anymore considered mitigations against the vulnerability in CVE-2021-44228. This system property has been used by amazee.io for an infrastructure wide mitigation and is still in place, it will protect in most situations but there are specific cases where it's still possible that the vulnerability could be triggered. Therefore we require our customers to upgrade to versions of Lagoon Base Images (currently only Solr) that include a patch for CVE-2021-44228 and CVE-2021-45046.
Lagoon Base Images: - Solr 7 and 8: We believe that Solr 7 and 8 are possibly vulnerable to CVE-2021-45046. There is no mitigation from an infrastructure point of view possible. We are waiting for the Solr community to release new versions of Solr 7 and 8 which include log4j 2.16.0 and will release new versions of Lagoon Base Images as soon as these new Solr Versions become available. You can follow along this process here https://github.com/uselagoon/lagoon-images/issues/364 - Solr 6: There is no mitigation required as it does not use log4j version 2 - Elasticsearch 6 and 7: There is currently no information that they are vulnerable to CVE-2021-45046, especially as they are completely protected from CVE-2021-44228 in the first place, see https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability-cve-2021-44228-esa-2021-31/291476.
Lagoon Logging Infrastructure: - Our Logging Infrastructure is based on Elasticsearch 7 which is currently considered not vulnerable to CVE-2021-45046
We are continuing monitoring the development of this log4j vulnerability and will update this page accordingly if new information should be released.
Posted Dec 15, 2021 - 01:59 UTC
Update
After additional information has been released by the maintainers of Solr and Elasticsearch we can announce that Solr 6 and Elasticsearch 6-7 are not vulnerable to the exploit of CVE-2021-44228:
Therefore the amazee.io security team has decided to: - Remove the mitigation for all Elasticsearch pods across our infrastructure, this work has been concluded a couple of minutes ago - Leave the mitigation for all Solr pods in effect as this protects Solr 7 and 8 which are currently assumed to be affected by the vulnerability.
Posted Dec 13, 2021 - 19:58 UTC
Monitoring
All Solr containers of Version 7, 7.7 and 8; Elasticsearch 7 and 8 are now mitigated against CVE-2021-44228.
Solr 6 is still vulnerable and there is no possibility for mitigation on an infrastructure or platform level. We strongly recommend customers still using Solr 6 to upgrade to newer Solr versions in order to mitigate against this any other RCE vulnerabilities of Solr.
Posted Dec 10, 2021 - 21:45 UTC
Identified
We are performing an emergency maintenance to patch all containers running Solr versions 7, 7.7, and 8 and Elasticsearch 6 and 7 derived from the Lagoon base images across our infrastructures against this CVE. Further information can be found in the GitHub discussion here: https://github.com/uselagoon/lagoon-images/issues/357
Please note that this patch will not protect sites running older versions of Solr, notably version 6, which is well beyond its End Of Life date. If your site uses Solr 6, it will still remain vulnerable to this exploit. We recommend the maintainers of these sites upgrade their Solr version as soon as possible in order to be protected from this and other RCE vulnerabilities open against older versions of the log4j packages.
Posted Dec 10, 2021 - 20:48 UTC
This incident affected: Germany (de3.lagoon), Finland (fi2.lagoon), Australia (au2.lagoon), USA (us2.lagoon, us3.lagoon), United Kingdom (uk3.lagoon), and Switzerland (ch4.lagoon).