In November 2023, Elastic published CVE-2023-49921 for a security issue affecting their Elasticsearch product, specifically in the Watcher module. This bug let DEBUG-level logging record and expose the raw search results—including sensitive document contents—to log files when certain logging levels were enabled. In this article, I’ll explain the bug, who is affected, show example configurations and code, the impact, and what to do to keep your Elasticsearch data secure.

What Is Affected?

Elastic’s Watcher is an alerting feature inside Elasticsearch, often used for setting up automated monitoring and alert workflows. The issue affects:

Define watches that use the search input AND

- Have DEBUG or TRACE (or lower) level logging enabled for at least org.elasticsearch.xpack.watcher.input.search or any parent logger (org.elasticsearch.xpack.watcher.input, org.elasticsearch.xpack.watcher, or even broader loggers)

Vulnerability Details

When a Watch is triggered and executes a search input, the entire result of that search would be logged if DEBUG logging was enabled on the right logger. This means all document content returned by the search gets written to the log files.

Since Elasticsearch can store sensitive info like PII, authentication tokens, business secrets, or even user-generated data, this leak has major consequences.

Example log entry with sensitive data

[2023-11-21T02:48:23,889][DEBUG][o.e.x.w.i.s.SearchInput  ] [node-1] Search input returned:
{
  "took": 3,
  "timed_out": false,
  ...
  "hits": {
    "hits": [
      {
        "_id": "user-2891",
        "_source": {
          "username": "alice",
          "email": "alice@secretcompany.com",
          "password": "hunter2"
        }
      }
    ]
  }
}

*Anyone with access to the logs could read the password and other sensitive info right out of the file.*

Exploit Scenario

Imagine you’re running Elasticsearch for your organization. You define a Watch to alert on user activity, using a search input:

{
  "input": {
    "search": {
      "request": {
        "indices": ["users"],
        "body": {
          "query": {
            "match_all": {}
          }
        }
      }
    }
  }
}

An admin (perhaps for troubleshooting) sets the DEBUG log level

PUT /_cluster/settings
{
  "persistent": {
    "logger.org.elasticsearch.xpack.watcher.input.search": "DEBUG"
  }
}

Now every run of that Watch logs the full result—including user emails, passwords, or other personal data—to /var/log/elasticsearch/elasticsearch.log (or wherever your logs go).

If you search logs for "password" or "email", you might find entire user records unintentionally stored outside Elasticsearch’s controlled access model—potentially visible to sysadmins, IT, or anyone with log access.

In Elasticsearch’s Watcher codebase, the problematic code looked roughly like

// Pseudocode for vulnerable logging in Watcher
SearchResponse searchResponse = ... // execute search input
logger.debug("Search input returned: {}", searchResponse); // full search output!

DEBUG logging doesn’t sanitize or mask sensitive fields, so everything is written as-is.

Your Watches use a search input

- You have enabled DEBUG/TRACE log level for any affected logger

Not Affected If

- You never adjusted DEBUG/TRACE logging in these loggers

You’re on versions 7.17.16 or 8.11.2+

Versions fixed:
- 8.11.2
- 7.17.16

(patched by removing the sensitive output from DEBUG logs)

Upgrade Elasticsearch to 7.17.16 or 8.11.2+

- Download latest

`bash

PUT /_cluster/settings
{

}

}

Recommendations

- Never enable DEBUG or TRACE log level in production unless you absolutely need to, and always reset after troubleshooting.

- Elastic Security Advisory: CVE-2023-49921
- Elastic Stack 8.11.2 Release Notes
- Elastic Stack 7.17.16 Release Notes
- Watcher Documentation
- Original disclosure

Final Thoughts

CVE-2023-49921 is a textbook example of how debug logging can accidentally become a security liability. If you rely on Elasticsearch Watcher—and especially if you use custom watches or search inputs—check your logger settings and your logs. Always keep production environments shipshape with prompt updates, tight log handling, and minimal debugging.

Stay tuned for more exclusive, easy-to-follow breakdowns of real-world CVEs!

Timeline

Published on: 07/26/2024 05:15:10 UTC
Last modified on: 07/26/2024 13:47:08 UTC