CVE-2023-39951 - How OpenTelemetry Java Instrumentation Leaked Your Email Content via AWS SES

OpenTelemetry has become a backbone for modern application tracing, providing engineers with deep insights into application health and performance. However, as with any powerful tool, it comes with its own share of risks. CVE-2023-39951 highlights a serious issue in OpenTelemetry Java Instrumentation that put confidential email data at risk—especially if you used it with AWS SES before version 1.28..

In this long read, we’ll break down what happened, who’s affected, how someone could exploit the bug, and—most importantly—how you can fix it. You’ll also see what the code looked like that caused the headache, and where you can find more information straight from the source.

What is CVE-2023-39951 All About?

CVE-2023-39951 is a vulnerability in OpenTelemetry Java Instrumentation, specifically for users instrumenting the AWS SDK v2 with Amazon Simple Email Service (SES) v1 API.

Here’s the core problem

- When your Java app sends emails using AWS SES (v1 API) and has OpenTelemetry Java Instrumentation (version < 1.28.) enabled, the *body* of your outgoing email—including things like the email subject and message—gets copied into the telemetry trace metadata.
- This happens because the code included all HTTP query parameters (which in SES POST requests contain email contents) in the trace’s url.path field.
- Traces are then sent to your telemetry backend. Now, anyone with access to your telemetry system can see potentially sensitive email contents, even if they have no access to your SES account.

Impact: Anyone using any OpenTelemetry Java Instrumentation version before 1.28. to instrument AWS SDK v2—and sending emails through SES v1—may have leaked email data into logs and monitoring systems.

Why Did This Happen?

In AWS SES v1, email content (recipient, subject, message) is sent as query parameters in a POST request. OpenTelemetry Java Instrumentation accidentally included these parameters in the URL path during tracing.

Here’s a simplified code example that demonstrates the mistake

// Hypothetical tracing logic before v1.28.
String urlPath = httpRequest.getUrl().getPath() + "?" + httpRequest.getUrl().getQuery();
// In SendEmail, query parameters contain: To, Subject, MessageBody (all sensitive)
tracer.spanBuilder("http.request")
      .setAttribute("url.path", urlPath)
      .startSpan();

What goes wrong here?

- httpRequest.getUrl().getQuery() grabs everything after the “?” in the URL, which, for SES's v1 POST, contains all the private contents of the email.

This is not just a privacy issue—it’s a data security problem.

Just using OpenTelemetry Java Instrumentation as recommended, with default configurations, was enough to trigger it.

No hacking is required. Here’s a realistic attack scenario

1. Alice’s team uses OpenTelemetry Java Instrumentation (older than v1.28.) to trace their microservices.

Bob, a member of operations, has access to the telemetry backend to diagnose performance.

4. Bob queries for traces and sees url.path metadata in traces that show not just the endpoint, but the *actual email subject and entire message body*.
5. Anyone with access to telemetry logs can read emails sent by the service—even if they aren’t authorized to access emails in production.

No external hacker is needed—insiders or even just misconfigured telemetry viewers have access to sensitive data.

How to Fix the Problem

The official fix is simple: upgrade OpenTelemetry Java Instrumentation to version 1.28. or newer.

If you absolutely cannot upgrade right away (e.g., due to testing cycles), consider

- Manually filtering or masking url.path fields containing SES API calls in your telemetry pipeline.

Let’s see a sample attack using the default behavior before v1.28.

import software.amazon.awssdk.services.ses.SesClient;
import software.amazon.awssdk.services.ses.model.SendEmailRequest;
import software.amazon.awssdk.services.ses.model.Message;
import software.amazon.awssdk.services.ses.model.Body;
import software.amazon.awssdk.services.ses.model.Content;

// Simulate a SendEmail call
SendEmailRequest request = SendEmailRequest.builder()
    .destination(...) // set destination
    .message(Message.builder()
        .subject(Content.builder().data("PRIVATE SUBJECT").build())
        .body(Body.builder().text(Content.builder().data("PRIVATE MESSAGE").build()).build())
        .build())
    .build();

SesClient client = SesClient.builder().build();
client.sendEmail(request);

// With OpenTelemetry Java Instrumentation <1.28. enabled, the trace will include:
// url.path: /?Action=SendEmail&Destination=...&Subject=PRIVATE+SUBJECT&Body=PRIVATE+MESSAGE

Check your traces! If you see email content like this in the metadata, you’re affected.

Official References and Further Reading

- OpenTelemetry Security Advisory: CVE-2023-39951 - Advisory on GitHub
- NIST CVE Database Entry: CVE-2023-39951
- Release Notes for Fix: OpenTelemetry Java Instrumentation v1.28.

In Summary

CVE-2023-39951 shows that even tracing tools can accidentally become leaks for your most sensitive data. If you use OpenTelemetry Java Instrumentation to trace AWS SES v1 email sends, update to v1.28. or newer ASAP, and check your trace logs for any accidental exposures.

Stay safe. Keep monitoring—but don’t leak your secrets!

- Bookmark the official CVE advisory for reference.

Timeline

Published on: 08/08/2023 22:15:00 UTC
Last modified on: 08/16/2023 16:44:00 UTC