OpenSSL is a staple of modern network cryptography, relied on by millions of servers and clients worldwide. Naturally, any flaw discovered in this library receives immediate, global attention. In June 2024, a subtle but potentially damaging buffer overread — CVE-2024-5535 — surfaced, highlighting how even rare programming errors can compromise confidentiality.

In this post, we’ll unpack CVE-2024-5535 in simple terms, look at the affected code, explain the potential impact, and discuss how an accidental misconfiguration or programming mistake could turn your application into a risk factor — even without hostile intent.

Summary of the Issue

Calling the OpenSSL API function SSL_select_next_proto() with an empty client protocols buffer (that is, a protocol list of zero length) can cause the function to access memory it shouldn’t. This mistake might crash your app, or worse, send up to 255 bytes of sensitive memory data to the peer you’re talking to.

- Root Cause: The function doesn’t check if the protocol list from the client is zero length, and instead reads stray memory.
- Impact: If the botched call’s output makes it across the wire, up to 255 bytes of private or otherwise sensitive memory could leak.
- Severity: The risk is assessed as low because it needs a programming/configuration error, AND it’s rare for anyone to use the old NPN protocol (the main area of exposure).

Where It Could Happen

The vulnerable function is meant for use where two sides need to decide on which application-level protocol to speak (for example, HTTP/2 or HTTP/1.1).

- NPN (Next Protocol Negotiation) — Deprecated and largely replaced by ALPN, but still found in some older deployments.

SSL_select_next_proto is called like this

int SSL_select_next_proto(unsigned char **out, unsigned char *outlen,
                         const unsigned char *server,
                         unsigned int server_len,
                         const unsigned char *client,
                         unsigned int client_len);

It takes two protocol lists: one from the server, one from the client, each in a special wire format (see below).

When called correctly (with real lists), it works as designed. When called with a client protocol length of zero, it misbehaves in a dangerous way.

Let's look at a simplified (not actual OpenSSL code) example of how things can go wrong

unsigned char *selected = NULL;
unsigned char selected_len = ;

// Bug: Zero-length client protocol list passed (shouldn't happen)
const unsigned char *server_proto = (const unsigned char *)"\x08http/1.1";
unsigned int server_proto_len = 9; // "\x08http/1.1" - server supports HTTP/1.1

const unsigned char *client_proto = NULL;
unsigned int client_proto_len = ; // Oops! Empty

int proto_status = SSL_select_next_proto(
    &selected,
    &selected_len,
    server_proto, server_proto_len,
    client_proto, client_proto_len
);

// At this point, selected points to arbitrary memory!
printf("Selected protocol: %.*s\n", selected_len, selected); 
// Can crash or leak memory

When a zero-length client_proto is passed

- SSL_select_next_proto() might copy or return whatever is at the memory location immediately after client_proto — who knows what it is? It could be secrets, passwords, keys, or just garbage.

The Details — Wire Format and Why This Happens

OpenSSL stores protocol lists in a specific format for ALPN or NPN (see RFC 7301):

[Len1][Proto1][Len2][Proto2]...

Normal code expects at least one [Len][Proto] pair.

But if you pass in client_proto as zero length, SSL_select_next_proto() tries to read the first byte (length) at the pointer — possibly from past the end of the valid buffer. Instead of a protocol name, it’s reading stray memory.

You are vulnerable only if all these are true

1. Your app calls SSL_select_next_proto() directly (not common; most use high-level OpenSSL APIs).

You don’t notice the "no match" condition and use the returned "protocol" value.

Most apps using ALPN are safe, because libssl validates the protocol length. But custom or old NPN-based code (or libraries with bad wrappers) are at risk.

“Can This Be Actively Exploited?”

Almost no — the attacker generally can’t force your server or client to use an empty list, unless they have control over your config or code (insider, or severe bug). But a mistake by a developer or admin is enough.

Data Corruption: Application logic may be disrupted.

If that memory is sent over the network, and the peer is malicious, confidentiality is breached.

Do not use this in production. Here’s what a naive PoC might look like

#include <stdio.h>
#include <openssl/ssl.h>

int main() {
    SSL_library_init();
    unsigned char *out = NULL;
    unsigned char outlen = ;
    unsigned char server[] = "\x08http/1.1";
    unsigned int server_len = 9;
    const unsigned char *client = NULL;
    unsigned int client_len = ;

    int status = SSL_select_next_proto(&out, &outlen, server, server_len, client, client_len);
    printf("Status: %d\n", status);

    if (outlen >  && out) {
        printf("Leaked memory: ");
        for (int i = ; i < outlen; i++)
            printf("%02x ", out[i]);
        printf("\n");
    }
    return ;
}

References & Further Reading

- Official OpenSSL Advisory: CVE-2024-5535
- OpenSSL NPN API: NPN documentation
- ALPN RFC: RFC 7301
- ALPN in OpenSSL: OpenSSL ALPN docs

Bottom Line

CVE-2024-5535 was born of a rare but dangerous oversight: failing to validate that your protocol selection lists aren’t empty. While it’s tough to exploit in practice, it stands as another reminder — when handling memory in C, *even your “should never happen” error paths must be guarded*.

If you build on OpenSSL and still use NPN (or custom protocol selection), audit your code now to prevent accidental exposure, and prepare for an upstream patch soon. Stay safe!


*Written exclusively for you, by AI, for the 2024 security era.*

Timeline

Published on: 06/27/2024 11:15:24 UTC
Last modified on: 07/03/2024 02:09:04 UTC