CVE-2024-12797 - How a Raw Public Key TLS Authentication Bug in OpenSSL Can Let MITM Attacks Slip Through

---

Introduction

In March 2024, a significant vulnerability—CVE-2024-12797—was disclosed in OpenSSL affecting some clients using RFC725 Raw Public Keys (RPK) for TLS or DTLS server authentication. This issue may allow attackers to perform Man-In-The-Middle (MITM) attacks against certain OpenSSL clients without being detected, if they use raw public keys.

Below, we’ll break down what’s vulnerable, see why the bug slipped through, look at a code snippet, discuss exploit scenarios, and explain how to tell if you’re exposed.

What’s the Issue? (In Simple Terms)

If you use OpenSSL to connect to a server, you normally check that the server identifies itself with something you trust, most often a major certificate (X.509 certificate chain). But OpenSSL also allows using “raw public keys” as a simpler alternative, according to RFC725.

The idea:
- Both ends (client and server) configure RPKs they trust, instead of relying on heavyweight certificates.

THE PROBLEM: If a client connects to a server _using RPKs_ and turns on peer verification with SSL_VERIFY_PEER, the TLS handshake is supposed to abort immediately if the RPK from the server is unknown or invalid. But due to this bug, the handshake does not fail—not automatically.

- Impact: The client assumes the handshake succeeded even if the server sent a public key that isn’t trusted.
- Result: MITM attacks are easier – the attacker presents their own key, and the client may accept it!

You enable RPKs (SSL_CTX_set1_raw_public_key()) on both sides

- You rely on the handshake to “just fail” on server authentication failure, trusting SSL_VERIFY_PEER

How Does the Exploit Work?

Pretend you wrote a program that uses OpenSSL and lets a user enable RPKs. You set SSL_VERIFY_PEER because you want handshake to fail if the key is wrong.

Due to CVE-2024-12797, even if the server shows the wrong or unknown public key, the handshake doesn't abort! Unless you call SSL_get_verify_result() and check the result, your code might just keep going as if everything is OK.

Proof of Concept: Minimal Example

Let’s make a toy client using RPKs (stripping out all error handling for clarity).

#include <openssl/ssl.h>

int main() {
    SSL_CTX *ctx = SSL_CTX_new(TLS_client_method());
    // Enables Raw Public Key support
    SSL_CTX_set1_raw_public_key(ctx, ...); // set expected RPKs here 

    SSL *ssl = SSL_new(ctx);
    SSL_set_verify(ssl, SSL_VERIFY_PEER, NULL);  // "Trust but verify"

    // connect to server, setup BIO, etc...

    if (SSL_connect(ssl) == 1) {
        // At this point, you'd EXPECT handshake to abort if RPK doesn't match
        // Due to bug, handshake succeeds even if the wrong RPK is sent!
        long verify = SSL_get_verify_result(ssl);
        if (verify != X509_V_OK) {
            // Only here do you catch the failure.
            printf("RPK verification failed!\n");
            // Handle appropriately.
        } else {
            // Connection is authenticated!
        }
    }
}

Key Point:
- If your application doesn't explicitly call SSL_get_verify_result() after the handshake, it won’t notice RPK validation failed!

Fixes & Safe Coding Practices

- Update OpenSSL when patches are released (OpenSSL advisory).
- If you must use RPKs, always call SSL_get_verify_result() _after_ every handshake and _enforce failure_ if result != X509_V_OK.
- Double-check your library: Any code path using RPK authenticating the server must check the result explicitly.

References

- OpenSSL Security Advisory Mar 2024 (CVE-2024-12797)
- RFC725: Using Raw Public Keys in TLS/DTLS
- Original OpenSSL RPK commit discussion
- NVD Entry for CVE-2024-12797

In Summary

CVE-2024-12797 is a classic example of how “secure by default” can be undermined by overlooked edge cases. If you use OpenSSL’s RPK support, enforce server key verification in your code manually. Don’t just rely on handshake failure. Update your OpenSSL version as patches land.

Stay safe and double-check your authentication!

*If you want more in-depth advice or sample code for a particular use case, leave a comment or reach out to your project’s security team.*

Timeline

Published on: 02/11/2025 16:15:38 UTC
Last modified on: 02/18/2025 14:15:27 UTC