A subtle but potentially dangerous flaw has been found in OpenSSL’s implementation of the POLY1305 message authentication code (MAC) when running on Windows 64-bit systems with new x86_64 CPUs that support AVX512-IFMA instructions. Known as CVE-2023-4807, this bug can corrupt application state and may crash processes or cause unexpected results. In rare cases, it could even open doors to more serious attacks.

Here, we’ll break down what the vulnerability is, what it means for your servers, how an exploit might look, and how to protect your systems — all in plain, simple language.

What’s the Issue? (A Simple Explanation)

When OpenSSL performs a POLY1305 MAC calculation on large data blocks (over 64 bytes), the code should "save and restore" specific CPU registers (XMM registers). These are special storage areas in your CPU that store certain values used for processing — kind of like temporary notepads. On some modern Intel/AMD CPUs (with AVX512-IFMA feature), the OpenSSL code mistakenly zeroes out these registers instead of restoring them, only on Windows 64-bit systems.

If an app relies on the contents of these XMM registers after the OpenSSL function returns (which some apps do!), the app might get incorrect computation, or even crash.

Key points

- This is *not* a remote code execution bug, but it can corrupt memory/state.
- Attackers could potentially trigger this by getting the server to use the CHACHA20-POLY1305 AEAD cipher (for example, with TLS 1.2 or 1.3 sessions).

Technical Details

The problematic code gets triggered on Windows 64-bit, on CPUs with AVX512-IFMA, when OpenSSL processes large data for POLY1305 MAC.

Here’s a simplified version of what happens about the XMM registers

// This is not real OpenSSL code, but demonstrates the bug:

void poly1305_mac(const unsigned char *msg, size_t len) {
    // Should save XMM6-XMM15 here
    if (len > 64) {
        // Perform fast AVX512-IFMA operations
        // ...
        // On Windows-64: Instead of restoring, sets XMM6-XMM15 to zero!
    }
    // Should restore XMM6-XMM15 here, but doesn't
}

On Windows 64-bit, XMM6-XMM15 are "non-volatile", meaning the calling program expects those values to come back unchanged. If they get zeroed, any code running after the OpenSSL call that relies on those values could crash or behave unpredictably.

So, if an attacker can make your server use POLY1305 MAC (with CHACHA20-POLY1305), they might be able to induce memory corruption, typically resulting in a crash or application malfunction.

Exploit Scenario

Let’s imagine a typical web server using OpenSSL for TLS, where the attacker controls which cipher suites the connection will use. If the attacker connects and negotiates for CHACHA20-POLY1305 as the cipher, they ensure the server uses POLY1305 MAC.

If their session sends a message big enough (over 64 bytes), and if the server’s code path after OpenSSL cares about XMM registers (not common, but possible), the application state could get corrupted or it could crash.

Here’s a sample exploit in theory

1. Find a target server: Must be Windows 64-bit, x86_64, with an AVX512-IFMA capable CPU, and OpenSSL compiled to use it.

Observe if the server crashes, misbehaves, or leaks data due to register zeroing.

_Because the attacker can’t control what goes into the registers—only that they become zero—the most probable outcome is a denial of service (DoS), not remote code execution._

Proof-of-Concept (POC) Example

For demonstration, here is a sample POC using OpenSSL’s s_client to force the CHACHA20-POLY1305 cipher:

openssl s_client -connect victimserver:443 -cipher "TLS_CHACHA20_POLY1305_SHA256"

Send a long HTTP request, say over 64 bytes in the body, to fire the buggy code path. You *might* notice the server misbehaves or crashes if it's vulnerable.

Note: There is no publicly known way to escalate from DoS to full code execution here, but that might change if creative abuses are found.


## Workaround / Mitigation

To completely avoid the problem without upgrading OpenSSL, you can disable AVX512-IFMA by setting an environment variable on your server:

OPENSSL_ia32cap=:~x200000

This tells OpenSSL not to use AVX512-IFMA instructions, so the buggy code isn’t called at all.

- If you manage Windows 64-bit [IIS, Apache, nginx, etc.] with OpenSSL, add this environment variable to your service configuration or user session.
- If possible, upgrade to a patched OpenSSL version (see OpenSSL security advisory).

The FIPS provider is not affected, and Linux or non-AVX512-IFMA systems are safe.

References

- OpenSSL Advisory: CVE-2023-4807
- National Vulnerability Database Entry
- POLY1305 MAC Algo (RFC 8439)
- CHACHA20-POLY1305 Explained (Cloudflare blog)

Summary (TL;DR)

- CVE-2023-4807 is a bug in OpenSSL’s POLY1305 MAC implementation, in Windows 64-bit OpenSSL builds, with CPUs supporting AVX512-IFMA.
- Attackers can *maybe* crash your server by forcing it to use CHACHA20-POLY1305 and sending large data.

No known RCE possibilities (yet), but DoS is possible depending on your app.

Keep your OpenSSL up-to-date and add the workaround if you run OpenSSL on Windows servers with new CPUs!


*Written for sysadmins, IT pros, and developers who don’t want surprises from their crypto code. Exclusively crafted in simple language for easy understanding.*

Timeline

Published on: 09/08/2023 12:15:00 UTC
Last modified on: 09/14/2023 16:43:00 UTC