CVE-2023-5363 - Truncation and Overruns in Key/IV Handling in OpenSSL Symmetric Ciphers

*Published: Exclusive, Simple American Language, with Examples and Exploitation Details*

Introduction

CVE-2023-5363 is a recently discovered vulnerability in OpenSSL (versions 3. and 3.1) that affects how key and initialization vector (IV) lengths are processed for certain symmetric ciphers. This bug can potentially lead to the truncation or overrun of these parameters, especially when using new parameter APIs. In simple terms, this mistake could break the security basics of encryption by causing the IV to repeat or be too short, threatening the secrecy of your encrypted data.

This long read will help you understand what went wrong, how to spot if you’re affected, and, for the curious, show an example of how it can be misused.

Which API?: EVP_EncryptInit_ex2(), EVP_DecryptInit_ex2(), EVP_CipherInit_ex2()

- Which Ciphers/Modes?: RC2, RC4, RC5, CCM, GCM, OCB (especially GCM, CCM, OCB)
- What Goes Wrong?: When you set the key and IV lengths using the parameter array (OSSL_PARAM), OpenSSL _does not_ reliably process changes to key or IV length afterward. That is, if you want to change the key or IV size by passing values in OSSL_PARAM, it silently ignores your changes. This can result in:

- Truncated IV

- If the IV is accidentally shortened, it's more likely to repeat, violating a critical law of encryption—IVs must be unique, especially for modes like GCM.

- Key/IV Overruns

- Reading more than is supplied could cause a crash or unexpected behavior, but these are less likely to be exploitable directly.

> Most applications should not be affected, unless they explicitly set key or IV lengths in a non-default way, but if you are, it's bad.

Demonstration: Example Code Showing the Bug

Let’s see a short example using OpenSSL to explain this. This shows “what you’d intend,” and “what actually happens” with the vulnerable API.

#include <openssl/evp.h>
#include <stdio.h>
#include <string.h>

// Dummy keys for demonstration
unsigned char key[32] = {};  // 256-bit key
unsigned char iv[16]  = {};  // 128-bit IV

int main() {
    EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();
    int outlen;

    // You plan to use only 128 bits of IV, set via OSSL_PARAM
    OSSL_PARAM params[] = {
        OSSL_PARAM_construct_size_t("ivlen", (size_t[]){12}),
        OSSL_PARAM_END
    };

    // "ivlen" param is actually ignored! Key and IV lengths are set before this is read.
    EVP_EncryptInit_ex2(ctx, EVP_aes_256_gcm(), key, iv, params);

    // In reality, OpenSSL will use the length of the IV you provided (16), not 12.
    // This can cause mismatches, if the decryption side uses a different length.

    // ... (rest of encryption)
    EVP_CIPHER_CTX_free(ctx);
    return ;
}

Result:
The function ignores your attempt to set the IV/key size using the parameter array. It uses the default lengths instead!

Potential Exploit Scenario

Imagine a system that follows NIST GCM guidance and constructs a deterministic IV, as best practice demands. If the IV you intend is 12 bytes, but it becomes truncated to less (say, only 8 or 10 bytes get used), you could accidentally repeat IVs for different encryptions!

IV reuse in GCM/OCB/CCM means confidentiality is lost, and with enough message analysis an attacker might decrypt data or, worse, forge encrypted messages.

Example Impact:

If two messages are sent with the same truncated IV, all the classic GCM attacks (like those described here) become possible.

Who’s Affected?

- OpenSSL 3. and 3.1 only. (Earlier versions and most common usages are not affected. OpenSSL's FIPS module is also safe.)
- Only if you use the new _ex2() API and you try to adjust key/IV sizes using OSSL_PARAM in API call.

If you use standard EVP APIs, or if you’re just using TLS (for HTTPS, etc), you’re probably safe.

References (Original Details)

- Official OpenSSL Security Advisory for CVE-2023-5363
- OpenSSL Github Commit Fix for CVE-2023-5363
- NIST GCM Guidance: SP 800-38D, Section 8.2.1

So far, there are no reports of real-world exploits. The main reasons

- You need a rare and unusual app setup (explicit tweaking of IV/key sizes).

Most web and TLS apps don’t use these APIs this way.

But:

For key management or cryptography libraries that give users lots of control, mistakes are possible.

- Test or check any application that calls the new _ex2() functions _and_ passes OSSL_PARAM for IV/key length.

Best Solution: Always upgrade OpenSSL if you’re on 3..x or 3.1.x.

Simple Advice

- Don’t try to change key/IV sizes using parameters unless you are sure your OpenSSL is patched.

Upgrade OpenSSL if you build or maintain security-sensitive applications.

- If using custom cryptography, run thorough tests with edge case key/IV sizes.

Conclusion

CVE-2023-5363 is a perfect example of how small mistakes with cryptography parameters can open the door to much bigger security disasters. It mostly threatens custom cryptography setups—not standard TLS/SSL use.

Stay safe: patch quickly, don’t tweak what you don’t need, and verify your cryptography code!


Exclusive for this post. For more technical content, follow the official OpenSSL Announcements.

Timeline

Published on: 10/25/2023 18:17:43 UTC
Last modified on: 11/09/2023 13:55:31 UTC