A vulnerability has been discovered and resolved in the Linux kernel, and it concerns the crypto: RSA implementation. The issue arises due to a lack of check for allocation failure, which could potentially lead to a NULL dereference. In this post, we will discuss the details of this vulnerability, how it affects the Linux kernel, and the proposed patch that resolves the issue.

Details

The vulnerability lies in the crypto: RSA implementation of the Linux kernel. Static checkers have pointed out that the mpi_alloc() allocation can fail, which could lead to a NULL dereference. Although small allocations like this are unlikely to fail in current kernels, it's a good practice to have a check in place to ensure that it doesn't cause problems.

The following code snippet showcases the problem in the original code

// Allocate memory for the result of the multiplication operation
mpi_ptr_t *res_ptr = mpi_alloc(mpi_get_nlimbs(u) + 1);

// Now perform the multiplication operation with the newly allocated memory
mpi_mul(res_ptr, u, v);

In this code snippet, the allocation function mpi_alloc() may fail to allocate memory and return a NULL pointer. However, this return value is not checked, and the code directly proceeds to perform the multiplication operation, which can lead to a NULL dereference.

The proposed patch to resolve the issue is quite simple and ensures that the static checkers will not flag this issue.

Here's the new code snippet with the added check for allocation failure

// Allocate memory for the result of the multiplication operation
mpi_ptr_t *res_ptr = mpi_alloc(mpi_get_nlimbs(u) + 1);

// Check for allocation failure
if (!res_ptr)
    return -ENOMEM;

// Now perform the multiplication operation with the newly allocated memory
mpi_mul(res_ptr, u, v);

By adding the check for allocation failure using if (!res_ptr), we can ensure that the code doesn't proceed with the NULL dereference if the allocation fails.

Original References

You can find the original discussion and the proposed patch in the Linux kernel mailing list archives:

[PATCH] crypto: rsa - add a check for allocation failure

https://lkml.org/lkml/2022/7/16/229

Conclusion

The discovery of this vulnerability in the Linux kernel's crypto: RSA implementation highlights the importance of constantly analyzing the codebase for potential issues. Even though the probability of small allocations like this failing in the current kernels is low, adding the check ensures robustness and makes the static checkers happy.

Developers should always be vigilant and implement necessary checks to prevent null dereferences and other common programming errors, thereby ensuring the overall security and stability of the software.

Timeline

Published on: 02/26/2024 16:27:48 UTC
Last modified on: 04/17/2024 18:47:08 UTC