Linux powers much of the internet. Modern storage speeds depend on protocols like NVMe over TCP (NVMe-oF/TCP), allowing fast access to SSDs over networks. But sometimes, a small bug can turn into a real problem. That’s exactly what happened with CVE-2023-6535, a flaw in the Linux kernel’s NVMe driver. This post explains what's wrong, how it works, why it matters, and even includes example code for researchers.

What is CVE-2023-6535?

CVE-2023-6535 is a security vulnerability found in the Linux kernel’s NVMe (Non-Volatile Memory Express) over TCP driver. The bug allows an attacker on the network to send malicious (crafted) TCP packets to machines using NVMe over TCP. If they do, these packets can trigger a NULL pointer dereference in the kernel’s NVMe code. When that happens, the kernel crashes (kernel panic), leading to a full denial of service (DoS) for that system.

In short: Bad packets from a remote attacker can instantly bring down a Linux server using NVMe-oF over TCP—with no authentication needed.

The NVMe over TCP feature lets a server access NVMe SSDs using a network.

- There’s a bug in the kernel's nvme-tcp driver (found in files like nvme-tcp.c in /drivers/nvme/host/).
- If an attacker sends a specific, malformed TCP packet to the service, the driver receives it and tries to handle it.
- But the received data is sometimes not checked properly. The driver may use a pointer without checking if it’s actually set (not NULL).

The system then panics and reboots or hangs — causing full denial of service.

No authentication is necessary. Anyone who can reach the NVMe/TCP service can do this.

Technical Details

You can view the upstream patch and discussion on lkml.org.

The bug specifically affects the NVMe/TCP transport protocol in the kernel. In the code, the sequence of processing incoming packets fails to properly validate a structure before dereferencing one of its members.

The simplified vulnerable code pattern looks like this (example, not the actual code)

// In drivers/nvme/host/tcp.c
void nvme_tcp_handle_packet(struct nvme_tcp_queue *queue, struct sk_buff *skb)
{
    struct nvme_tcp_hdr *hdr;
    hdr = (struct nvme_tcp_hdr *)skb->data;

    // Vulnerable: Not checking if queue->xprt is NULL before dereference
    if (queue->xprt->connected) {
        // Work with the transport
    }
}

If queue->xprt is NULL because of crafted input, the kernel tries to access a NULL pointer and crashes.

The patch adds a check to make sure the pointer isn’t NULL before dereferencing it:

if (queue->xprt && queue->xprt->connected) {
    // Safe now
}

The driver processes the broken packet, hits the NULL pointer, and crashes the kernel.

Here’s a proof-of-concept (PoC) in Python that demonstrates a simple connection to an NVMe-oF service (you would need to craft a malicious packet for a real test):

import socket

target_ip = 'TARGET_IP'
target_port = 442

# You would need to build the specific malformed NVMe/TCP packet as per the bug
malicious_packet = b'\x00' * 64  # Placeholder

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.connect((target_ip, target_port))
    s.sendall(malicious_packet)

The actual exploit would craft the packet based on the protocol details to reach the bad code path. The kernel org security advisory and Red Hat advisory provide more context.

Update your kernel!

- Distros like RHEL, Ubuntu, Debian, and SUSE have pushed patched kernels (Red Hat security page).
- If you compile your own kernel, apply the upstream patch.
- Firewall NVMe/TCP ports (default 442) from untrusted networks.

- CVE-2023-6535 at NVD
- Patch upstream in Linux kernel
- Red Hat CVE-2023-6535 page
- LKML Discussion
- Exploit DB page (if/when available)

Bottom Line

CVE-2023-6535 is a reminder that even core parts of Linux have bugs that can have big impacts. A single malformed packet from anywhere on the network can crash critical infrastructure if it uses NVMe-oF over TCP. Always keep critical systems updated, keep services shielded from public networks, and follow security advisories.

If you run NVMe-oF with Linux, update now and stay safe!

Timeline

Published on: 02/07/2024 21:15:08 UTC
Last modified on: 03/12/2024 03:15:06 UTC