CVE-2024-42294 - Linux Kernel Deadlock Fixed Between `sd_remove` & `sd_release`

CVE-2024-42294 identifies a major deadlock in the Linux kernel block subsystem. The flaw could cause filesystem operations, such as formatting or writing, to hang indefinitely when SCSI devices are removed. The root cause was an ABBA (Acquire-Before-B, Acquire-After-A) deadlock between the sd_remove and sd_release code paths. This vulnerability affected systems using SCSI disks, including servers and embedded devices using USB storage.

This article breaks down the vulnerability, explains the technical details, shows relevant logs and code, and describes how the kernel development team fixed the issue.

Background

The Linux block layer manages storage devices (like HDDs/SSDs/USB drives). SCSI disks use specialized handlers in the kernel (sd.c). When a device is removed (physically disconnected or via software), two kernel threads may attempt to clean up resources at the same time:

- sd_remove — Handles device unplug/removal.
- sd_release — Handles the last file handle (/dev/sdX) being closed.

This race triggers a kernel deadlock. Below is a real test-system log demonstrating the issue.

Kernel Log Example

[ 2538.459400] INFO: task "kworker/:":7 blocked for more than 188 seconds.
[ ... ]
[ 2538.459462]  del_gendisk+xdc/x350
[ 2538.459466]  sd_remove+x30/x60
[ ... ]
[ 2538.461001] INFO: task "fsck.":15415 blocked for more than 188 seconds.
[ ... ]
[ 2538.461054]  sd_release+x50/x94
[ 2538.461058]  blkdev_put+x190/x28c

Let’s visualize the sequence (simplified)

| Task 1 (sd_remove) | Task 2 (sd_release) |
|-------------------------|-------------------------|
| sd_remove | bdev_release |
| del_gendisk | mutex_lock(&disk->open_mutex)|
| __blk_mark_disk_dead | sd_release |
| blk_freeze_queue_start | scsi_execute_cmd |
| ++q->mq_freeze_depth | blk_queue_enter (waits for freeze_depth=!) |
| (holds queue lock) | (wants queue, stuck) |
| mutex_lock(&disk->open_mutex) | |

Key Point:

After the queue is frozen, sd_remove tries to lock disk->open_mutex, while sd_release (which already holds disk->open_mutex) is stuck waiting for the frozen queue, causing a deadlock.

Root Cause: Missing Flag and Lock Order

SCSI did not set GD_OWNS_QUEUE, so QUEUE_FLAG_DYING wasn't set. The queue could be frozen while there were still active users, leading to deadlock.

T2: lock open_mutex → try to lock queue (B then A)

With tasks holding one lock and waiting for the other, the system hangs.

Code Snippet (Before Fix)

Here’s a simplified excerpt from the vulnerable kernel code paths (sd_remove and sd_release).

// Called during device unplug
static int sd_remove(struct device *dev)
{
    // ...
    del_gendisk(disk);  // Triggers disk shutdown
    // ...
}

// Called during file close(/dev/sdX)
static int sd_release(struct gendisk *disk, fmode_t mode)
{
    mutex_lock(&disk->open_mutex); // Acquires open_mutex
    // ...
    blk_queue_enter(queue); // May block if queue is frozen
    // ...
    mutex_unlock(&disk->open_mutex);
}

The Fix

Solution:
*Avoid acquiring disk->open_mutex after queue freeze.*
Specifically, the locking/unlocking order in sd_remove was revised so that it cannot deadlock with sd_release.

Commit Link:
- block: fix deadlock between sd_remove & sd_release (kernel.org)

Relevant Patch Diff

 static void sd_remove(struct device *dev)
 {
-    del_gendisk(disk); // Old order: freeze queue, then lock
+    device_remove_disk(dev, disk); // New order: disables device earlier
     // ... rest of teardown ...
 }

Now, the device is removed *before* the queue freeze happens, ensuring no file handles can be opening/closing, breaking the ABBA deadlock cycle.

Exploitability

- Risk: Attackers with local access (or via crafted hardware/untrusted USB/SCSI) could intentionally trigger the deadlock, causing a local denial of service (DoS) — e.g., system hang, loss of availability, stuck processes.
- Impact: All Linux systems using SCSI storage (including USB sticks, external hard drives, VMs with hot-removal SCSI) were at risk.

Further References

- CVE-2024-42294 entry at MITRE (reserved)
- Upstream kernel git commit log
- Linux Kernel Mailing List — Patch Discussion

Summary

CVE-2024-42294 highlights how subtle lock-order bugs in core infrastructure can cause catastrophic impacts across millions of machines. The collaborative work of kernel developers and testers ensures that issues like these are found and fixed quickly.

Advice:
If you use Linux, especially with SCSI or USB storage, make sure you are running a kernel version including the above commit — or check with your vendor for a patched release.

Timeline

Published on: 08/17/2024 09:15:09 UTC
Last modified on: 08/19/2024 19:43:22 UTC