In the world of video processing, Libde265 is a widely used open-source library for decoding H.265/HEVC video streams. But in October 2022, a serious bug—CVE-2022-43236—was revealed: a stack buffer overflow vulnerability in version 1..8. This flaw, lurking in the put_qpel_fallback<unsigned short> function (from fallback-motion.cc), can let an attacker crash any software using Libde265 by playing a specially crafted video file.

This post walks you through what happened, how the bug works (with example code), how attackers could exploit it, and references for further reading.

Background: What is Libde265?

Libde265 is a C++ library designed for decoding H.265/HEVC video streams. It’s used in various media players, web servers, and editing tools. Since it’s open-source, many projects rely on it for reliable and fast video playback.

The problem lies in the function template

template <typename pixel_t>
static void put_qpel_fallback(pixel_t* dst, long dststride,
                              const pixel_t* src, long srcstride,
                              int width, int height, int dx, int dy);

Specifically, the version instantiated as put_qpel_fallback<unsigned short>() in fallback-motion.cc.

The function is responsible for motion compensation in the video decoding process. Mistakes here can have serious consequences for memory handling.

What’s a stack buffer overflow?

A stack buffer overflow happens when a program writes more data to a buffer (array) located on the stack than what’s allocated. This can overwrite other important data—sometimes even control flow info—on the stack.

In this case, an attacker can craft a video file so that when Libde265 tries to process it, the decoder writes past the end of a stack-allocated array. This can make the program crash (Denial of Service/DoS). In some edge cases, it could even be abused for code execution, depending on how the stack is set up and what gets overwritten.

Code Walkthrough

Let’s look at the relevant part of the code. In fallback-motion.cc, you’ll find something like this (simplified for clarity):

// Simplified function: fallback-motion.cc
template <typename pixel_t>
static void put_qpel_fallback(pixel_t* dst, long dststride,
                              const pixel_t* src, long srcstride,
                              int width, int height, int dx, int dy)
{
    pixel_t temp[24 * 24]; // temp buffer on stack for processing

    // ...the code then fills 'temp' based on input width and height
    for (int y = ; y < height + 3; y++) {
        for (int x = ; x < width + 3; x++) {
            temp[y * (width+3) + x] = src[y * srcstride + x];
        }
    }
    // Processing steps
}

The buffer temp[24*24] is sized for a maximum width and height of 21, but the code does not properly limit the user's width and height parameters. So, if a malicious input file delivers bigger values, the loops can write out-of-bounds—leading to a stack buffer overflow.

Attack Scenario

An attacker creates a video file with certain frame dimensions and motion parameters, tricking Libde265 into calling this function with large width/height values, like 50x50. The code above would then attempt to write past the temp buffer, corrupting the stack and (at the very least) crashing the decoder.

Exploit Example

Suppose you have a video file that, when loaded by a vulnerable media player (using Libde265 v1..8), specifies enormous frame sizes or crafted motion vectors.

Building a Proof-of-Concept

While a real PoC would require deep understanding of the HEVC bitstream, here’s a pseudocode simulation that shows how triggering unexpected width/height can crash the software:

// Assume this is called with crafted parameters:
put_qpel_fallback<unsigned short>(
    dst, dststride, src, srcstride,
    50, 50, // width, height - much larger than 21!
    dx, dy
);
// Buffer temp[24*24] only has space for 576 elements
// But loop writes up to 53*53=2809 elements, overrunning stack!

You could fuzz the decoder with a tool like AFL++ to generate an input file that triggers the crash:

afl-fuzz -i testcases/ -o findings/ -- libde265_decoder -i @@

After running for a while, you'll likely get a crash looking like

* stack smashing detected *
Aborted (core dumped)

Exploit Impact

This is a Denial of Service (DoS) vulnerability. By supplying a malicious HEVC file, an attacker can:

Potentially escalate to code execution in rare cases, depending on system protections

If your project uses Libde265 to handle user-uploaded or user-provided video files, you’re at risk.

They avoid exceeding buffer bounds in this function

If you use Libde265, upgrade to at least v1..9.

References & Further Reading

- Official Libde265 Security Advisory
- CVE-2022-43236 at NVD
- OSS-Fuzz bug report #52709
- Explaining Stack Buffer Overflows

Update Libde265: Make sure you’re using at least v1..9.

2. Test your video stack: If you allow users to upload or process video files, fuzz your software for similar bugs.
3. Enable stack protection: Use compiler options like -fstack-protector-strong and modern OS mitigations.

Stay safe! Never trust user input, and keep your dependencies updated.

Timeline

Published on: 11/02/2022 14:15:00 UTC
Last modified on: 02/27/2023 15:18:00 UTC