Week 1

Part 1: the most boring patch I'll write all summer

VLC's libplacebo output had one function, vlc_placebo_Create(), that sets up the whole GPU in a single shot. The interop work needs to get in the middle of that: Vulkan extensions must be requested before the device is created (you can't add them later), and the interop code needs the raw Vulkan handles afterwards. So my first MR splits it, and changes nothing else:

Before:  vlc_placebo_Create()              // everything at once

After:   vlc_placebo_New()                  // allocate
         vlc_placebo_Connect(params)        // create, with room to inject extensions
         vlc_placebo_GetVkHandles()         // raw handles for interop code

VLC tries Vulkan first and quietly falls back to OpenGL if it fails, and that fallback only survives if creation stays one atomic step. That's why it's a single Connect() taking extension lists, not separate create-instance and create-device calls. A patch that does nothing by design is also the easiest kind to review, which is exactly what my mentor asked for.

Part 2: the assumption that didn't survive contact with the code

VLC has an experimental Vulkan output (my mentor wrote it) that already solves hardware import, and my job is to move that logic over to libplacebo. I assumed it imported each frame as separate Vulkan images, one per color plane, since that's the shape libplacebo likes. Reading the actual code: no. It imports one multi-planar VkImage per frame and uses a driver feature, VkSamplerYcbcrConversion, to turn the brightness and color planes into RGB at sample time.

The conflict: libplacebo owns its pipelines and samplers internally, and there's no way to hand it an externally created YCbCr sampler. So the import code can't move over as-is. Frames have to be exposed plane by plane (Vulkan can address each plane of a multi-planar image through an "aspect"), with libplacebo's shaders doing the color math. A real restructuring, not the copy-paste port in my notes.

Part 3: asking the person who wrote it

I mailed my mentor, who is also the author of that code. He used the YCbCr sampler simply because he didn't want to write conversion shaders by hand, and he pointed out it's buggy on some implementations anyway. So, this week's decisions: libplacebo's shaders do the color conversion with frames wrapped per plane, and NVDEC goes first, because CUDA shares memory with Vulkan without the extra layout headaches that Intel and AMD's path brings. Prove the idea where it's simplest as first step.

The code is on https://code.videolan.org/ahmedsobhy/vlc/-/merge_requests/1