Week 10: getting VA-API to produce frames and rethinking the contract

Getting VA-API to produce frames

This week I built the VA-API interop. The work is close to what we already did for NVDEC, which is the whole idea: both are meant to plug into the same layer and hand decoded frames to the renderer without copying them through system memory. I started from the module on my mentor's vulkan-vout branch and edited it until it fit this tree.

One edit was about how the decoder describes a frame. The copied module built one multi-planar Vulkan image for the whole frame, and that does not work everywhere, since some drivers only offer the plain linear layout for those formats and a decoder never produces linear output. Each exported layer now gets its own single-plane image.

The second edit was synchronization, and that is where things started being a design question.

Synchronization: the wrong kind of semaphore

NVDEC needed a timeline semaphore to order its work, so a timeline is what the layer shares between modules. VA-API wants something else. A dma-buf comes with a sync file, which is a fence meaning the decoder has finished, and handing that to Vulkan lets the graphics card wait instead of the processor. That is the better arrangement, and it is what the vulkan-vout module does.

Vulkan does not allow it here. A sync file has to be imported temporarily, and a temporary import cannot target a timeline semaphore. So the one synchronization object the layer offers is the one object that cannot accept what VA-API has.

For now the module waits on the processor instead. That is correct, and it measures as free on the files I can test, but it gives up something the hardware supports and I do not want to leave it there.

The cache: memory that moves

The video was correct, every frame that appeared looked right, and it ran at something like two frames per second, visibly repeating frames.

Back in week 5 I gave the display a cache: wrap each image into a libplacebo texture once and reuse that wrapper, because rewrapping every frame is undefined behaviour. I wrote at the time that the images do not change for the life of the output. That was true, and it was true only because NVDEC copies each frame into images it owns. VA-API imports the decoder's own memory, and the decoder keeps moving to a different surface. That is what I mentioned last week.

The temporary fix is to rebuild the wrapper whenever the module hands back a different image. It works, and it is a patch rather than a design. The core is inferring what the module meant by comparing handles every frame.

What I want to change, and what happens next

For synchronization there are two reasonable options. The vulkan-vout branch treats its semaphore as a timeline only when the module reports a non-zero value and as binary otherwise, so one object covers both. The other, which I prefer, is a field in the probe phase where a module says what kind of synchronization it needs.

For the cache I am leaning towards moving it into the module, so an interop hands back a finished texture rather than a raw Vulkan image. The module already knows the lifetime of its own memory, and the software interop already works that way. I am not certain, since it changes NVDEC as much as VA-API.

Both are contract changes rather than patches, and both affect a module that already works, so I would rather agree them with Thomas. That is the the work I expect to be doing next week.

In the meantime VA-API is producing output. H.264 works, seeking works, and there are no validation warnings and no dropped frames.