opengl: render YUY2 at full definition
Some YUV 4:2:2 formats are packed: two consecutive pixels are stored
using 4 values (for example [Y1 U Y2 V]
).
But in OpenGL we cannot have both:
- a single texture, and
- a correct native interpolation for both Y and UV components.
To avoid the problem, the current implementation just dropped the Y2 value, so the pictures were rendered at half the horizontal resolution.
To render at full definition, upload the single plane into two separate OpenGL textures:
- one in
GL_RG
, to read each Y value in a different texel; - one in
GL_RGBA
, to read both U and V values in a single texel.
As a consequence, pic->i_planes
is not necessarily equal to
interop->tex_count
anymore (there might be 1 picture plane but 2
textures).
Fixes #26712 (closed)
From #26712 (closed):
Alternatively, we could upload the same picture twice (
😱 ), once inGL_RG
to access the Y components, once inGL_RGBA
to access the UV components, and keep the native OpenGL interpolation.
I choose this solution because it is the most straightforward to implement (with a limited impact on source code and architecture) and it provides correct rendering of YUV 4:2:2 pictures "easily".
More details from this comment:
Here is the storage of the original plane:
Y0 U0 Y1 V0 Y2 U2 Y3 V2
Y4 U4 Y5 V4 Y6 U6 Y7 V6
When uploading to a GL_RG
texture, the texels are split as follow:
+-----+-----+-----+-----+
|Y0 U0|Y1 V0|Y2 U2|Y3 V2|
+-----+-----+-----+-----+
|Y4 U4|Y5 V4|Y6 U6|Y7 V6|
+-----+-----+-----+-----+
So we can access Y values from the shader using .r
. For example, if we call texture2D()
with coordinates inside the square formed by Y0-Y1-Y4-Y5, the resulting first component will be an interpolation of Y0 Y1 Y4 and Y5 values.
When uploading to a GL_RGBA
texture, the texels are split as follow:
+-----------+-----------+
|Y0 U0 Y1 V0|Y2 U2 Y3 V2|
+-----------+-----------+
|Y4 U4 Y5 V4|Y6 U6 Y7 V6|
+-----------+-----------+
This allows to access U and V values together using .g
and .a
. For example, if we call texture2D()
with coordinates inside the square formed by Y0-Y2-Y4-Y6, the resulting second component will be an interpolation of U0 U2 U4 and U6, and the resulting forth component will be an interpolation of V0 V2 V4 and V6.
EDIT: to test a YUYV 4:2:2 stream: vlc v4l2:///dev/videoX:chroma=YUYV
(comment)