YUY2 to RGB to YUY2 lossless?

[zerowalker]

Is it possible to convert YUY2 to RGB and then back to YUY2 losslessly?
I am pretty sure you can't but better to ask.

In this case the source would be YUY2 that's been converted to RGB.
So i got no control over that conversion.

[feisty2]

YUV 4:2:2 -> YUV 4:4:4 (point resampling) -> fake RGB (R=Y, G=U, B=V)
fake RGB -> YUV 4:4:4 (Y=R, U=G, V=B) -> YUV 4:2:2 (point resampling)

[zerowalker]

But, i can't do that if i don't control the first conversion right?
Cause what i wana do is use OBS (Capture software) which composites everything in RGB as it's worked on the GPU.
So everything will be converted, so if i capture a Console (YUY2) it will be converted to RGB.

Thanks:)

[feisty2]

then no way to get lossless
resampling (other than point) is lossy

[zerowalker]

Hmm, well maybe that can be possible in some way, though what is the fake RGB, i get what it means, but are you actually converting from YV24 to RGB or just pretending it's RGB?

[colours]

If you cannot control the first conversion, but you do know exactly what it does, you might be able to (almost) losslessly reverse it.

There are 2689428 legal YCbCr triplets with BT.709 coefficients and 8 bpc precision, and they all map to different RGB triplets if exact rounding is used, so it's possible in theory to exactly reverse a YCbCr→RGB conversion. (The same probably holds for BT.601 coefficients or whatever OBS uses, but I'm too lazy to look them up right now to confirm.)

However, if OBS does dithering or error diffusion or anything more fancy than just rounding when converting from YCbCr to RGB, then it's nigh impossible to recover the original.

The chroma resampling shouldn't be too hard to reverse losslessly; if it uses MPEG-2 chroma placement, then it's as simple as converting to 4:4:4 and dropping every other chroma sample (as in SeparateColumns().SelectEven()). If it uses MPEG-1 chroma placement, then you're out of luck, unless the first conversion was done with bilinear/bicubic, in which case you can use the aptly-named Debilinear and Debicubic filters.

[zerowalker]

Wait, so what is there else than Chroma Resampling, is it the colors themselves, how much does it differ if there was say rounding going on, cause if it's transparent than that's great.
If it's more than that, not so great;P

I am trying to see how OBS handles it, asking the developer, but if i have to guess i would think it's going through ffmpeg, though don't take my word on that one.

[zerowalker]

Here are samples.

Virtualdub:
https://s31.postimg.org/hrcjbtm7v/adwwadwa000000.png (https://s31.postimg.org/hrcjbtm7v/adwwadwa000000.png)

OBS - Set to Rec709 (For Video Capture) and RGB as the end result:
https://s31.postimg.org/v9jfo3yd7/2016_06_20_17_59_59000000.png (https://s31.postimg.org/v9jfo3yd7/2016_06_20_17_59_59000000.png)

Those settings were the ones that looks closes to Virtualdub, which i assume should be accurate.

EDIT:

Looking at them it seems that there is more than just color changes, different brightness as well as some other subtle changes (look at the noise).

[zerowalker]

No comments on the images?
Or maybe no useful information can be obtained from them?

[kolak]

Use e.g. fmtconv for such a task. It will be close to lossless and way better than Vdub or avisynth standard filters.
If I remember well standard way was something like 65dB PSNR where fmtconv 85dB.

[zerowalker]

Wait, how come it's different?
Shouldn't Virtualdub just capture it losslessly without any conversion, meaning i could use that as a source of how it should look?

[raffriff42]

VirtualDub assumes YUV sources are Rec601 when previewing them: ignore the preview display and evaluate your final encoded output using your regular media player. (Passing the YUV signal through VirtualDub without altering it, the colors in the preview display won't matter)

[zerowalker]

The screenshots are from Avisynth set to Rec709 if i recall correctly.
It was that way both OBS and Virtualdub looked very similar, but there is still a difference, so something is going on, guessing the YUV->RGB conversion.

[kolak]

Wait, how come it's different?
Shouldn't Virtualdub just capture it losslessly without any conversion, meaning i could use that as a source of how it should look?

I'm taking about YUV->RGB->YUV conversion, as per your original question. Not all solutions give the same results and none is lossless.

[zerowalker]

Oh, my bad i thought you meant how it was captured in virtualdub compared to others, even though it was YUY2 hence my confusion, thanks:)

But from the images i posted, is it possible to revert the OBS one to look the same as the virtualdub one?
Or has it been ruined?

EDIT:

Okay doing some testing i can make them look kinda alike now, i think i messed up something on the old tests.

How was i supposed to convert to YUY2 without resample the chroma again, i didn't get the "SelectEvery" thing.

EDIT 2:

Or wait, there seems to be quite some bleeding occuring in the OBS version, even in RGB mode, so something must happen to it during the conversion, weird.

[zerowalker]

Okay spoke with the Developer, here is the Conversion: https://github.com/jp9000/obs-studio/blob/master/libobs/data/format_conversion.effect

Here are two images showing the Chroma Bleeding issue.

Virtualdub:
https://abload.de/img/wdadawwadawd000000-ma8lxpq.png

OBS (Converted to YV24 rec709 for more correct color representation):
https://abload.de/img/2016-06-2116-38-50000s9aaq.png

[zerowalker]

Guess the conversion is very different when done in shaders like that, from my understandings it's averaging 2 pixels or something and that might be the issue, and that it might expect it to be left oriented.
Been looking at it and tried playing around, but sadly it's out of my league, and comparing it to other YUV to RGB shaders by googling didn't help much either as they are set up differently.

[pandy]

Instead YCbCr you can use YCoCg (YCoCg-R) - it should be fully reversible.

[nevcairiel]

Instead YCbCr you can use YCoCg (YCoCg-R) - it should be fully reversible.

Even with YCoCg you need a higher bitdepth for the converted image, if you store both in 8-bit you will have loss.

[zerowalker]

But the only thing i know is that the conversion is made in that shader i linked.
And that conversion to RGB causes some weird chroma bleeding which, so i got that problem first sadly, as i doubt you could reverse that effect.
It basically looks like it's been converted to YV12 before hand or something, look at the images above, It's very subtle though, but it's clearly there.

[pandy]

Even with YCoCg you need a higher bitdepth for the converted image, if you store both in 8-bit you will have loss.

Yes but this can be easily done on 10 bit and nowadays 10 bit looks like standard - NVidia done lot of work around this area.

[raffriff42]

Upsampling YUY2 to RGB shouldn't cause noticeable color bleeding if done right. Are you sure there's no chroma bleeding in the YUY2 original?

Try ChromaWarpSharpener (forum.doom9.org/showthread.php?p=1751572#post1751572) and see if it helps.

[zerowalker]

Upsampling YUY2 to RGB shouldn't cause noticeable color bleeding if done right. Are you sure there's no chroma bleeding in the YUY2 original?

Try ChromaWarpSharpener (forum.doom9.org/showthread.php?p=1751572#post1751572) and see if it helps.

Look at the images i provided some posts back.

One is directly from Virtualdub (YUY2 lossless, no conversion).
Other is through OBS, which would be (YUY2->RGB->YV24(to mimic the color better for the image)).

There is a "Chroma Halo" in one of them.

You can see the shader doing the conversion in that post as well.
I sadly don't understand what it's doing, but the Developer said it's Averaging by 2 pixels or something, cause it's faster that way.

I know about ChromaWarpSharpener or well at least something similar,
It's very good to help with chroma bleeding, but in this case it should be able to be prevented.

There is something wrong with the conversion, or maybe not wrong, but it's a bit inaccurate.

[colours]

It would help a lot if you could post the completely unprocessed input and output for OBS; in other words, the YUY2 data for one frame (in ebmp or y4m format) and the RGB output from OBS (in bmp, png, etc.).

Having an additional conversion to RGB done by VirtualDub or a matrix conversion complicates things somewhat.

[zerowalker]

I can't access the unprocessed input, it's being forced convert to RGB, and the best i can do is save it as RGB:(
I haven't done any RGB conversions in VirtualDub, Virtualdub can give me lossless data. OBS gets lossless data and converts it to RGB, i don't know if there is something wrong upon obtaining the YUY2 data, but i doubt it.
The developer himself seems to think it's the conversion and averaging that's causing this.

I was hoping to modify the shader into some conversion that avisynth uses, but sadly it's not that easy as copy pasting haha:P

[vivan]

I've no idea what that shader is doing (and it looks way too complicated for chroma upsampling and color conversion), but the most basic thing that people screw up is chroma location. They assume that chroma is located in the center of the corresponding luma block, which is wrong:
http://lea.hamradio.si/~s51kq/subsample.gif

If you can modify conversion then the best conversion is no conversion at all. Just pass values you have instead of doing any math on them, and switch to point resampling if you don't want to calculate proper coordinates.

[zerowalker]

Well i can't bypass the conversion.

But from my understanding it's based that the Chroma is in the center, think at least the developer said that, and he said that it might be that it should be left instead.

I however don't know how to modify it, as i have no idea what the values are, and playing around with it yielded only blockyness as a results, no chroma change.

[zerowalker]

Here are some more pictures.

Source is PS2 Component.

OBS YUY2 Decoded as Rec601 to RGB32:
https://abload.de/img/obs-ratchet-480p-rec6bdsf2.png

OBS YUY2 Decoded as Rec709 to RGB32:
https://abload.de/img/obs-ratchet-480p-rec7q9s1x.png

VirtualDub YUY2 Decoded as Rec601.
https://abload.de/img/virtualdub-ratchet-48x4sfz.png

EDIT:

OBS RGB PointResize x6:
https://abload.de/img/2016-06-2403-47-55000kysej.png

VirtualDub YUY2 -> RGB (For PointResize x6):
https://abload.de/img/awdawdwad000000oqsuk.png

[zerowalker]

Have spoken more with the developer, and he thinks the cause is that OBS isn't interpolating the Chroma.

"it's reading the chroma as it is for that specific pixel, that's all"

Here is another picture at how it can look (the red hood thing, it's blocky).

OBS RGB:
https://abload.de/img/2016-06-2501-57-01000w1jtr.png

Virtualdub -> Coring=True -> RGB:
https://abload.de/img/zvdgsghs0000449jk9h.png

[raffriff42]

ConvertToYUY2
YtoUV(
\ UToY8.Sharpen(0.6, -0.3).Spline64Resize(Width/2, Height, src_left=-0.25, src_top=0.0)
\ , VToY8.Sharpen(0.6, -0.3).Spline64Resize(Width/2, Height, src_left=-0.25, src_top=0.0)
\ , ConvertToY8
\ ).ConvertToYUY2

Was trying to get a YUY2 version of your "OBS RGB" screenshot, with a little less blockiness. The idea being to sharpen chroma in the horizontal (which in YUY2 is subsampled ie half luma resolution) and blur in the vertical (which is not subsampled).

src_left and src_top are there to adjust out any chroma shift.

post #29, image #1, "OBS RGB": original (left) and YUY2 using above script (right):
https://www.dropbox.com/s/uhyw2r9hjp8tgr1/OBS%20RGB%20crop2.png?raw=1 https://www.dropbox.com/s/y5y8frn7l127me9/OBS%20RGB%20crop2%20YUY2%20S06%20B03%20L025.png?raw=1

https://www.dropbox.com/s/tc350dnov4e98rg/OBS%20RGB%20crop1.png?raw=1 https://www.dropbox.com/s/6pit676y74mbzfc/OBS%20RGB%20crop1%20YUY2%20S06%20B03%20L025.png?raw=1

[zerowalker]

raffriff42; That's quite nice, look less blurry then normally, seems like a good workaround:o

vivan; Does Converting from YUV to RGB need interpolation, the developer stated that he isn't doing that and that seems to be the cause.

[huhn]

is there no RGB output for this console in question?

[zerowalker]

huhn, sadly no, or or well, through Scart i guess.
But component is only YUY2, and my Capture card only supports it.

But the problem (or the visual anomalies) exists in the YUY2 to RGB conversion, which seems to be caused by no interpolation.
I would like to hear from ppl who know how this works and tell me how Avisynth handles conversion as it pretty much looks identical between YUY to RGB.
Cause i am confused if there should be interpolation from a previous statement saying no conversion is the best conversion:S

[huhn]

4:2.2 is not centered so yes it should be interpolated.

[zerowalker]

Ah, then i get it, it's what the developer said as well, i just suck at this stuff.
Well i got the shader, and he told me that i just need to sample the two neighboring pixels or something.
Sadly i have no clue how to do that cause, in programming terms my guess would be that in theory, you take the value of both pixels and divide by 2.
But maybe that's just my simplistic view of things?

[huhn]

first of all forget pixels.

there are people in this forum that know this stuff way better than me.

i would say this is a bug in obs if they don't take care of this correctly.

[zerowalker]

Okay, meanwhile i will try some other alternative ways of capturing the data, though will do that in another thread as it's not within this topic.

[raffriff42]

If this unnamed console did not output properly positioned or interpolated chroma in the first place (as I suspect), you would have to adjust the interpolation and positioning yourself, as best you can. Experiment with different Sharpen and position parameters in my script snippet above.

[zerowalker]

The console has a name, and it's Playstation 2, my bad;P

Well i think that the Console produce the output just fine, the issue here is just that OBS doesn't interpolate, so i have confirmed where the problem lies, not much to be done about that except looking at the shaders.
I know at least that it Can be fixed, and that's good enough for now for me, and there are workarounds like you presented to prevent major chroma quality loss:)

And i also got Plan B which is to capture with Graphs instead, don't prefer that way though as i always have issues -_-.

:D