Converting to/from RGB for offline editing of frames

[wonkey_monkey]

For a while I've been making single-frame edits to videos by copying the frames out of VirtualDub, painting on them in Photoshop, then reinserting them via AviSynth. I make sure to use the same matrix in both directions, but now I realise that I'm blurring chroma by doing this because I've going from YV12 to RGB (via YV24) and back again, and each time the chroma is being resampled. I suspect the problem might be exacerbated by the default chromaresample of "bicubic" which, in bicubicresize's case at least, additionally softens the image.

Setting chromaresample to "lanczos4" seems to give the best improvement (sinc actually seems to sharpen chroma), but is there another option? Some other way of converting which might better keep the representation of chroma?

[poisondeathray]

chromaresample using "point" or nearest neighbor - that will just duplicate and drop the same pixels you just duplicated

[poisondeathray]

But there seems to be a chromaresampling bug in avisynth+. There seems to be chroma shifting vertically down each iteration . It's related to a bug I posted about earlier in the avs+ thread

[wonkey_monkey]

I tried "point" but it doesn't seem to work across multiple conversions:

https://i.imgur.com/GeGPt3A.png

On the left is a single "point" conversion to RGB. On the right is a conversion to RGB, back to YV12, and back to RGB.

Not sure if it's a bug, exactly, or just an unintended by-product.

Apart from the shift, it may not work well if my painting creates sharp colour details.

[poisondeathray]

Definitely a bug with the avs internal functions. Nearest neighbor is supposed to just duplicates pixels (when you're doing 2x multiples such as 4:2:0 to 4:4:4, doubling the chroma plane dimensions , the same duplicated chroma samples are supposed to be dropped when going back to 4:2:0)

You don't get the shifting in vapoursynth each iteration with point . avsresize in avisynth should work too (I'll test that next)

It's probably the same bug I posted about 32bit processing that I suspected had to do with chroma resampling (I tested the chroma interpretation placements as well , MPEG1 (center), MPEG2 (left) )

It's easier to see the bug in testcharts

[wonkey_monkey]

I think I'll use lanczos4 (in fact I think I'll use it or spline16 for all chroma conversions, because of this issue (https://forum.doom9.org/showthread.php?t=175681) which I think is also replicated here, then I'll compare the old RGB with the new RGB to narrow down where pixels have actually changed, so that I can avoid too much resampling.

It's either that or I have to write my own YV12 image editor with, at the very least, clone stamp and healing brush...

[Cary Knoop]

Here is that great opportunity to make that final switch to Vapoursynth! ;)

[poisondeathray]

I think I'll use lanczos4 (in fact I think I'll use it or spline16 for all chroma conversions, because of this issue (https://forum.doom9.org/showthread.php?t=175681) which I think is also replicated here, then I'll compare the old RGB with the new RGB to narrow down where pixels have actually changed, so that I can avoid too much resampling.

It's either that or I have to write my own YV12 image editor with, at the very least, clone stamp and healing brush...

lanczos and spline will just blur too, because of the interpolation . Lanczos4 will actually cause ringing on sharp colored borders like lines.

Point (nearest neighbor) is clearly the better way to go in this situation (when it works like it's supposed to, that is) , because there is no interpolation when doing 2x multiples. That operation is lossless

So the only losses you incur will be from the RGB<=>YUV conversions (you can reduce that by using 16bit png/tiff) , instead of the additional chroma up/down scaling from lanczos or spline or any kernal other than point/nearest neighbor




Here is an animated gif illustration of the avs bug (avs+ r2915 mt x64; but also affects avs classic x86)

Left is avs internal "ConvertToXX" ; right is avsresize in avs+, the same zlib/zimg library used in vapoursynth internal resizer. MPEG2 placement is used (as default convention for SD and HD video), but I tested others, but still exists
https://i.postimg.cc/SKygJwW5/avs-chroma-pointresize-bug.gif


#starting with YV12 source
ConvertToRGB(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")
ConvertToRGB(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")
ConvertToRGB(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")




#starting with YV12 source
z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")
z_ConvertFormat(pixel_type="YV12", resample_filter="point", colorspace_op="rgb:709:709:f=>709:709:709:l")
z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")
z_ConvertFormat(pixel_type="YV12", resample_filter="point", colorspace_op="rgb:709:709:f=>709:709:709:l")
z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")
z_ConvertFormat(pixel_type="YV12", resample_filter="point", colorspace_op="rgb:709:709:f=>709:709:709:l")




Here is that great opportunity to make that final switch to Vapoursynth! ;)

yes, but avsresize does work here

Another reason for vapoursynth is float support for import/export. That YUV <=> RGB conversion can be lossless , so you can do lossless RGB processing in other programs

[StvG]

But there seems to be a chromaresampling bug in avisynth+. There seems to be chroma shifting vertically down each iteration . It's related to a bug I posted about earlier in the avs+ thread

Check this post if you didn't already - https://forum.doom9.org/showthread.php?p=1571315#post1571315

[poisondeathray]

Check this post if you didn't already - https://forum.doom9.org/showthread.php?p=1571315#post1571315

hahaha look above that post. I totally forgot about that.

But the avs documentation should be changed to clarify this and that it's not strictly "nearest neighbor" . Maybe even include Gavino's workaround

http://avisynth.nl/index.php/Resize#PointResize

It uses a Point Sampler or Nearest Neighbour algorithm


I prefer vapoursynth (and avsresize's) handling, it just works the way you expect it should

[wonkey_monkey]

Here is that great opportunity to make that final switch to Vapoursynth! ;)

Too old. Too old to begin the training.