aWarpSharp4xx: a script similar to aWarpSharp2 but supports all planar colorspaces

[Reel.Deel]

I recently had to process a handful a pictures which where 4:2:2. Unfortunately aWarpSharp2 only works with YV12, so I came up with aWarpSharp4xx, a wrapper function that allows aWarpSharp2 to be used with any planar YUV colorspace.
Usage is almost identical, the only difference is that when processing YV16/YV24/YV411 chroma=1 is identical to chroma=0. When processing Y8 or YV12 aWarpSharp2() is used directly so it will be faster and more memory efficient than processing other colorspaces.
Obvious requirements are AviSynth 2.6 and aWarpSharp2. Hopefully someone will find this useful :)

One thing I'm unsure about is the scaling of the luma mask used to guide the chroma. Right now I'm just using BicubicResize() to scale the mask to the appropriate dimensions. Would interesting to know how aWarpSharp2() does this internally.

function aWarpSharp4xx (clip src, int "thresh", int "blur", int "type", int "depth", int "depthC", int "chroma", bool "lsb", string "cplace")
{
depth = default(depth, 16)
depthC = default(depthC, src.IsYV24 ? depth : depth/2) #depth/3 is claimed to work better
chroma = default(chroma, 4)
lsb = default(lsb, false)
cplace = default(cplace, "MPEG2")

Assert(IsPlanar(src), "aWarpSharp4xx: input clip must be planar YUV")
Assert(0 <= chroma <= 6, "aWarpSharp4xx: argument chroma must be an integer between 0 and 6.")
Assert(cplace == "MPEG2" || cplace == "MPEG1", "aWarpSharp4xx: chroma siting must be MPEG2 or MPEG1")

src8 = lsb ? src.DitherPost(mode=-1) : src

IsY8(src) || IsYV12(src) && (chroma != 4 && chroma != 6) ? \
Eval("""
warp = ConvertToYV12(src8).aWarpSharp2(thresh=thresh, blur=blur, type=type, depth=depth, chroma=IsY8(src) ? 1 : chroma)
output = IsY8(src) ? ConvertToY8(warp) : warp
""") : \
Eval("""
y = ConvertToY8(src8).ConvertToYV12()
u = UtoY(src8).ConvertToYV12()
v = VtoY(src8).ConvertToYV12()

# warp each channel independently
y_warp = aWarpSharp2(y, thresh=thresh, blur=blur, type=type, depth=depth, chroma=1)
u_warp = aWarpSharp2(u, thresh=thresh, blur=blur, type=type, depth=depthC, chroma=1)
v_warp = aWarpSharp2(v, thresh=thresh, blur=blur, type=type, depth=depthC, chroma=1)

# warp chroma by guiding it with the luma edge mask
cshift = cplace == "MPEG2" && !src.IsYV24 ? -0.5 : 0
y_mask = aSobel(y, thresh=thresh).aBlur(blur=blur, type=type).BilinearResize(u.width(), v.height(), src_left=cshift)
u_warp2 = aWarp(u, y_mask, depth=depthC, chroma=1)
v_warp2 = aWarp(v, y_mask, depth=depthC, chroma=1)

# "fill with zeroes"
blank = BlankClip(u, pixel_type="Y8", color_yuv=$808080)

# luma/chroma processing
output = chroma==0 || \
chroma==1 ? YtoUV( blank , blank , y_warp ) : \
chroma==2 ? YtoUV( u , v , y_warp ) : \
chroma==3 ? YtoUV( u_warp , v_warp , y_warp ) : \
chroma==4 ? YtoUV( u_warp2 , v_warp2 , y_warp ) : \
chroma==5 ? YtoUV( u_warp , v_warp , y ) : \
chroma==6 ? YtoUV( u_warp2 , v_warp2 , y ) : \
src8
""")
output8_16 = lsb==true ? output.Dither_convert_8_to_16() : output
y_ld = chroma < 5 ? 3 : 2
uv_ld = Min(chroma, 3)
output16 = Dither_limit_dif16(src, output8_16, thr=1.0, elast=1.5, y=y_ld, u=uv_ld, v=uv_ld)
Return(lsb==true ? output16 : output8_16)
}


I'm not an expert at AviSynth scripting so if something can be done a bit more efficiently please let me know.

[real.finder]

I was thinking of good "warp chroma by guiding it with the luma" away from aWarpSharp2 to get better result

and never think I will get it with this script, but it better than aWarpSharp2 :)

yes, your script result different from original aWarpSharp2 in yv12 after delete "IsYV12(src)" from "IsY8(src) || IsYV12(src) ? \" line

look like original aWarpSharp2 do the "warp chroma by guiding it with the luma" after aWarpSharp2 in luma even with chroma=6

so this is my little edition to make it work with yv12 too

*deleted*

edit: original one above now work with yv12 in some case, but if you want my edition it can be find below

edit: the different from original aWarpSharp2 is that the original use type=0 and aBlur use type=1, so if we use type=0 in aWarpSharp4xx it will be like the original one

[Reel.Deel]

look like original aWarpSharp2 do the "warp chroma by guiding it with the luma" after aWarpSharp2 in luma even with chroma=6


If I'm understating you correctly you're saying that aWarpSharp2(chroma=6) warps the luma first and then uses it to guide the chroma channels? I'm not sure how aWarpSharp2 works internally but I did a little test that confirms the luma channel is simply copied from the input clip when chroma=6.
original = last
filtered = last.aWarpSharp2(chroma=6)
mt_lutxy(original,filtered, "x y - abs 255 *", u=-128, v=-128)

[feisty2]

If I'm understating you correctly you're saying that aWarpSharp2(chroma=6) warps the luma first and then uses it to guide the chroma channels? I'm not sure how aWarpSharp2 works internally but I did a little test that confirms the luma channel is simply copied from the input clip when chroma=6.
original = last
filtered = last.aWarpSharp2(chroma=6)
mt_lutxy(original,filtered, "x y - abs 255 *", u=-128, v=-128)

Warpsharp detects the "exact" edge, and pixels around the edge would move along to the edge by vectors, "guide by luma", that's telling you you are processing chroma planes but you get the vector and edge reference from luma plane

[Reel.Deel]

Sigh....Yes I know what "guide by luma" means. But that's not what we're discussing...

[feisty2]

Yeah, you wanna know will warpsharp process luma if set "guide by luma", well, no, cuz you don't need to process it first to use it as a reference of detecting steps

[real.finder]

If I'm understating you correctly you're saying that aWarpSharp2(chroma=6) warps the luma first and then uses it to guide the chroma channels? I'm not sure how aWarpSharp2 works internally but I did a little test that confirms the luma channel is simply copied from the input clip when chroma=6.
original = last
filtered = last.aWarpSharp2(chroma=6)
mt_lutxy(original,filtered, "x y - abs 255 *", u=-128, v=-128)


the luma will copied from the input clip, yes. but I am talking about chroma, and just a wonder, not sure about that

[Reel.Deel]

the luma will copied from the input clip, yes. but I am talking about chroma, and just a wonder, not sure about that

I tried guiding the chroma multiple ways but I cannot reproduce how aWarpSharp2(chroma=6) does it internally. I cannot get the same output, not even remotely. :confused:

With my script with chroma=6 it goes like this: source-->aSobel-->aBlur-->BicubicResize (to scale mask to appropriate dimensions) this edge mask is then used to guide the chroma.

From your first post I thought you meant that aWarpSharp2(chroma=6) does something like this (internally): source-->aWarpSharp2-->aSobel-->aBlur - this edge mask is then used to guide the chroma channels.

[real.finder]

I tried guiding the chroma multiple ways but I cannot reproduce how aWarpSharp2(chroma=6) does it internally. I cannot get the same output, not even remotely. :confused:

With my script with chroma=6 it goes like this: source-->aSobel-->aBlur-->BicubicResize (to scale mask to appropriate dimensions) this edge mask is then used to guide the chroma.

From your first post I thought you meant that aWarpSharp2(chroma=6) does something like this (internally): source-->aWarpSharp2-->aSobel-->aBlur - this edge mask is then used to guide the chroma channels.

waiting SEt to know how it work, but anyway, I like it now like this, Yet knowing how the original work is also interesting

[zero9999]

here (https://gist.github.com/line0/7518e4c9d5014275bac0)'s a little modification of the script to enable stacked 16-bit support. I've also cleaned it up a bit and added a very minor optimization for yv12+chroma==1.

Requires AVS+ because eval blocks are nasty.

Oh and Reel.Deel, as for the differences you're seing, maybe consider this:

<_08> uh, line0, just remembered: original awarp uses half depth for chroma
<_08> though depth/3 works better


Edit: fixed for chroma shift and use faster resizer for the mask

[real.finder]

according to http://avisynth.nl/images/Colorspace-subsampling.png

I make some changes on the script


# make aWarpSharp2 work on other planar, for avs 2.6
# v1.33

function aWarpSharp4xx (clip osrc, int "thresh", int "blur", int "type", int "depth", int "chroma", int "cdepth", string "cplace", bool "bic", bool "lsb")
{

lsb = default(lsb, false)
src = lsb ? osrc.DitherPost(mode=-1) : osrc

depth = default(depth, 16)
chroma = default(chroma, 4)
cplace = default(cplace, "MPEG2")
cdepth = default(cdepth, IsYV24(src) ? depth : depth/2)
bic = default(bic, true)

Assert(IsPlanar(src), "aWarpSharp4xx: input clip must be planar YUV")
assert(0 <= chroma <= 6, "aWarpSharp4xx: argument chroma must be an integer between 0 and 6.")
assert(cplace == "MPEG2" || cplace == "MPEG1", "aWarpSharp4xx: chroma siting must be MPEG2 or MPEG1")

IsY8(src) || (IsYV12(src) && (chroma == 0 || chroma == 1 || chroma == 2)) ? \
Eval("""
warp = ConvertToYV12(src).aWarpSharp2(thresh=thresh, blur=blur, type=type, depth=depth, chroma=IsY8(src) ? 1 : chroma)
output = IsY8(src) ? ConvertToY8(warp) : warp
""") : \
Eval("""
y = ConvertToY8(src).ConvertToYV12()
u = UtoY8(src).ConvertToYV12()
v = VtoY8(src).ConvertToYV12()

# warp each channel independently
y_warp = aWarpSharp2(y, thresh=thresh, blur=blur, type=type, depth=depth, chroma=1)
u_warp = aWarpSharp2(u, thresh=thresh, blur=blur, type=type, depth=cdepth, chroma=1)
v_warp = aWarpSharp2(v, thresh=thresh, blur=blur, type=type, depth=cdepth, chroma=1)

# warp chroma by guiding it with the luma edge mask
cshift = IsYV411(src) ? -1.5 : (cplace == "MPEG1" && IsYV12(src)) || IsYV24(src) ? 0 : -0.5
y_mask = bic ? aSobel(y, thresh=thresh).aBlur(blur=blur, type=type).BicubicResize(u.width(), v.height(), src_left=cshift) : \
aSobel(y, thresh=thresh).aBlur(blur=blur, type=type).BilinearResize(u.width(), v.height(), src_left=cshift)
u_warp2 = aWarp(u, y_mask, depth=cdepth, chroma=1)
v_warp2 = aWarp(v, y_mask, depth=cdepth, chroma=1)

# "fill with zeroes"
blank = BlankClip(u, pixel_type="Y8", color_yuv=$808080)

# luma/chroma processing
output = chroma==0 || \
chroma==1 ? YtoUV( blank , blank , y_warp ) : \
chroma==2 ? YtoUV( u , v , y_warp ) : \
chroma==3 ? YtoUV( u_warp , v_warp , y_warp ) : \
chroma==4 ? YtoUV( u_warp2 , v_warp2 , y_warp ) : \
chroma==5 ? YtoUV( u_warp , v_warp , y ) : \
chroma==6 ? YtoUV( u_warp2 , v_warp2 , y ) : \
src
""")
output8_16 = lsb==true ? output.Dither_convert_8_to_16() : output
y_ld = chroma < 5 ? 3 : 2
uv_ld = Min(chroma, 3)
Return(lsb==true ? Dither_limit_dif16(osrc, output8_16, thr=1.0, elast=1.5, y=y_ld, u=uv_ld, v=uv_ld) : output8_16)
}


and here http://pastebin.com/aVGgz0f8 the one that will work like original one

and this will be faster http://pastebin.com/09sXzKfa work with last aWarpSharp2

[cretindesalpes]

A few months ago I needed a 4:4:4 aWarpSharp2 for my toolchain and wrote a modification to the original plug-in:

awarpsharp2-2015.10.25.zip (https://ldesoras.fr/src/avs/awarpsharp2-2015.10.25.zip)

I wrote this quickly so I’m not sure everything works well. If the plug-in is reported to work correctly, I’ll post the modification in the official aws2 thread.

[Reel.Deel]

Time for an update! See 1st post (http://forum.doom9.org/showthread.php?p=1701178#post1701178).


Add "depthC" parameter, independent depth for chroma, if this value is not specified it defaults to depth/2, except in YV24 where it defaults to just depth. Idea borrowed from _08.
Add "cplace" parameter, chroma placement, "MPEG2" for DVD, Blu-ray, web videos and just about any other 'consumer format'. "MPEG1" for some JPGs, MJPEG, and anything else that might use it :).
Add "lsb" parameter, 16-bit processing - true: if input clip is stack16, false: input clip is a regular 8-bit clip. Output will always be the same format as the input. Dither (http://forum.doom9.org/showpost.php?p=1386559&postcount=3) is now required.
Luma mask used to warp chroma is now resized with BilinearResize.
YV12 optimization: if "chroma" does not equal 4 or 6 and cplace="MPEG1" then use aWarpSharp2 directly.
Note: chroma warp/output is different than aWarpSharp2 (really only when using chroma=4 or 6), I personally prefer aWarpSharp4xx. real.finder also showed me some examples where aWarpSharp4xx is indeed better but as always YMMV.
For those who use AviSynth+, script can be found here (https://gist.github.com/Reel-Deal/986abfec2e3a375fdcd2).

If you find any bugs or if you have any suggestions and or comments please don't hesitate.

Big thanks to line0 for all the help! :thanks:

A few months ago I needed a 4:4:4 aWarpSharp2 for my toolchain and wrote a modification to the original plug-in:

awarpsharp2-2015.01.29.zip (http://ldesoras.free.fr/src/avs/awarpsharp2-2015.01.29.zip)

I wrote this quickly so I’m not sure everything works well. If the plug-in is reported to work correctly, I’ll post the modification in the official aws2 thread.

Thank you very much cretindesalpes! Will definitely test it out!

[TheSkiller]

Thanks for the update. :) Not being limited to YV12 when you're working with a lossless analog 4:2:2 capture – and even stack16 support – is really a relief.


"MPEG1" for some JPGs, DV, and anything else that might use it :).Wait! DV does not use MPEG1 chroma placement. It uses it's own placement where Cb and Cr samples (for interlaced PAL DV) are vertically placed between luma samples by half a luma pixel. Where for interlaced MPEG2 the placing has different offsets for both fields (don't know the numbers though).

[Reel.Deel]

Wait! DV does not use MPEG1 chroma placement.

You're indeed correct, must of been too early in the morning :p.
I don't know too much about interlaced chroma placement and I don't think many people use DV nowadays anyways. Regardless this script is meant for progressive input only.

Edit: reel.finder added this: cshift = IsYV411(src) ? 1.5
src_left=1.5, would that be correct for progressive DV (NTSC) (http://avisynth.nl/images/Colorspace-subsampling.png)?

[Reel.Deel]

A few months ago I needed a 4:4:4 aWarpSharp2 for my toolchain and wrote a modification to the original plug-in:

awarpsharp2-2015.01.29.zip (http://ldesoras.free.fr/src/avs/awarpsharp2-2015.01.29.zip)

I wrote this quickly so I’m not sure everything works well. If the plug-in is reported to work correctly, I’ll post the modification in the official aws2 thread.

I see you only mentioned YV24, does it work correctly with the other planar colorspaces also? I tried with YV16 and didn't notice anything wrong but I just wanna be sure.

[cretindesalpes]

4:1:1 is not supported.

[real.finder]

A few months ago I needed a 4:4:4 aWarpSharp2 for my toolchain and wrote a modification to the original plug-in:

awarpsharp2-2015.01.29.zip (http://ldesoras.free.fr/src/avs/awarpsharp2-2015.01.29.zip)

I wrote this quickly so I’m not sure everything works well. If the plug-in is reported to work correctly, I’ll post the modification in the official aws2 thread.

ColorBars(width=720, height=480, pixel_type="RGB32")
ConvertToYV12()
aWarpSharp2()

I got in avspmod


Traceback (most recent call last):
File "avsp.pyo", line 9061, in OnMenuVideoToggle
File "avsp.pyo", line 13855, in ShowVideoFrame
File "avisynth.pyo", line 462, in GetFrame
WindowsError: exception: access violation reading 0xFFFFFFFF

[real.finder]

I edit this post http://forum.doom9.org/showpost.php?p=1714103&postcount=2

aWarpSharp4xx(type=0,cplace="mpeg1") in yv12 will be like aWarpSharp2()

so this mean that aWarpSharp2 warp in mpeg1 right? Is the cretindesalpes aWarpSharp2 fix this?

[cretindesalpes]

real.finder:

I could reproduce the bug once, but after upgrading the projects to MSVC 2013 and recompiling it, I couldn’t show it anymore. So I updated the version in my previous post. Could you try it again?

And regarding your second question, yes, awarpsharp2 uses MPEG-1 chroma placement, and no, I didn’t fix anything about this issue.

[real.finder]

real.finder:

I could reproduce the bug once, but after upgrading the projects to MSVC 2013 and recompiling it, I couldn’t show it anymore. So I updated the version in my previous post. Could you try it again?

And regarding your second question, yes, awarpsharp2 uses MPEG-1 chroma placement, and no, I didn’t fix anything about this issue.

in avs25 ok, in avs26 same crash

and I note that y8 not supported

[Reel.Deel]

in avs25 ok, in avs26 same crash
I cannot reproduce this with AviSynth+ r1576 and the latest AvsPmod (http://forum.doom9.org/showthread.php?t=153248&page=58#post1733655).

and I note that y8 not supported

I might help if you're a bit more verbose.


With the script below I get this error message:

Avisynth read error:
Filter error: GetPlaneWidthSubsampling not available on Y8 pixel type.

ColorBars(width=640, height=480, pixel_type="YV12")
ConvertToY8()
aWarpSharp2()

If I add chroma=1 then it works correctly on Y8. Same with aWarp and aWarp4. Regarding aBlur and aSobel, they both work since chroma=1 by default, any other value and you get the same error as above. aBlur for some strange reasons works when chroma=2 :confused:.


@cretindesalpes

aWarpsharp2 as it is now processes the chroma channels with (blur+1)/2 and depth/2. This is ok for subsampled colorspaces but for YV24 I think is better to use the full blur and depth values. A depthC parameter would be lovely :). Any thoughts?

Edit: I forgot that _08 had already incorporated the full depth on YV24 and the depthC parameter. I believe _08 also made some changes regarding the Y8 problems above (haven't fully tested it yet). Maybe you can incorporate his changes so we don't have multiple versions floating around. Anyways here's _08's mod in case you're interested: awarpsharp2-2015.04.06.zip (https://raw.githubusercontent.com/Zeght/Morefun/master/awarpsharp2-2015.04.06.zip)

[real.finder]

I cannot reproduce this with AviSynth+ r1576 and the latest AvsPmod (http://forum.doom9.org/showthread.php?t=153248&page=58#post1733655).

Anyways here's _08's mod in case you're interested: awarpsharp2-2015.04.06.zip (https://raw.githubusercontent.com/Zeght/Morefun/master/awarpsharp2-2015.04.06.zip)

I aready say that here http://forum.doom9.org/showthread.php?p=1745892#post1745892

yes, _08's mod ok in Y8 but has the same crash in avs 2.6

[cretindesalpes]

Thanks. I’ll check it when I get a moment and try to offer a correct support for Y8.

[bxyhxyh]

@cretindesalpes
Does awarpsharp2 you modified support 16bit stacked clip?

[bxyhxyh]

So it doesn't support stacked 16bit clips.

One thing that bugs me is it crashes when I use it on 1480x800 YV12 clip and on 714x468 YV12.
Called padding(6,6,6,6) on first video then works.