SmoothUV plugin and RainbowSmooth derainbow script

[jackoneill]

A spatial derainbow filter and a script that uses it, ported from the corresponding Avisynth plugin (http://avisynth.nl/index.php/SmoothUV) and script (http://avisynth.nl/images/Rainbow_smooth.avsi).

Hope it works.

https://github.com/dubhater/vapoursynth-smoothuv

[ChaosKing]

Radius. Must be between 1 and 5.
Larger values smooth more.

4 and especially 5 have a green overlay, is this normal? Tested on 1080p source.

Or just use this
clip = core.std.BlankClip(format=vs.YUV420P8)
clip = clip.smoothuv.SmoothUV(radius=5, threshold=90)


Thx for a new toy :D

[jackoneill]

I suspect 4 and 5 are broken in the Avisynth plugin as well (I didn't test it).

[ChaosKing]

Just tried the avs version. Looks ok even with radius 7.
I used the dll from http://avisynth.nl/index.php/SmoothUV

[jackoneill]

That's weird. According to the source code, it should throw an error when radius is greater than 5.

[ChaosKing]

The changelog says something different :D
v1.4.0 2003/09/02 - Fixed HQ mode for radius > 4
- Added radius 7

[ChaosKing]

Heres a small compare script. I didn't know that ffms2 could load web stuff :eek:
avsdll = r"D:\AvisynthRepository\AVSPLUS_x86\Avisynth.dll"

clip = core.ffms2.Source("https://www.animemusicvideos.org/guides/avtech31/images/avs/rainbow-orig.jpg").std.DuplicateFrames([0]*20)
clip = clip.resize.Bicubic(format=vs.YUV420P8, matrix_s="709")
r3 = clip.smoothuv.SmoothUV(radius=3)
r4 = clip.smoothuv.SmoothUV(radius=4)
r5 = clip.smoothuv.SmoothUV(radius=5)

avs3 = core.avsw.Eval(' SmoothUV(radius=3) ', clips=[clip], clip_names=["last"], avisynth=avsdll)
avs7 = core.avsw.Eval(' SmoothUV(radius=7) ', clips=[clip], clip_names=["last"], avisynth=avsdll)


crop=200
clip = core.std.StackHorizontal([
core.std.CropRel(clip, crop,crop,0,0).text.Text("source"),
core.std.CropRel(r3, crop,crop,0,0).text.Text("radius 3"),
core.std.CropRel(avs3, crop,crop,0,0).text.Text("Avisynth r3"),
core.std.CropRel(r4, crop,crop,0,0).text.Text("radius 4"),
core.std.CropRel(r5, crop,crop,0,0).text.Text("radius 5"),
core.std.CropRel(avs7, crop,crop,0,0).text.Text("Avisynth r7"),

])

clip.set_output()

[jackoneill]

Looks like the source code is older than the DLL. :(

[foxyshadis]

I'm trying to get retdec built and installed, maybe that'll provide a bit more insight into what the binary-only is doing.

[jackoneill]

I'm trying to get retdec built and installed, maybe that'll provide a bit more insight into what the binary-only is doing.

retdec sounds interesting! I was able to disassemble the DLL in IDA Free, but to also decipher all the asm is a bit too much effort...

[jackoneill]

I figured out what's wrong when radius is 4 or 5. It calculates a sum which has to fit in 14 bits, so it can't be more than 16383. Radius 4 (9 * 9 * 255) is just too many pixels to fit.

This is easily fixed by using the pmulhuw instruction (introduced in SSE) instead of the pmulhw instruction (introduced in MMX).

https://github.com/dubhater/vapoursynth-smoothuv/releases/tag/v2


* Fix bad output with radius 4 and 5 (especially 5).
* Allow radius 6 and 7.
* Better precision in the calculations.

[ChaosKing]

Good job! I tested rad7 and the output is very similar to the avisynth version. I also noticed that smoothuv blurs the color red a bit or rather the black outlines.
But why is the mask parameter absent in r2? :P

[jackoneill]

Good job! I tested rad7 and the output is very similar to the avisynth version. I also noticed that smoothuv blurs the color red a bit or rather the black outlines.
But why is the mask parameter absent in r2? :P

Maybe you're looking at SmoothUV's parameters instead of RainbowSmooth?

[ChaosKing]

D'oh, you're right.