HybridFuPP 0.98a (11/18/2007)

[FuPP]

Here is a new version of this script :)

Download HybridFuPP 0.98a (http://www.unite-video.com/phpbb/download.php?id=2144)

Including : miscellaneous improvements, dll updates (which enables multithreading...), and a few new concepts related to my will to influate as strongly as possible on codecs behaviour, in order to get the best possible encoded picture.

And this is the reason why I have worked on this "best possible picture" concept (always subjective...), trying a kind of psycho-visual approach : I have tried to isolate what my eyes looked first in a motion picture (an what they looked less) and have included my conclusions in that script. Of course, all of this is purely empirical :)

Some of these concepts are included in preset HybridQ, but you can of course use all of them with other presets, you will have then to add some parameters beside the preset.


New parameters :

auto resizer adjustment : when activated, used resizer will be automatically degraded where the script estimates it is possible : it means for instance that even if you have chosen a lanczos resizer, a bicubic resizer could be possibly applied on some parts of the picture (where the script estimates that only a few details will be lost)

ARadjust = true/false


Edges Enhancement Automatic settings : when used, the script will calculate automatically (according to your source size, target size and type of resizer used) best edges enhancement settings : this enables keeping automatically a sharp picture even when downsizing (ex : 720 * 432 -> 608 * 256)

E_Auto_Str : true / false


Deblocking Motion Only : when you have a medium quality source, you can sometimes see blocks when picture is in motion. This setting enables to deblock only moving parts of the picture. That way, static areas are kept untouched (deblocking causes often softness even where blocks are barely visible)

DB_M_Only = true / false


Skin Processing : when some faces are present on a picture, your eye will give priority to them rather than to the background, for instance. That is the reason why I have implemented a skin detection mask and have enhanced that parts of the picture, forcing the codec to allocate more bits to them. Faces are therefore more detailed.

Skin = true / false
Skin_Type : 0 (=auto, adaptive), 1 (warm colored skins), 2 (cold colored skins) -> depends on the type of light used in the scene


Dark Scenes Enhancer : some codecs (like MPEG4-ASP) are not always efficient on dark scenes ; encoded picture is sometimes abnormally blured. This new function will improve this

DS_Enh_Thr : max luma threshold where effect will be applied


rem 1 : Because of some performance issues, parameter used to control both dark scenes and skin enhancement strength is the same : Enh_Str

rem 2 : Neither Motion Only Deblocking nor Dark Scenes Enhancement are activated by default in presets.

rem 3 : For now, supplied documentation is the 0.92b one.

rem 4 : MT ( SetMTMode(2,0) ) can be used with HybridFuPP : it causes a huge boost in performances for multi cores or processors owners


In summary : if you want to discover latest functionalities, I suggest you to use preset 'HybridQ', without changing parameters for the beginning (excepted if you want to activate Motion Only Deblocking [deblock=true, DB_M_Only=true] or dark scenes enhancer [DS_Enh = true])

Preset HybridQ has been mainly optimised for 2 pass encodes and low bitrates (500 - 700 with XVID, for instance)



Regards,

FuPP


ps : You MUST use supplied dll to get the script working correctly

EDIT : added addgrain.dll in the archive...

[buzzqw]

:thanks: FuPP!

thanks for let us playing with your script again!

BHH

[Adub]

You rule FuPP! I can't wait to play with this!

[FuPP]

So, have both a good play ;)

[Terranigma]

Thanks a lot for this script/filter. :D
Pure genius. :)

[FuPP]

Pure genius. :)

I'm afraid I'm definitely not :)

@all : any feedback ? I guess there are probably a few things to tweak a litlle bit...

[Spuds]

Fupp,

Here are a few thoughts as I look at the script, which is awesome, I keep learning from it each time I look !

Have you considered other than deen for the initial static clip generation, to me deen is difficult to control on details and given you are doing lots of mask work a less destructive denoiser for the initial static clip seems appropriate, like removegrain(mode 4/5/17). Granted you have made it very easy to change :)

I'm a little confused on the D2 mask, probably have not looked at it enough to see how its being used in its full context. Currently its mt_logic(mt_lutf(clip, clip, "avg", "x 75 <= 255 0 ?"), mt_logic(DM(Clip, DS_Enh_Thr),DM(Clip, D_Thr+8), "xor"), "and") I don't think much if anything is going to get through that mt_lutf expression, well maybe some dark reds ... should the x be y or is it defensive by nature?

Function mdenoise, I think mvdegrain1 would do a better job than mvdenoise plus you get idx capability for a performance boost.

In the motion mask creation MM , it has gamma=50. I thought gamma was only 1 or 2 for linear or quadratic, perhaps I've misunderstood.

[FuPP]

Have you considered other than deen for the initial static clip generation

Yes I did. But did not find time to test extensively other ones (like fft3dfilter or removegrain). Moreover I don't think deen is a bad one. It is quite versatile and, if correctly used, gives good results. You will notice that settings I use are quite conservative (radius = 1, min = 0.1).

Of course some others are probably better (and more sophisticated) but I will include them only after testing and if I am sure that they do not give too much speed penalty.

Of course, you know that you can change the denoiser used in HybridFuPP 'on the fly'

HybridFuPP([width], [height], preset = "[Whatever you like]", N2="fft3dfilter(blah blah blah)")

Or you can use HybridFuPP([width], [height], preset = "[Whatever you like]", N1="fft3dfilter(blah blah blah)", N2="")if you prefer denoising before resizing

given you are doing lots of mask work a less destructive denoiser for the initial static clip seems appropriate

Not sure to get you here... the initially denoised frame is covered by frames sometimes blured, sometimes by non denoised frames, or sometimes by frames with noise added, so what is the problem ? :)

mt_lutf(clip, clip, "avg", "x 75 <= 255 0 ?")

will return sometimes a white frame, sometimes a black one :)


Of course, merged with the entire expression, it will return the mask [mt_logic(DM(Clip, DS_Enh_Thr), DM(Clip, D_Thr+8),"xor")], or not (depending average luma). You can do the same using scriptclip(...averageluma() > 75 ? ....)

Function mdenoise, I think mvdegrain1 would do a better job than mvdenoise plus you get idx capability for a performance boost.

No idea, need to test. Not sure to get the point with idx...

In the motion mask creation MM , it has gamma=50. I thought gamma was only 1 or 2 for linear or quadratic, perhaps I've misunderstood.

the more, the less sensitive to noise (not limited to 2 for me)

FuPP

[foxyshadis]

Definitely look at MVDegrain; MVDenoise hasn't been worked on in some time and produces artifacts from time to time, and it is slower. The mvtools docs have some good info on how idx is used to speed up subpixel searches.

As for deen, the only filter I know with comprable speed and denoising ability is probably vaguedenoiser, but I think deen does a fine job as used here.

[FuPP]

Hi,

Will look at MVDegrain. Spuds talks about mvdegrain1. Is it a typo or a different one ?

FuPP

[Spuds]

Its either MVDeGrain1 or MVDeGrain2 (and if Fizick's comes through a mvdegrain3 some day!).

MVDeGrain1 -- uses one forward and one backward, like what you have now.
MVDeGrain2 -- uses two forward and two backward, better but slower.

The idx parameter is a kind of buffer, prevents the function from having to recompute a vector if it already did it on the previous frame.

Not sure to get you here... the initially denoised frame is covered by frames sometimes blured, sometimes by non denoised frames, or sometimes by frames with noise added, so what is the problem ?
That was my point, I think :) ... why start with a frame that deen kind of smoothed the details out of IMO. I like deen a lot but even with the 'gentile' parameters you set I think it removes to much.

will return sometimes a white frame, sometimes a black one
So its very defensive by nature ... from my limited testing there is not much is getting through that mask, need to try some really dark clips so I better understand.

Thanks!

[FuPP]

I have included here a few snapshots showing what this new version can do for you :) Encoded with XVID @ ~600 kbps

1 - picture 1 : simple lanczos
2 - picture 1 : HybridFuPP (preset HybridQ)
3 - picture 2 : simple lanczos
4 - picture 2 : HybridFuPP (preset HybridQ)

FuPP

edit : snapshots description

[Terranigma]

Well right now, they're appending approval. I have a few suggestions if you don't mind. Could you make it so that we could use resolutions divisble by 2 instead of 4 (or is there some sort of admonition preventing this from being possible?) Also, could you possibly add spline16/36 as possible resizing methods?

[FuPP]

mod 2 should be now possible (If I remember correctly, it was an avisynth limitation, not sure though).

Spline36 is already available : use HybridFuPP(...,...,Resizer=6)

FuPP

[Spuds]

I think for mod 2 to work you would have to convert out of yv12 colorspace and then back to it in the resizer function.

Also an update the blocksize logic to allow for a %2 option else mvanalyse will not be happy.

There is also the assert statement that would need to allow a %2 to pass.

[FuPP]

@Spuds : Thanks for your suggestions : I have never understood what all that lines in the script meant and your guidance could be precious :p (just joking :) )

I'm not a fan of silent conversions (if they are needed) but will think about that.

[linx05]

Thank you so much for the update! I'm not too all knowing about avisynth but your script helps me so much!

[Spuds]

@Fupp ... Well there are so many lines in the function it was obvious to me that you had forgotten how it worked. Glad I could help you out :p

Honestly I took your 'should now' statement as currently capable not as should now be 'do-able' if that makes sense. I'll stop helping now, don't need another poke in the old eye :)

[FuPP]

@Spuds : Any help is always welcome :)

@linx05 : Thanks !

Has anyone tested this new version ? Comments or suggestions would be welcome too !

FuPP

[EasyStart]

I have a problem when running HybridFupp in multi threaded mode. My avs file displays about half a second of video and then Windows Media Classic closes by itself. Here's my avs statement


mt("""HybridFuPP( preset="HybridQ" )""", threads=4 )


It is fine when I run HybridFupp without mt().

Easystart

[FuPP]

Hi,

I'm afraid that using MT with HybridFuPP that way is not a good idea. You should rather try :

SetMTMode(2,0)
HybridFuPP(...)

FuPP

[Razorholt]

What are the parameters that are contributing to the compressiblity? Can someone please give me some examples? I'm running compressibility tests over there: http://forum.doom9.org/showthread.php?t=136363

Thanks,
- Dan

[FuPP]

Hi,

You could try something like :

HybridFuPP(width,height,preset="HybridQ",E_Auto_Str=false,E_Str_X=35,E_Str_Y=20)
or
HybridFuPP(width,height,preset="HybridQ",skin=false,E_Auto_Str=false,E_Str_X=35,E_Str_Y=20)

reduce E_Str_X and E_Str_Y to get more compression (but get less sharpness)


FuPP

[Adub]

*puts on necromancer hat*

Yo Fupp!!

I found a small bug in the latest HybridFupp. It occurs when downscaling a 1280x720p video source using the "anime1" preset to 848x480.

It has to do with E_Auto_Str being enabled, and it's calculations result in a pretty high number (199) for E_str_x, which is then passed to unfilter. The result of this is that edges light up like Christmas trees.

Tracking down the bug is as far as I can manage, as I don't understand the mathematics of calculating the respective E_str_x and E_str_y.

*takes off necromancer hat*

[rkalwaitis]

Will there be any updating of this script? I think it is pretty good and a bit quicker than others I have used.

[rkalwaitis]

Ive reread all of the write-ups with this script and noticed that it was designed to work with the XVID codec in a bitrate range of 500 to 800. What matrix is recommend for the script and what were the XVID settings?

[FuPP]

Can't remember having said that... I have simply said that "Preset HybridQ has been mainly optimised for 2 pass encodes and low bitrates (500 - 700 with XVID, for instance)".

HybridFuPP can be used with any kind of codec, with any kind of bitrate range and any settings :)

FuPP

[rkalwaitis]

Oh my bad, optimized for two pass xvid was the example. I took it to be just for xvid. I must be honest, having read that I tried one of my older movies using your script and xvid at 700kbps. Very nice results. I never had much luck with making really nice xvid products. None of the usual blur and blocks. So far I don't seem to have to worry about what quantization matrix to use. I will of course play more with this. This seems to do the trick, without being miserably slow. (I use VirtualDub of course via avisynth)

What do I need to change to enhance the sharpness just a tad. This---E_Auto_Str=false, E_Str_X=??, E_Str_Y=??. Not sure what X and Y should be. I think the default is x=40 and y = 25. The higher the numbers the sharper the image?

Any chance of a show function that displays before and after images side by side, so the change is much easier to see? Similar to Lato's MCTemporalDenoise.

Awesome script. Thanks for the hard work

Robert

[FuPP]

E_Str_X and E_Str_Y are used to set horizontal and vertical sharpening strength. The higher, the stronger.

E_Auto_Str normally allows an automatic calculation of E_Str_X and E_Str_Y parameters when set to true. It is broken in HybridFuPP 0.98a (hey, it is alpha...;)).

About show function : there are already a lot of things that can be shown when using HybridFuPP. Simply use show="..." inside the command line (for instance, show="EM" will display Edge mask and Motion)

E : Edges
M : Motion
R : Deringing areas
D : Dark areas
X : Dark scenes enhancement areas
B : Bright areas
S : Skin detected areas
A : Auto-adjusted resizer algorithm areas

P : display all parameters currently used

All of them can be combined...


Any chance of a show function that displays before and after images side by side

But I can do that :)

[FuPP]

except that source clip and final one will not always have the same size :(

This is then not easy to "stack" them.

FuPP

[rkalwaitis]

okay ill play with the sharpening aspect of it. Although Im not the best with scripting, perhaps I can come up with something similar to what Lato does with his scripts. If he doesnt mind helping me out a bit. Ill have to ask him.

K

[rkalwaitis]

Yes your show option is very nice that is what I was thinking about, but stack over the original input or side by side.

[FuPP]

@rkalwaitis : I did that for you (quickly done, maybe buggy)

use show="-" (ie 'minus') for a vertically stacked comparison
show="|" (ie 'pipe') for a horizontally stacked comparison
show="/" (ie 'slash') for a comparison using interleaved frames

All these modes can be combined with other parameters in command show (ex : show "EM/")

edit : sorry, need to find a way to host the file...

[FuPP]

here it is : HybridFuPP 0.983a (http://files.filefront.com/HybridFuPPavsi/;13309703;/fileinfo.html)

[rkalwaitis]

oooh a new toy, thanks FuPP, you rock
Ill play with it and let you know what I think.

K

[rkalwaitis]

FuPP, messed around with the E_Str_X and E_Str_Y settings a bit. I noticed that the preset of 46,46 used by anime1 and your anime2 seemed to provide the best results even on non anime(I think I read that right). What I did was upped the values to to 55,55 and upped E_Str_N to 9. I also added an edgesmooth fast function after the hybrid call.

My script looked like this. Is this crazy? The edgesmoothfast function needs the sangnom.dll and was not set to hiQ as it seemed to oversmooth a bit although the picture was quite pleasing. Dont worry about the pc2tv function. That is just me playing around.

Import("C:\Users\baba\Desktop\HybridFuPP\HybridFuPP.avsi")
LoadPlugin("C:\Users\baba\Desktop\HybridFuPP\UnFilter.dll")
LoadPlugin("C:\Users\baba\Desktop\HybridFuPP\mvtools.dll")
LoadPlugin("C:\Users\baba\Desktop\HybridFuPP\mt_masktools.dll")
LoadPlugin("C:\Users\baba\Desktop\HybridFuPP\deen.dll")
LoadPlugin("C:\Users\baba\Desktop\HybridFuPP\AddGrain.dll")
LoadPlugin("C:\Users\baba\Desktop\MCTemporalDenoise\SangNom.dll")##needed for edgesmoothfaster
LoadPlugin("C:\Users\baba\Desktop\SeeSaw\MaskTools.dll")##needed for pc2tv
LoadPlugin("C:\Users\baba\Desktop\SeeSaw\RSharpenSSE2.dll")##needed for pc2tv
LoadPlugin("C:\Users\baba\Desktop\SeeSaw\RepairSSE2.dll")##needed for pc2tv
LoadPlugin("C:\Users\baba\Desktop\SeeSaw\RemoveGrainSSE2.dll")##needed for pc2tv
LoadPlugin("C:\Users\baba\Desktop\SeeSaw\DenoiseSharpen.dll")##needed for pc2tv

DGDecode_mpeg2source("C:\Users\baba\Desktop\FuPP\VTS_02_1.d2v", cpu=4, info=3)
ColorMatrix(hints=true, threads=0)

function EdgeSmoothFast(clip clp, int "strength", bool "hiQ", bool "twoway")###a Didee function I ran across
{
str = default( strength, 48 )
hiQ = default( hiQ, false )
twoway = default( twoway, false )

xpos = hiQ ? 0.5 : 0.0
xmul = hiQ ? 2 : 1
ox = clp.width()
oy = clp.height()

clp.pointresize(ox*xmul,oy*2)
sangnom(order=0,aa=str)
twoway ? ( hiQ ? turnleft().sangnom(order=1,aa=str).turnright().bicubicresize(ox,oy, 0.0,0.5, 0.5,0.5,ox*2,oy*2)
\ : bicubicresize(ox,oy, 0.0,0.5, 0,0.5,ox,oy*2).turnleft()
\ .pointresize(oy,ox*2).sangnom(order=0,aa=str).bicubicresize(oy,ox, 0.0,0.5, 0.5,0.5,oy,ox*2).turnright()
\ ) : bicubicresize(ox,oy, 0.0,0.5, 0,0.5,ox*xmul,oy*2)
}


function pc2tv_smooth(clip clp)###a Didee function I ran across
{
pc2tv = clp.coloryuv(levels="PC->TV")
pc2tvd=yv12lutxy(clp,pc2tv,"x y - 4 * 128 +","x y - 4 * 128 +","x y - 4 * 128 +",U=3,V=3)
yv12lutxy(clp,pc2tvd.removegrain(19),"x y 128 - 4 / -","x y 128 - 4 / -","x y 128 - 4 / -",U=3,V=3)
}

HybridFupp(preset="HybridQ",E_Auto_Str=false,E_Str_X=55,E_Str_Y=55,E_Str_N=9).EdgeSmoothFast().pc2tv_smooth()

[FuPP]

I don't know EdgeSmoothFast() function but I suppose it is a kind of anti-aliasing filter, right ?

As sharpening with HybridFuPP using high strength can cause aliasing, it can be a good idea to use this function after mine (hard to say without being able to see the result, with and without EdgeSmoothFast, though).

Be careful using E_Str_N : il produces grain on edges, forcing the codec (when encoding) to allocate more bits on edges, but can probably cause mosquito noise if too strong. The result will depend on the codec and bitrate used.

E_Str_N is a bit experimental and not useful when bitrate is sufficient.

FuPP

[rkalwaitis]

Here is the function

function EdgeSmoothFast(clip clp, int "strength", bool "hiQ", bool "twoway")
{
str = default( strength, 48 )
hiQ = default( hiQ, false )
twoway = default( twoway, false )

xpos = hiQ ? 0.5 : 0.0
xmul = hiQ ? 2 : 1
ox = clp.width()
oy = clp.height()

clp.pointresize(ox*xmul,oy*2)
sangnom(order=0,aa=str)
twoway ? ( hiQ ? turnleft().sangnom(order=1,aa=str).turnright().bicubicresize(ox,oy, 0.0,0.5, 0.5,0.5,ox*2,oy*2)
\ : bicubicresize(ox,oy, 0.0,0.5, 0,0.5,ox,oy*2).turnleft()
\ .pointresize(oy,ox*2).sangnom(order=0,aa=str).bicubicresize(oy,ox, 0.0,0.5, 0.5,0.5,oy,ox*2).turnright()
\ ) : bicubicresize(ox,oy, 0.0,0.5, 0,0.5,ox*xmul,oy*2)
}

[rkalwaitis]

E_Str_N with to high of settings I did not know. Thanks.

How do I keep my resizer or just have hybridfupp simply not resize?

Spline64Resize seems to keep more detail than other resizers. Would this be an option for HQ or Hybrid preset?

Thanks for the help
K

[FuPP]

HybridFuPP offers a wide choice of resizers. Put width and height in the command line and force resizer if needed

ex : HybridFupp(704,200, preset="HybridQ", resizer = 6, E_Auto_Str=false, E_Str_X=55, E_Str_Y=55, E_Str_N=9)

Resizer = 0 : BilinearResize
Resizer = 1 : BicubicResize(0.333, 0.333)
Resizer = 2 : BicubicResize(0.2, 0.4)
Resizer = 3 : BicubicResize(0, 0.5)
Resizer = 4 : BicubicResize(0, 0.6)
Resizer = 5 : LanczosResize
Resizer = 6 : Spline36Resize

I can add Spline64Resize if you wish.

if you force resizer to something between 3 and 6, you may want to reduce E_Str_X and E_Str_Y strength.

Be aware that using hybridQ, the script will automatically switch to a softer resizer in some areas of the picture (where it estimates it can do it without losing to much details). If you want to disable this behaviour, use ARAdjust = false.

HybridQ is really optimized for low bitrates encodes but you can try it with higher bitrates and see if it fits to you. Other presets (except animes ones) are more conventionnal.

[rkalwaitis]

AHA that is the trick. I could not figure out how to turn off the switching between resizers. ARAdjust = false. About adding Spline64Resize, I like that resizer. I almost always use it. Is there much of a difference between it and Spline36Resize? Isnt your listing of resizers in order of detail retained. I guess spline36 is next in the line if it how your resizers go. So in short yes please add it so I can monkey around with it.

Thanks
K

[FuPP]

here it is :

HybridFuPP 0.984a (http://postdownload.filefront.com/13333398//593c354a0e44b04c3de3f5f0a08dc1a0837b2f7832929157c3b54a9d462692e2bdfb280d1f996aba)

Regards,

FuPP

[rkalwaitis]

okay gonna run a test and compare. Thanks

K

[rkalwaitis]

I really like being able to use only the spline64resize option throughout the entire movie. It seems (to me) to give a much better picture. I edited my copy of the script to make one of your bicubic resize filters 0,0.75. It was 0,0.66. I assume it was the bicubic sharp resizer. So Ive noticed that the bicubic sharp setting is usually 0.075. I tried to use the N1 and N2 functions with some successes(at least I think so) and some failures. I was trying to LRemoveDust(17,2) or (4,2) and I would get colored bands across the image. Not sure if it was caused by the function itself or by the removegrain package used by those functions. I did have some very good results using degrainmedian and no banding (it was blue lines going horizontally across the image. The were 4 or 5 of them.

[rkalwaitis]

I tried to use HybridFupp with my MT dll version of avisynth and it does not allow for spline64resize, just for your info. I dont understand why, but that is the case. Ill look for a plugin.

K

[Reuf Toc]

Spline64resize was introduced in avisynth rev 2.58 and IIRC the MT version of avisynth is based on v2.57, that's why it doesn't work

[FuPP]

Bicubic resize starts to ring when c is above 0.6 this is the reason why it seems more rational to switch then to lanczos.

About your problems with external denoisers : try first tu use them without HybridFuPP with same settings and same picture size ans see if you get the same or not.

MT : there is a non official 2.5.8 with MT enabled. You should find it easely in the same forum. Fyi, I get sometimes some random crashes when using HybridFuPP with MT (did not test with 2.5.8 MT enabled though).

FuPP

[rkalwaitis]

Thanks Reuf Toc. Ill just look for it another way :) FuPP I like the show functions. I like to see the side by side/top and bottom comparison as I change the settings for your script. I use MPC to play the avs file. I suppose I should use AVSEdit.

[rkalwaitis]

Okay Ill change the setting back on the resize. Also I did not know about the other 2.5.8 MT Ill check it out.

Ill play with the external renoisers and see what the issues are and try to let you know. Ill try to figure out how to post images on this site so you can see.

thanks FuPP, you are as always great for helping.

K

[rkalwaitis]

0.984a is missing skin_type in the function command line. I just added it to the version I have and it worked.