HybridFuPP 0.882b is out (05/14/2004)

[FuPP]

This avisynth script (avsi) enables both resizing (selectable algorithm), and filtering. Distinct filters can be applied to the different parts of the picture. The main idea is to strongly filter the less visible parts, and very softly the more visible. HybridFuPP uses the best decisions thanks to its ability to detect edges, motion, dark and very lit areas.

for edges -> sharpening filter (optional)
for motion -> spatial filtering (can be adjusted)
for dark and highly lit areas -> strong spatial filtering (can be adjusted)
for other parts -> very soft spatio-temporal filtering (can be adjusted)

HybridFuPP was originally designed for rather clean sources ; the spatio-temporal filter used is though powerful enough to enable noisy sources processing (when increasing the corresponding setting).

Moreover, HybridFuPP enables the user to define its own processing chains. Lot of things are therefore possible !

HybridFuPP is supplied with all necessary filters and a doc.

Link : click with right button to download ! (http://fupp.chez.tiscali.fr/HybridFuPP/HybridFuPP0882b.zip)

@mods : I have chosen to create a new thread for more readability and because a lot of discussions in the original one were no more valids. But if you don't feel it relevant, please merge both threads.

changelog :

- 0.882b :

- includes a bug fix for destination width or height lower than 256

- 0.881b :

- a bug when using "dering = true" or preset 'anime' has been corrected

- 0.88b :

- No more multiple resizing
- Standard filtering chains have been changed a lot
- Like dark areas, very lit areas are now processed too
- Chroma processing has been improved
- New and more efficient motion detection engine (thanks Manao). The old one, and an improved version of it, are though available too (use parameter MM_Type)
- New presets : "very low", "low", "medium", "high" (default), "very high" and "anime"
- « Debug » mode is now called « Show ». As previously, it enables to display the masks. Using it is easier too.
- New parameters (C_Str = chroma filtering strength, L_Thr = light areas threshold, LP_Str = dark and light areas filtering strength). Some others are no more used (chroma, dark, D_Str, Eresizer, Nresizer, CH…)
- Dering no more returns an error when used with non mod 16 destination width and height. It will simply be disactivated if needed.

- 0.85a to 0.87a :

- Non official alpha versions

- 0.84b :

- It is now possible to create personal processing chains
- Advanced debug mode (displayed with colors, modes can be now combined)
- Added a strength parameter for dark areas processing
- More speed
- No more limits concerning final size for chroma and dark (called ‘special’ in 0.7x versions) modes
- Deringing (optional) is now done after resizing. Mod 16 size limit concerns now only final size.
- Be careful : some parameters have been renamed (ex : special -> dark).
- Parameters order has changed ! so be careful if you used unnamed parameters in previous releases : ex HybridFuPP(448,320,12,7,10,30,9,32,17,1,5,true,true,true,30,0)

- 0.80a to 0.83a :

- Non official alpha versions

- 0.71b :

- Added a threshold parameter for special mode (ie dark areas processing)

- 0.7b :

- Special processing of dark areas, ie soft resizing + strong filtering (optional). Compressibility : up to 12% more
- It is now possible to choose resizing algorithm for edges
- Motion spatial filtering has been re-introduced (because of users request)
- A little bit less speed if dark areas processing is used (activated by default), a little bit more than 0.6b if it is not used

- 0.6b :

- Resizing algorithm and settings can be now selected for picture parts displaying no motion nor edges
- Edge detection algorithm changed (uses Msharpen now)
- Default values have been changed
- Presets have been changed
- It is now possible to override parameters included in a preset
- Spatial filtering has been removed when processing motion (offered a very weak compressibility gain)
- General picture quality improvements
- Speed improvement

- 0.5b :

- Added presets
- Changed defaults for better quality
- Changed spatio-temporal filter position in the chain

- 0.4b :

- Added parameters for both spatio-temporal and spatial filtering
- Changed default value for Edge detection threshold from 15 to 10
- Chroma smoothing is now done after resizing. Dering is still done before. Will see later...

- 0.3b :

- Errors with non mod16 sources are better managed

- 0.2b :

- First downloadable version

[Soulhunter]

THX... ;)

[Chainmax]

Awesome. I will test this with The Simpsons 1º Season, Spawn 1º Season and Grave of the Fireflies next week. :)
As promised, I'll post sample clips of the results. Would including the AVS scripts and the XviD settings help with troubleshooting?

[edit]
I have Deen for v2.5 beta2, what is the difference between that and the version you're bundling with HybridFuPP?
[/edit end]
[edit2]
How come none of the presets have sharpening enabled now?
[/edit2 end]

[Soulhunter]

Btw, a very nice readme you wrote... :)

Nice to read n' good to understand !!!


Congrats, a real paragon for some other scripts... :rolleyes:


Bye

[FuPP]

@soulhunter
Thanks !

@chainmax
First, sorry for the delay. This version was planned for last week...

Every piece of information is always useful. I personally do not use Xvid, but if I have questions following your tests, I will be able to provide all the details supplied to people that know it well.

Btw, xvid is already very optimized ; this script could be therefore even more useful for people using some other codecs.

It seems that there are sometimes confusion about Deen versions. HybridFuPP requires the latest one. This is the one provided inside the package.

Last, I've removed oversharpening in presets, but removed too some filters that hurt sharpness in some cases (like undot). Btw, if you need even more sharpness, you can use E_Str_X and E_Str_Y parameters.

Thanks in advance for the time you will spend !

FuPP

[DDogg]

FuPP, this is bit of a lazy question so pardon me in advance. Just to test this for full DVD resolution MPEG2, are the defaults ok to use as in "HybridFuPP()" ?

I also noticed it requires the older version of Deen as the newer (http://forum.doom9.org/showthread.php?s=&threadid=69052&highlight=deen) one will throw an error [this was incorrect information. Pardon me and disregard]. Was this by plan? Frankly, I have used the newer simply because it was newer. Never did any significant tests between them.

[FuPP]

Hi DDogg,

Defaults are ok. But with such a resolution, I would even try preset="medium" (or lower ?).

About Deen, I do not get your point. The version in this thread seems to be the same than the one I provide. Anyway, the one in HybridFuPP's package is the last one ! ;)

Regards,
FuPP

[Didée]

FuPP: :thumbsup:

Soulhunter: :D :p

[scharfis_brain]

Chainmax, your PM box is full

[FuPP]

Hi Scharfis !

two good news :

1/ I have included your idea about motion mask in the Type-3 mask (MM_Type = 3). Thanks again.

2/ I have corrected the doc putting back the missing 'c'. But did not spread it yet. You will have to wait next version :).

FuP*

[scharfis_brain]

hey, just call me Scharfi
"Scharfis" is the German form for the english term "Scharfi's"
(oh yes and I decided to give more attention to the 'c' by my signature, because You're not the first one dropping it :D )

I have seen you've implemented my ideo about motionmasking :)

how does it perform out compared to the other methods?

I think false detected motion won't appear anymore, but which side effect do occur?

[Chainmax]

FuPP, mf once told me on a PM that a median cut filter (like UnDot) can sharpen the picture. Are you sure it's the other way around?

scharfis_brain, your PM box is full as well. I tried to send you a PM a couple of hours ago and I couldn't. I cleaned my inbox now, so send me whatever you want.

[DDogg]

FuPP, thanks for the reply. I'll try preset="medium".

You are absolutely correct about the version of Deen in your package being the newest. I would have swore I had downloaded from that thread I referenced. I must have been mistaken and actually had the older one in my plugin Dir.

[DarkNite]

A much improved version (from what I've seen so far), and only a week late! ;)

Now that Scharfi has mentioned it I find myself looking for "side effects." Once I actually wake up I might even find some. :D

From my experience with anime sources I'd suggest taking advantage of the min variable in Deen v1.0 beta 1 (which must have been made with smoothing of gradients and flat surfaces in noisy anime on MarcFD's mind). When using Deen on anime I usually go with min=0.25 as a default. IMHO it has a cleaner effect than just using a large radius or high thresholds, and gives a decent compression boost while maintaining more detail.

It would definitely be good for dark areas at least, or am I just misunderstanding the filter's application within your script?

If you're script is taking advantage of this, and I just didn't see it with my half asleep eyes I apologize ahead of time.

/me stumbles towards coffee maker

Edit: My friend just cracked me up with "Hey, is there a flow chart for this script?"

[FuPP]

Originally posted by scharfis_brain
I have seen you've implemented my ideo about motionmasking :)
how does it perform out compared to the other methods?
It is of course better and more accurate than previously. But it still has the same issue : it only detects moving edges.

Manao's algorithm goes further...

Originally posted by Chainmax
a median cut filter (like UnDot) can sharpen the pictureLet's say that I have not experienced the same, and see undot bluring things (slightly though).

Originally posted by Darknite
When using Deen on anime I usually go with min=0.25 as a defaultFor now, the script uses 0.1, which is even more conservative. But preset anime is very new, so there's room for improvement and all suggestions will be welcome :).

It would definitely be good for dark areas at leastFor dark area processing, I use min = 0. Not sure what your question was though...
My friend just cracked me up with "Hey, is there a flow chart for this script?Your friend should really go back to bed ;)

[acrespo]

When I enable dering parameter, I receive an error in this line:

Final = Dering ? Eval("Final." + DE) : Final

If I change "Final." with "Final" the problem gone.

I am calling HybridFuPP with width=640, height = 480 and preset="anime"

[FuPP]

Yeah ! It seems I managed to create a bug a few seconds before releasing ! See post #1 for new version...

FuPP

[amango]

I can't download the file!

"Le site demandé n'est pas accessible par cette url."

[FuPP]

Please, use button right to download and choose "Save as..."

Regards,
FuPP

[Soulhunter]

Originally posted by Didée
Soulhunter: :D :p No, I meant mf's scripts... :D


Bye

[Chainmax]

Originally posted by FuPP:
But with such a resolution, I would even try preset="medium" (or lower ?).
How come? Since he's making full DVD resolution MPEG2, shouldn't he have as sharp a picture as possible?

[FuPP]

spatio-temporal filtering values used in HybridFuPP are very weak, as filtering is done after resizing.

As there is no resizing here, stronger values can be used ; this is the reason why I tell DDogg he can use preset medium or even lower.

Btw, in that case, because of the resolution used and all the processing done (ie mask calculations + filtering), HybridFuPP is probably very slow.

I wonder if DDogg was patient enough to wait till the end ;)

Regards,
FuPP

[mikeson]

I'm sorry for this question, but it makes me wondering. Can HybridFuPP be compared to Dust and how well does it perform?

[FuPP]

AFAIK, Dust uses full motion compensation ; that way it can apply temporal filtering on moving areas also. Some people around here may be more precise about Dust, as I don't know it very much.

HybridFuPP detects moving areas too, but blurs that parts.

HybridFuPP has other features like dark and light areas processing and edge enhancement (optional). In addition, HybridFuPP is rather modular, as you can define your own filtering processes. In summary, HybridFuPP is an... hybrid thingie ;)

FuPP

[DDogg]

I wonder if DDogg was patient enough to wait till the end Old doggs are always patient with great filters :). I have done one encode and the results were promising from a compression and quality standpoint, but my method was quick and casual so I don't want to report the results yet. In the next few days I'll set up 4 or five full OPV encodes using CCE 2.67.00.27 and will report back to this thread.

[DarkWave]

Hi,

@FuPP
1st thx for your great script. i´m encoding stargate season 7 atm and it looks promising.

just one question:
in your manual you wrote
MM_Type

Motion mask type. Three masks are available : 1 and 3 are rather close from each other as they detect moving edges (3 is though more “accurate”).

2 is the default mask : all the moving areas are detected, but this mask is a little bit more noise sensitive.

what is the real difference between all three masks?

greetz
DW

[FuPP]

Masks 1 and 3 are based on the SansGrip's nomosmooth algorithm (implemented in masktools) : it is very sensitive to moving edges. The difference between 1 and 3 is that 1 is always one frame "late". 3 is a bit better but consumes more time.

Type 2 is far more powerful because of a true motion detection engine (more sophisticated). The only drawback is that it is more sensitive to noise (will be far better in next version, since Manao has improved a lot his engine)

You can see the difference between each mask by using the following command line :

HybridFuPP(x, y, MM_Type = t, show="M")

x is your destination width
y is your destination height
t is the type of motion mask (1, 2 or 3)

(replace with your values)

1 is the fastest. 2 is the slowest. 3 is probably between both.

Default is 2.

FuPP

[Th3-S4int]

I want to test the Script, but it doesnt work for me!
Everytime i start the script via MPC or VdubMod i get the following error:

Avisynth: caught an access violation at 0x01f51856
attempting to read from 0x00000058

my script is:

LoadPlugin("C:\Programme\Ripping\GordianKnot\MPEG2Dec3.dll")
LoadPlugin("C:\Programme\Ripping\GordianKnot\unfilter.dll")
LoadPlugin("C:\Programme\Ripping\GordianKnot\msharpen.dll")
LoadPlugin("C:\Programme\Ripping\GordianKnot\Deen.dll")
LoadPlugin("C:\Programme\Ripping\GordianKnot\Motion.dll")
LoadPlugin("C:\Programme\Ripping\GordianKnot\Masktools.dll")
# SOURCE
mpeg2source("D:\THE_LAST_SAMURAI_DISC_1\VIDEO_TS\DVD2Avi.d2v",idct=6)
#
# CROPPING
crop(0,78,720,420, align=true)
HybridFuPP(576,224, preset="medium")

Can somebody help me?

[DarkWave]

Originally posted by FuPP
Masks 1 and 3 are based on the SansGrip's nomosmooth algorithm (implemented in masktools) : it is very sensitive to moving edges. The difference between 1 and 3 is that 1 is always one frame "late". 3 is a bit better but consumes more time.

Type 2 is far more powerful because of a true motion detection engine (more sophisticated). The only drawback is that it is more sensitive to noise (will be far better in next version, since Manao has improved a lot his engine)

You can see the difference between each mask by using the following command line :

HybridFuPP(x, y, MM_Type = t, show="M")

x is your destination width
y is your destination height
t is the type of motion mask (1, 2 or 3)

(replace with your values)

1 is the fastest. 2 is the slowest. 3 is probably between both.

Default is 2.

FuPP

Hi FuPP,

thx for your response and the explanations.
i´m encoding stargate sg-1 right now, using a slightly modified hybridfupp with different values and options. as speed doesn´t matter to me i´m sticking to mm_type 2 ;)

i´d like to thank you for your hard work and your understandable explanations (i was just going through some thread from didee and scharfi :D)

DarkWave

[FuPP]

Originally posted by Th3-S4int

Can somebody help me?

I can do that ;)

This is simply a bug when width or heigth are lower than 256. It is corrected in 0.882b.

:)

FuPP

[FuPP]

I´d like to thank you for your hard work and your understandable explanations
I thank you too for your kind words but I really don't know how you can find my explanations understandable : my english is sometimes very bad :D

Cheers,
FuPP

[DarkNite]

(will be far better in next version, since Manao has improved a lot his engine)

Entering search mode... must know more! :)

Just dropped in to say I'm still very pleased with the current version, and happily recommending it to many others. *Any improvement is more than welcome though, even if it makes the script produce results in mfps. :D


* Obscure mf script "speed" reference provided by mf Script Users of America Association, aka "the geek sitting next to me"

[Chainmax]

After much delay, here is the Grave of the Fireflies clip (37secs, 5.95Mbytes):

Clip (http://www.31012.com/~stuff/GotFClip4.avi)

XviD settings:

XviD-1.0-RC4-05042004 (Koepi's version)
Options used: HVS-Best-Picture matrix
B-Frames 2/1.50/1.00
Packed Bitstream
Closed GOV
Chroma Optimizer
Motion Search Precition 6
VHQ 4
Chroma Motion
Turbo ;-)
I-Frame interval = 240
Quantizers: 1-6/1-8/1-12
Trellis quantization
Credits @ quantizer 16

AVS script:

Telecide(order=1,guide=1,post=2,vthresh=25,blend=false,chroma=true,back=1,bthresh=15)
Decimate(cycle=5,mode=2,quality=3)
crop(8,4,704,472,align=true)
hybridFupp(592,320,preset="anime",resizer=5)
MSharpen()

I see some oversmoothing "splotches", if you know what I mean. Maybe the Deen settings in anime preset are too strong? Average video bitrate is ~850kbps, BTW.

[FuPP]

Could you give me a link to a sample of your source please ? (same part than the encoded one) ?

FuPP

[Chainmax]

You mean you want a vob sample, right? That's going to take a while since I have to copy the DVD to the harddrive again and look which vobs contain the clip (which is composed of different parts of the movie). I'll see what I can do.

[FuPP]

you can try these instead :

- remove HybridFuPP and encode. Then check if you see the same
- If no, try to put HybridFuPP again and add M="" to the command line

if it is still the same, then I'll need a sample.

Could you confirm me the version you've used ?

FuPP

[Chainmax]

I used v0.881b. I hope to be able to make the vob samples sometime this week.

[Chainmax]

I had to make three separate clips. The first clip (10.5Mbytes) is already available, the other two are being uploaded.

First Clip (http://www.31012.com/~stuff/Clip1.vob)

[Chainmax]

The second (10.1Mbytes) and third (8.5Mbytes) clip are available:

Second Clip (http://www.31012.com/~stuff/Clip2.vob)

Third Clip (http://www.31012.com/~stuff/Clip3.vob)

[FuPP]

Thanks a lot Chainmax ! You should really clean up your PM box though ;)

Cheers,
FuPP

[Chainmax]

No problem :). I cleaned my inbox, you can send me your PM now.

[wh00t]

hi,

thx for your great script, works very well. i've used it mainly on some avi's up to now, in order to transcode 'em into svcd's with cce.

nevertheless i've experienced some slow speeds when it comes to transcode the avs into mpg2 with cce v2.67 and i'd like to know if it's something normal. you have to know i've upgrated my puter not long ago from an amd thunderbird @ 1.4 ghz to a p4e (prescot) @ 2.8ghz with twinmos cl 2.5 ddr-ram 400mhz (1024kb total) on an asus i865i based motherboard with pat & hyperthreading enabled. i've to say speed difference is minor compared to before. i'll get 0.74 fps max with cce and this script (it uses ~54% cpu-time only too):

Import("*\AviSynth 2.5\plugins\Hybridfupp.avsi")
LoadPlugIn("*\AviSynth 2.5\plugins\MPEG2DEC3\MPEG2Dec3.dll")
LoadPlugIn("*\AviSynth 2.5\plugins\UnDot\undot.dll")
LoadPlugIn("*\AviSynth 2.5\plugins\Unfilter\Unfilter_Avisynth_2.5_Alpha\unfilter.dll")
LoadPlugIn("*\AviSynth 2.5\plugins\FluxSmooth\fluxsmooth.dll")
LoadPlugIn("*\AviSynth 2.5\plugins\MaskTools\MaskTools-v1.4.9\MaskTools.dll")
LoadPlugIn("*\AviSynth 2.5\plugins\msharpen.dll")
LoadPlugIn("*\AviSynth 2.5\plugins\deen.dll")
LoadPlugIn("*\AviSynth 2.5\plugins\Motion.dll")
AVISource("*.avi")
HybridFuPP(480,480,preset="high",E_Str_X=40,E_Str_Y=10,M_Thr=50,LP="",dering=true)
ConvertToYUY2(interlaced=false)

note: i've *'d actually path out, but it's proper in the avs aswell as the required versions of the plugins (although there might be some left over from older hybidfupp's, been lazy you know ;) )

.. to sum it up, am i set sth wrong, or is it all i can expect?

thanks

[FuPP]

Which avisynth version do you use ?

Do you use plugins provided with latest HybridFuPP (ie 0.822b ?). I can see masktools v1.4.9. It does not seem to be the right one.

Btw what was the original size of your clip ?

FuPP

[wh00t]

i'm using AviSynth 2.54, furthermore i've been using plugs provided w/ 0.882b (..i've just not deleted old ones/del'd them out of the avs, as i've been coping first part of the avs from beginning over.. as i said i've been lazy ;) ).

uhm,

I can see masktools v1.4.9. It does not seem to be the right one.


you're right i still use the old one, as there'd been a bunch of files in the attached masktools, and i wasn't sure which'd be needed.

clip size is 182.558.720 Bytes and it's a toon.

thanks for your quick response

[wh00t]

oh.. and btw. final result was great. it's just the speed that i'm concerned of

[FuPP]

You really should use latest avisynth 2.5.5 alpha (HybridFuPP has been tested with it).

It's the same about Masktools : some optimizations had been removed in some versions and put back in some others. Please use the one provided with HybridFuPP.

It's the same about motion.dll (motionvectors-v0.6), and for all other filters : use the one provided inside the package.

and then report, please

FuPP

EDIT : oh, and I guess when you're speaking about 0.74 fps max, you mean 0.74 * 25 = 18.5 fps :)

EDIT2 : about your clip size, I was talking about width and height

FuPP

[wh00t]

ok, done with updating all plugins, which have been outdated. meanwhile, i've encoded a little more.. but still 0.74 max (<- dunno if that's the actual fps, but it's cce's indicated "speed" i get). so that would be a good or a bad performance compared to your experience, if i may ask?

widh & height of the original avi is 576x432.

oh, and btw. do you probably know some settings for your script to remove noise some more, without making the picture too blurry (..as i'm using a relatively noisy/distorted avi as basis)?

thanks

[FuPP]

Could you try the same with QuEnc, it would be easier for me to compare (don't forget to tell me the settings you use).

It seems to be quiet poor results for a P4 2.8. You should get better performances.

And I confirm : this is actually 18.5 fps, not 0.74 :D

Regards,
FuPP

[wh00t]

QuEnc 0.51 - settings:

mpg2: yes
bitrate: 2250
use vbr: yes
hq: yes
2pass: yes
------------------
advanced settings:
------------------
standart, but with
2540 max bitrate
4:3 (its fs source)

process priority:
high (it's been using 91% cpu-time)

... oh, and i had to change:

ConvertToYV12(interlaced=false), as it doesn't seem to work with yuy2

=> 17.5 fps at start, then goes down to ~14 fps.

hm.. meanwhile i think its those filters hybridfupp is using.

[FuPP]

Yes. But now i can compare with what I get with may XP 2600+... Stay tuned...