HybridFuPP : new 0.84b

[FuPP]

Here is a script (directly inspired from mf hybridresize) doing resizing + edge, motion and dark areas aware filtering :

- edges are sharply resized (oversharpening is though optional)
- moving areas of each frame are resized with a soft algorithm and spatialy filtered
- dark areas are softly resized and strongly filtered
- other areas are resized with the algorithm of your choice and slightly filtered with a temporal filter.

This function is designed for resizing + filtering, and delivers a sharp image while enabling better compressibility than using bilinear resizing !

You must use avisynth 2.55 load mpeg2dec3, unfilter, undot, fluxsmooth, msharpen (new, since 0.6b) and masktools (v 1.4.14) in your main script (or have them in your avisynth plugin directory)

download avsi 0.84b (2004/03/14) + readme (click right on link and save target)

changelog :

0.84b:

- It is now possible to create personal processing chains
- advanced debug mode (displayed with colors, modes can be now combined)
- added a strength parameter for dark areas processing
- more speed
- no more limits concerning final size for chroma and dark (called ‘special’ in 0.7x versions) modes
- deringing (optional) is now done after resizing. Mod 16 size limit concerns now only final size.
- be careful : some parameters have been renamed (ex : special -> dark).
- parameters order has changed ! so be careful if you used unnamed parameters in previous releases : ex hybridFupp(448,320,12,7,10,30,9,32,17,1,5,true,true,true,30,0)

0.80a to 0.83a : not released

0.71b :
- added a threshold parameter for special mode (ie dark areas processing)

0.7b :

- special processing of dark areas (experimental), ie soft resizing + strong filtering (optional). Compressibility : up to 12% more
- It is now possible to choose resizing algorithm for edges
- motion spatial filtering has been re-introduced (because of users request)
- a little bit less speed if dark areas processing is used (activated by default), a little bit more than 0.6b if it is not used


0.6b :

- resizing algorithm and settings can be now selected for picture parts displaying no motion nor edges
- edge detection algorithm changed (uses Msharpen now)
- default values have been changed
- presets have been changed
- it is now possible to override parameters included in a preset
- spatial filtering has been removed when processing motion (offered a very weak compressibility gain)
- general picture quality improvements
- speed improvement

- 0.5b : added presets (low, medium, high, high2)
changed defaults for better quality
changed spatio-temporal filter position in the chain
- 0.4b : add parameters for both spatio-temporal and spatial filtering
changed default value for Edge detection threshold from 15 to 10
chroma smoothing is now done after resizing. Dering is still done before. Will see later...
- 0.3b : errors with non mod16 sources are better managed
- 0.2b : first downloadable version- All feedback will be greatly appreciated -

original script (it has been kept for people who want to understand further discussions, but latest script is included in the downloadable file):

LoadPlugin("C:\video\avsfilters\yv12\mpeg2dec3.dll") 
LoadPlugin("C:\video\avsfilters\yv12\masktools.dll") 
LoadPlugin("C:\video\avsfilters\yv12\undot.dll") 
LoadPlugin("C:\video\avsfilters\yv12\unfilter.dll") 
LoadPlugin("C:\video\avsfilters\yv12\fluxsmooth.dll") 

mpeg2source("F:\Films\test.d2v",idct=7,cpu2="ooxxox",moderate_h=10,moderate_v=20).crop(24,80,672,416,align=true) 
blindpp(quant=2,cpu2="ooooxo",moderate_h=30,moderate_v=50) 
HybridFupp(448,320,255,40,12) 
colorYUV(off_y=5,gain_y=25,cont_v=-60) 
undot() 
addBorders(16,128,16,128) 


function HybridFupp(clip input, int width, int height, int "sharpness", int "sh1", int "sh2") 

{ 
resizemask = input.edgemask(3,8,255,255,"sobel",Y=3, V=1, U=1).inflate().Levels(0, 1.0, 24, 0, 255).\ 
BilinearResize(width, height).GreyScale().ConvertToRGB32() 

MoMask = motionmask(input,thY1=30,thY2=30,thSD=15,y=3,u=1,v=1).fity2uv().inflate().Levels(0, 1.0, 24, 0, 255).\
BilinearResize(width,height).greyscale().converttorgb32() 

soft = input.BilinearResize(width, height) 

MEdgesharp = Mask(input.LanczosResize(width, height).unfilter(sh1,sh2).ConvertToRGB32(), resizemask) 
MMotionblur = Mask(soft.unfilter(-10,-10).ConvertToRGB32(), MoMask) 
Addedgedsharp = Layer(soft.fluxsmooth(3,5).ConvertToRGB32(), MEdgesharp, "add", sharpness) 
AddMotionblur = Layer(Addedgedsharp , MMotionblur, "add", 255).converttoyv12() 

return AddMotionblur 

} 

I'm quite happy with results (thanks mf, Kurosu and Manao !)

FuPP

[FuPP]

I made an avsi version, probably easier to use. Sharpness, sh1 and sh2 are now optional (defaults are 255,40,12)

Remember : avsi files are automatically loaded, so there's no need to load the function in your final script.

1/ avsi file :

the following lines should be copied and pasted in notepad, and then saved as "hybridfupp.avsi" in your avisynth plugins directory (by default : C:\Program Files\AviSynth 2.5\plugins) :

---------------------------------------------------------------------
function HybridFupp(clip input, int width, int height, int "sharpness", int "sh1", int "sh2")

{
sharpness = Default(sharpness, 255)
sh1 = Default(sh1, 40)
sh2 = Default(sh2, 12)

resizemask = input.edgemask(3,8,255,255,"sobel",Y=3, V=1, U=1).inflate().Levels(0, 1.0, 24, 0, 255).\
BilinearResize(width, height).GreyScale().ConvertToRGB32()

MoMask = motionmask(input,thY1=30,thY2=30,thSD=15,y=3,u=1,v=1).fity2uv().inflate().Levels(0, 1.0, 24, 0, 255).BilinearResize(width,height).greyscale().converttorgb32()

soft = input.BilinearResize(width, height)

MEdgesharp = Mask(input.LanczosResize(width, height).unfilter(sh1,sh2).ConvertToRGB32(), resizemask)
MMotionblur = Mask(soft.unfilter(-10,-10).ConvertToRGB32(), MoMask)
Addedgedsharp = Layer(soft.fluxsmooth(3,5).ConvertToRGB32(), MEdgesharp, "add", sharpness)
AddMotionblur = Layer(Addedgedsharp , MMotionblur, "add", 255).converttoyv12()

return AddMotionblur
}
--------------------------------------------------------------------



2/ script examples

ex 1 : crop, 448x320 resizing and addborders are used for svcd specification

LoadPlugin("C:\video\avsfilters\yv12\mpeg2dec3.dll")
LoadPlugin("C:\video\avsfilters\yv12\masktools.dll")
LoadPlugin("C:\video\avsfilters\yv12\unfilter.dll")
LoadPlugin("C:\video\avsfilters\yv12\fluxsmooth.dll")

mpeg2source("F:\Films\test.d2v",idct=7)
crop(24,80,672,416,align=true)
hybridFupp(448,320)
addBorders(16,128,16,128)


ex 2 : similar to previsous post, more optimized for compressibility

LoadPlugin("C:\video\avsfilters\yv12\mpeg2dec3.dll")
LoadPlugin("C:\video\avsfilters\yv12\undot.dll")
LoadPlugin("C:\video\avsfilters\yv12\masktools.dll")
LoadPlugin("C:\video\avsfilters\yv12\unfilter.dll")
LoadPlugin("C:\video\avsfilters\yv12\fluxsmooth.dll")

mpeg2source("F:\Films\test.d2v",idct=7,cpu2="ooxxox",moderate_h=10,moderate_v=20)
crop(24,80,672,416,align=true)
blindpp(quant=2,cpu2="ooooxo",moderate_h=30,moderate_v=50)
hybridFupp(448,320)
undot()
addBorders(16,128,16,128)


Just a last word : I have only used ideas (and code) coming mostly from other people, so I thank them once more !

[jkwarras]

Wow, thanks a lot for this script, I'll test it as sson as posible

If I've undertand correctly (please correct me if I'm wrong), it apply different levels of resizing/bluring/denoising depending on the motion of a frame. So, if for ex. we have an lateral inside car scene, with the driver in front and behind him there's the windows, with all the streets and cars, etc.. passing very quickly. It's that filter keeping the details of the non-moving area (the driver) while bluring the behind scene (moving area: 'paysage' in window)? I mean, applying different levels of denoising/filtering in different parts of the same frame?

If it's this it's incredible!!

One question:

How do you think this script will behave along with the QMF script? This last one is based in motion and apply different levels of denosing/filtering to a *whole* frame.

Regards.

[FuPP]

1/ Yep ! + edge shapening + non-edge softening (by bilinear resizing)

2/ This is the difference : QMF blurs the whole frame when there's motion. "Mine" blurs only moving parts of picture (so result should be, theoretically speaking, "better").

[Didée]

AAAAAAAArgh !!

Such a thing is what I wanted to script in the next two weeks or so, I already started with the baselines at end of January.



Okay, over the weekend I'll look closer at it, to find out if we had all the same ideas ...

However, it's very nice to see that people other than mf (and, very seldom, me) are converting good ideas into scripts. Thank you!

[Announcement] And BTW, I'm not completely out of the game: this weekend I'm going to post a little script ... eye candy, time dandy.


FuPP: technical question!

For the same purpose than you, I was playing with "MotionMask" also. But I had non-trivial problems with it:
When using scenechange thresholds low enough to catch the motion just like I wanted it to, THEN very often MotionMask returned black masks in high-motion parts, which is a very bad thing obviously (blurring drops out where you need it the most). IIRC I had to up the SC threshold to something >20~30, or even more (not sure atm). Now I see you use a SC thresh of only (15) ... don't you have problems with high-motion falsly detected as scenechange?

- Didée

[FuPP]

Yes, calibrating parameters is very touchy. The ones I use gave me good results for the sources I have tested. Maybe you could have a look with these and tell me what happens with samples you've used.

"Such a thing is what I wanted to script in the next two weeks or so, I already started with the baselines at end of January."

Sorry for that...

@Kurosu, Manao or Shodan :

I have a technical question too : Is it possible to have Mask function working with YV12 ?

btw, I have had problems using yv12Layer even if source (param1) and mask (param2) are previously converted to yv12. Is it normal ?

[jkwarras]

Originally posted by FuPP
This is the difference : QMF blurs the whole frame when there's motion. "Mine" blurs only moving parts of picture (so result should be, theoretically speaking, "better").

Cool! Thanks a lot for it, i'll try it It seems excellent for my DVD backups!

Well, I'm just guessing here and maybe this isn't of your interest but I think this script will also be really useful for DV filtering with no compressibility purposes. I mean, when you want to filter noise in DV and want to outut it to DV (not to MPEG-4). AS DV Data rate is constant you don't have to care about filesize, just filter efficiently.

As shooting DV is sometimes very difficult and you can get a lot of noise (for ex. on low lighting shooting) on certains scenes and less noisy (but still noisy) on other scenes. The fact is that if you apply denoising to the whole video (like C3D or fluxsmooth) you'll get suboptimal denoising, you can get too much denoising on parts than doesn't need it and get not enough denoising in parts that really need it (and you don't apply enough because you'll also apply too much into the less noisy parts). SO at the end you just have to do it at hand, which is kind of difficult. So, I'm wondering if such a script will be useful for this purposes, turning it into noise based. But maybe noise can be considered as motion. I'm not a programmer, just a final user and independent filmmaker and I really see an use of that, so please, forget about this if you feel is stupid

Regards

[Mug Funky]

hey, cool. i haven't touched video for a week or so, but i'll give this a try when i do.

if you're having trouble with mask and layer, why not use Overlay? fully YV12 masking and stuff... you need the jan 15th binary to get the full benefit (Photoshop style "ink types" like subtract, difference, etc)

[FuPP]

@jkwarras : you should probably use something like Peachsmoother for that purpose (excellent adaptive filtering by Linsey Dubb).

But I think DV sources are interlaced and I don't know if Peach handle this properly. Btw, you should find some posts in this forum about how filtering interlaced sources using filters that do no handle them natively.

Regards,

FuPP

[FuPP]

Originally posted by Mug Funky
hey, cool. i haven't touched video for a week or so, but i'll give this a try when i do.

if you're having trouble with mask and layer, why not use Overlay? fully YV12 masking and stuff... you need the jan 15th binary to get the full benefit (Photoshop style "ink types" like subtract, difference, etc)

yes, but what about masks ? (I guess overlay <=> layer ?)

[Manao]

@FuPP : no need of Mask if your are in YV12 : the Y channel already contains that information.

In fact,

mask = Mask(overlay_clip,mask_clip)
Layer(base_clip, mask, "add", 255)

Is equivalent to

MaskedMerge(base_clip,overlay_clip,mask_clip,Y=3, U=3, V=3)

Hence, your code can be changed into

function HybridFupp(clip input, int width, int height, int "sharpness", int "sh1", int "sh2") 

{
sharpness = Default(sharpness, 255)
sh1 = Default(sh1, 40) 
sh2 = Default(sh2, 12)

resizemask = input.edgemask(3,8,255,255,"sobel",Y=3, V=1, U=1).inflate().\
Levels(0, 1.0, 24, 0, 255).Levels(0,1.0,255,0,sharpness).BilinearResize(width, height)

MoMask = motionmask(input,thY1=30,thY2=30,thSD=15,y=3,u=1,v=1).fity2uv().\
inflate().Levels(0, 1.0, 24, 0, 255).BilinearResize(width,height)

soft = input.BilinearResize(width, height) 

MEdgesharp = input.LanczosResize(width, height).unfilter(sh1,sh2)
MMotionblur = soft.unfilter(-10,-10)

Addedgedsharp = MaskedMerge(soft.fluxsmooth(3,5), MEdgesharp, resizemask) 
AddMotionblur = MaskedMerge(Addedgedsharp , MMotionblur, MoMask )

return AddMotionblur 
}

Note that the sharpness is used inside a Levels

What exactly is the issue with YV12Layer ?

@Mug Funky : Overlay accept YV12 as input, but AFAIK, it converts it to RGB before treating it.

Edited : small typo, plus lines too long.

[jkwarras]

Originally posted by FuPP
@jkwarras : you should probably use something like Peachsmoother for that purpose (excellent adaptive filtering by Linsey Dubb).

I've never tried Peachsmoother, I didn't know it's adaptive filtering (good news), I was using C3D, fluxsmooth and recently Mipsmooth. I'll take a look at this one, thanks for the tip.

Regards

[Wilbert]

@Mug Funky : Overlay accept YV12 as input, but AFAIK, it converts it to RGB before treating it.]a

No, it converts the clips (including the mask) to YUV 4:4:4. Btw, layer doesn't work correctly in YV12.

[bond]

hm i dont get it to work

i tried both fupps and manaos version, fupps version reports an unrecognized exception in the "MoMask = motionmask..." line, manaos one line below "inflate()..."
(btw why does manao use \ in the MoMask command but fupp doesnt?)

[Manao]

Bond, you have to update your version of the masktools : http://www.geocities.com/manao47/Fil...ols-v1.4.9.zip

The \ is used in order to make a line break in the series of instructions, because my line was too long to fit the screen.

[bond]

thanks manao, i already had this great plugin

hm, maybe this is already a known issue (i have to admit i didnt search for it) but it seems the problem is caused by crop (if i comment crop() out it works)

i have a pal dvd source:
crop(2,80,716,418)
HybridFupp(640,256)

edit:
funny, with crop(0,78,720,420) or crop(2,82,716,416) it works

[Manao]

Your resolution is only mod 2 horizontally. It will work with mod4 resolutions. However, that was not an expected bug, thanks for reporting it. A new version should be released during the afternoon, without the bug ( if I find it ).

[FuPP]

Hi Manao ! Thanks a lot for your tips !

Originally posted by Manao
What exactly is the issue with YV12Layer ?

Here is a simple example (input is YV12) :

function HybridTest(clip input, int width, int height, int "sharpness", int "sh1", int "sh2")

{

Resizemask = input.edgemask(3,8,255,255,"sobel",Y=3, V=1, U=1).Inflate().Levels(0, 1.0, 24, 0, 255).\
BilinearResize(width, height).ConvertToRGB32()

MEdgesharp = Mask(input.LanczosResize(width, height).ConvertToRGB32(), Resizemask).Converttoyv12()

Resized_input = Input.Bilinearresize(width,height)

Addedgedsharp = yv12Layer(Resized_input, MEdgesharp, "add", sharpness)

return Addedgedsharp

}

This returns me an error : YV12Layer : Image formats don't match


Regards,
FuPP

[bond]

funny when i color the motion parts red (with .ConvertToRGB32().RGBAdjust(5,1,1,1) ) i see an effect i could describe as ghosting, meaning if a person moves also parts get blurred where the person was a frame before altough it isnt there anymore...
why that?


FuPP,
can you plz tell me why you also choose to add level(), inflate(), greyscale() aso and not only used edge/motionmask and a resizefilter?

thanks

[morsa]

Any before and after images to see its effect?