Could i improve this script? - Efficency wise

[Graigddu]

I've been using this script for testing on video i encoded to Xvid from originals but iv'e read here people say that deblocking
should be done after resizing but i always get errors about Mod16 if i resize after,

This script seems to give a good balance of speed to quality especially as iv'e been trying lossless HCEnc encoding also and since i added dup fluctuating pixels in walls and sky etc seems to be locked for lack of a better word,

some of the video is live action some animation,

I have also tried Deblock_QED but FunkyDeblock seemed to work better with this script anyway

SetMTmode(mode=5,threads=8)
SetMemoryMax(256)
# 16:9 encoding
LoadPlugin("C:\AviSynth 2.5\plugins\MT Filter\MT.dll")
LoadPlugin("C:\AviSynth 2.5\plugins\MaskTools-v1.5.8\MaskTools.dll") #version 1.58
LoadPlugin("C:\AviSynth 2.5\plugins\Dup\dup.dll")
LoadPlugin("C:\AviSynth 2.5\plugins\FluxSmooth\FluxSmooth.dll")
Import("C:\AviSynth 2.5\Functions\funkydeblock.avs")
DirectShowSource("C:\Video\Editing\Test Copy.avi", fps=23.976, audio=false, convertfps=true)
SetMTmode(mode=2,threads=8)
ConvertToYV12()
FadeIn(50)
Lanczos4Resize(720,436,0.0,0.6)
AddBorders(0,70,0,70)
AssumeFPS(25,1,False)
SetMTmode(mode=5,threads=8)
MT("""
funkydeblock(th=5)
dup(threshold=3.0,blend=true,copy=true,blksize=16)
TemporalSoften(4,4,8,15,2)
FluxSmoothST(5,7)
MSharpen(threshold=10,strength=100,highq=true)
""", 2)

[Graigddu]

Any Ideas ?

[Guest]

What do you mean by "efficiency wise"?

[steptoe]

I believe the basis of that script is from Avi2ISO, as I have played around with it myself and finally found an AVI to DVD solution that actually accepts avisynth scripts and functions without falling over and giving a script error, like all the others I have tried do

I have found one snag with it though, look at the source its saying its 23.976 FPS, but its being converted to 25 FPS so you'll get an incorrect FPS. I may be wrong, but I change that to whatever the source should be

So (if I am not wrong) should read AssumeFPS(23.976,1,False) but then again the FALSE statement might actually mean do NOT convert to 25 FPS, so in that case I am wrong

I reckon you can remove these statements without affecting it much, as I think the addborders is there to keep the aspect ratio in spec, but its such a tiny amount I don't think you would notice

FadeIn(50)
AddBorders(0,70,0,70)


Also, try removing all your 'additional' code at the end and have a look at MC_Spuds() function, as that now has deblocking, sharpening and smoothing as options along with a specific anime option to enable code specifically for anime which is what I assume you are working on converting back to DVD

http://avisynth.org/mediawiki/MC_Spuds
http://forum.doom9.org/showthread.php?t=131279&highlight=spuds

So you should end up with one function doing everything, instead of your filter chain having 5 filters, plus MC_Spuds() can remove any noise, grain and plenty of others

[IanB]

Do the FadeIn(50) at the end, using it with dup might cause strobing during slow fades.

You should move the AddBorders(0,70,0,70) to the end, there is no point in processing black borders with expensive filters unnecessarily.

Are you sure you really need all this processing, you may be suffering from filteritis, see your GP :pfunkydeblock(th=5)
dup(threshold=3.0,blend=true,copy=true,blksize=16)
TemporalSoften(4,4,8,15,2)
FluxSmoothST(5,7)
MSharpen(threshold=10,strength=100,highq=true)The deblock should probably be done on the original unresized source and dup might as well also be done on the smaller image (assuming you are upsizing).

[Graigddu]

Thanks for the answers much obliged,

What do you mean by "efficiency wise"?

I was just wondering if there was any filters that should be switched around for better usage or could be swopped for another filter with better stability,

Can't say that iv'e ever come across Avi2ISO,not that i remember anyway

Have to be honest the script is one that iv'e hashed together from others that iv'e seen in doom9 and other forums
and started out as a avisynth script used by The FilmMachine GUI that is where the FadeIn(50)
AddBorders(0,70,0,70) came from,although the GUI's was the basic resize and YV12 convert before hand,

The script is the end product of numerious tests with different filters some of which fell over and were discarded.

I did try FunkyDeblock earlier in the script but got a Mod16
error if used before resizing and AddBorders,and was unsure of where it would work then hence running it after resize,

Dup and TemporalSoften seemed to work really well with locking down moving pixels in walls etc,and i tried fluxsmooth
to see if it would smooth it a little,

[Sagekilla]

So basically you're using some scripts without even knowing what they're supposed to be doing? Do you have any real use for FadeIn()? Because it seems like since, if you've copied pasted from FilmMachine, perhaps not.

[Graigddu]

So basically you're using some scripts without even knowing what they're supposed to be doing?

Sorry but your wrong there,before i used the filters in question i did quite a bit of searching in this forum and elsewhere as to what the filters did and whether the filters were of any use to me i didn't just add any old filter i saw into a script if that's what your suggesting,

i wanted a deblocker as there was blockiness in dark scenes and funkydeblock worked best with my source, also dup was mentioned in several posts as have good noise reduction and did improved fluctuating pixels in walls etc,temporalsoften was mentioned as being able to blend noisy pixels,

and so forth.

I even did quite a bit of searching into the SetMTMode and MT filter as to which way to use and if stable with various filters

[Sagekilla]

The reason I ask is because Dup() is usually meant for duplicate detection.. not so much "noise reduction." FFT3DFilter() and MVDegrain() work well for denoising, and MVDegrain tends to keep the video extremely sharp while removing temporal fluctuations.

Also, Dup() inside of MT() seems a bit odd, because MT() splits a frame in half and that can mean the difference between finding a dupe or not.

Personally, I'd recommend posting a sample of your clip so we can help tailor your script.


Edit: What is your source video resolution? You'll get mod16 errors if the video source is not mod16. If it's something like 716x532 for example you can pad to mod16 (AddBorders(2,6,2,6)) deblock, then remove the padding (Crop(2,6,-2,-6))

[IanB]

AddBorders(2,6,2,6)) deblock, then remove the padding (Crop(2,6,-2,-6))Hell NO! Add all the extra padding only on the right side. Adding non 16 byte multiples to the left side means the internal frame copy is unaligned hence slower. More importantly you offset the blocks the filter is looking for. Deblocking data from DCT based codec one assumes some power of 2 (4, 8, 16, ...) for the block patterns....
W=Width()
H=Height()
AddBorders(0,0, (16-(W%16))%16, (16-(H%16))%16)
# Mod 16 restricted filters
...
...
Crop(0, 0, W, H)
...


Of course if some other processing step has offset the blocking pattern then you may have to add some border to the left side to alleviate the problem.

[Sagekilla]

Apologies, I don't know the internals of Avisynth ;) I know as much that it'd be wise to add borders for mod16 processing, but never knew you shouldn't use left and top.

[Guest]

The reason I ask is because Dup() is usually meant for duplicate detection.. not so much "noise reduction." I beg to differ. It was designed originally for suppressing small noise differences between duplicate frames to improve compressibility. As such, its main application is as a specialized temporal denoiser.

[Graigddu]

If i remember rightly Neuron2 i had asked you before about Dup's noise filtering after reading some posts here and that's
why i used Dup which has worked well on fluctuating pixels in dark areas by the way :thanks:

[Sagekilla]

Aha, I learn two new facts today :) Thanks for that, I'll have to try this out sometime.

[Graigddu]

The reason I ask is because Dup() is usually meant for duplicate detection.. not so much "noise reduction." FFT3DFilter() and MVDegrain() work well for denoising, and MVDegrain tends to keep the video extremely sharp while removing temporal fluctuations.

Edit: What is your source video resolution? You'll get mod16 errors if the video source is not mod16. If it's something like 716x532 for example you can pad to mod16 (AddBorders(2,6,2,6)) deblock, then remove the padding (Crop(2,6,-2,-6))

I'm pretty sure SageKilla that i'd had a look mvdegrain and was totally lost as to how to initiate mvdegrain 2,i usually use my scripts in AvsP so i can see the outcome i had already tried FFT3DFilter() and hadn't found it to my liking as it was quite slow especially as the puter i have is the only one i have and is used by all my family,probably also because i was using the wrong settings also?

Thanks for the heads up about pad to mod16 with FunkyDeblock i tried this with my source and it worked a treat in SetMTMode (mode=2,threads=12) Much Appreciated :)
although i did have to pad slightly by 2 to the left as somewhere in my script it caused a green right-hand edge to appear otherwise,

[Sagekilla]

If you need help with MVDegrain, it's done like so with the "vanilla" (official) builds:

source = last
bvec2 = source.MVAnalyse(isb=true delta=2, overlap=4, pel=2, idx=1)
bvec1 = source.MVAnalyse(isb=true, delta=1, overlap=4, pel=2, idx=1)
fvec1 = source.MVAnalyse(isb=true delta=1, overlap=4, pel=2, idx=1)
fvec2 = source.MVAnalyse(isb=true, delta=2, overlap=4, pel=2, idx=1)
source.MVDegrain2(bvec1,fvec1,bvec2,fvec2,idx=1)

Or if you use josey_well's multithreaded build (meaning you don't need MT() or SetMTMode with it):

source = last
vectors = source.MVAnalyseMulti(refframes=2, overlap=4, pel=2, idx=1)
source.MVDegrainMulti(refframes=2, idx=1)


You can also make a function called MVDegrain and stick it in your avisynth folder as MVDegrain.avsi:


function MVDegrain( clip source, int "delta", int "blksize", int "overlap", int "pel", int "idx" )
{
delta = default(delta, 2)
blksize = default(blksize, 8)
overlap = default(overlap, blksize/2)
pel = default(pel, 2)
idx = default(idx, 1)

b3vec = (delta==3) ? source.MVAnalyse(isb=true,delta=3,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()
b2vec = (delta>=2) ? source.MVAnalyse(isb=true,delta=2,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()
b1vec = (delta>=1) ? source.MVAnalyse(isb=true,delta=1,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()
f1vec = (delta>=1) ? source.MVAnalyse(isb=false,delta=1,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()
f2vec = (delta>=2) ? source.MVAnalyse(isb=false,delta=2,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()
f3vec = (delta>=3) ? source.MVAnalyse(isb=false,delta=3,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()

denoise = (delta==3) ? source.MVDegrain3(b1vec,f1vec,b2vec,f2vec,b3vec,f3vec,idx=idx) :
\ (delta==2) ? source.MVDegrain2(b1vec,f1vec,b2vec,f2vec,idx=idx) :
\ (delta==1) ? source.MVDegrain1(b1vec,f1vec,idx=idx) : NOP()

return(denoise)
}

[Gavino]

You can also make a function called MVDegrain and stick it in your avisynth folder as MVDegrain.avsi:

function MVDegrain( clip source, int "delta", int "blksize", int "overlap", int "pel" ) {
...
b3vec = (delta==3) ? source.MVAnalyse(isb=true,delta=3,blksize=blksize,overlap=overlap,pel=pel,idx=1) : NOP()
...

To make the function reusable, I would add idx as a parameter too:function MVDegrain( clip source, int "delta", int "blksize", int "overlap", int "pel", int "idx" ) {
idx = default(idx, 1)
...
b3vec = (delta==3) ? source.MVAnalyse(isb=true,delta=3,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()
... # and similarly using idx=idx

[Sagekilla]

That's a bit unnecessary though really.. I don't see any realistic situations where you'd be calling it twice in the same filterchain. If you were, there's not much to gain after the first anyway. If you wanted to do that though, you're better off using MVAnalyseMulti + MVDegrainMulti to use more than 3 frames.


Edit: Added in case anyone REALLY wants to call MVDegrain twice..

[Gavino]

That's a bit unnecessary though really.. I don't see any realistic situations where you'd be calling it twice in the same filterchain. If you were, there's not much to gain after the first anyway.
Unlikely you would want to call it twice on the same clip, but possibly on two independent clips in a more complex script.

But if you're going to the bother of defining a function, why not make it bulletproof, especially if the extra effort involved is minimal. To my mind, a 'function' that can only be called once is deficient and should come with a health warning.

[Didée]

To my mind, a 'function' that can only be called once is deficient and should come with a health warning. Very true. (Although my own scripts often don't respect that issue ... my solution is to post scripts that draw so much ressources that you [don't want to]/[can not] use it more than once, anyway :) )

But, your example does not make the function bulletproof. It offers the possibility of using it mor than once, but not automagically. Constructs like clip1.MVDegrain() + clip2.MVDegrain() still do fail. (could be rearranged to (c1+c2).function(), sure ... it's just a quick example for arguing.)

For more bulletproof-ness, you'd need to establish a global counter, like

global idx_counter = 1 # initiate this value when the function is imported

function MVDegrain( clip source, int "delta", int "blksize", int "overlap", int "pel", int "idx" )
{
idx = default(idx, idx_counter + 1) # idx specified?-use it. Not specified?-proceed with global counter.
global idx_counter = idx # update global counter
...
b3vec = (delta==3) ? source.MVAnalyse(isb=true,delta=3,blksize=blksize,overlap=overlap,pel=pel,idx=idx) : NOP()
... # and similarly using idx=idx

It's not fully bulletproof, you can still make it break with certain mixes of function calls with and without specifying 'idx'. But at least it works when the user don't [want to] care about 'idx' at all.

[Gavino]

my own scripts often don't respect that issue ... my solution is to post scripts that draw so much ressources that you [don't want to]/[can not] use it more than once, anyway :)
Yes, I have to say I've noticed that. ;)
In your case, there is another factor at play - the functions are so good that it seems ungrateful to complain about only being able to use them once!
For more bulletproof-ness, you'd need to establish a global counter ...
Yeah, I thought about that but wanted to keep it simple here. The idea was just to give the user at least the possibility to specify a different idx for further calls. Your solution goes further and allows the user to (more or less) forget about idx altogether.

[Sagekilla]

@Gavino, if you're doing such a complex script, you should really be custom tailoring those calls for the complex scripts ;) I made the function so that it's easier for Graigddu to use MVDegrain.

[Fizick]

Didee,
1. global in *.avsi is a time bomb IMHO ;)

2. I hope I can free some resources for you and your more complex scripts in next MVTools version (beta is ready) ;)