Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 31st January 2022, 00:46   #781  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
What do you mean without n dependencies? GreyWorld() only requires ExTools, RgTools (might drop it) and TransformsPack.
In any case I have already the idea for SC detection that will enable scene based filtering. This will actually turn things around workflow-wise, so probably it will require adapting some of my high-level filters(?) sigh!

The current problem with GreyWorld, aside of temporal stability is that I didn't manage to align the gamut assymetry so I have to look into that a bit further. Also I probably should look for color cast/tint only on the 10% brightest percentile to have a more accurate reading (after all it's called WHITE point), but I need to test.

As for stabilization, it's a bit of a rabbit hole which I tend to avoid (like the MVTools' Zopti optimizations) so I don't get stuck in things for too long. The thing with reusing motion vectors is that you probably don't want the jittery MV for degraining, it will mostly imply creating new vectors.

I just ported latest LSFmod to EX mod, ex_limitdif() (VERY optimized) and deep_resize() (initial version but already faster (x8) and higher quality than nnedi3_resize16() ). Will upload tomorrow.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 31st January 2022, 18:19   #782  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
Here are a few examples of deep_resize() compared to nnedi3_resize(). It's sharper while being over 4 times faster.
It's not a straight port since I added a few ideas of my own, to start with the flat resizer is now SincResize. Also the edge pass with nnedi3 which is naturally on the soft side is sharpened with LSFmod (very slightly), it's fast because of my last update. With this I noticed a big contrast change in the image, it wasn't because the sharpener but because nnedi3 and the support resizers. Well this contrast just happens to be low frequency "sharpening" akin to what you can do with ex_unsharp(), can't I undo it? Yes, I did a very small tweak to ex_unsharp() and now you can deconvolute, it gives a less coarse image while keeping high frequency sharpness. I will be uploading in the next few hours.

raw (sorry @Tormento for beating this sample lol)

spline36 (from 'Zopti' downscaled 720p)

nnedi3_resize16

deep_resize

deep_resize v2


spline36 (sample from FranceBB)

nnedi3_resize16

deep_resize

deep_resize v2
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 8th February 2022 at 04:50.
Dogway is offline   Reply With Quote
Old 1st February 2022, 12:02   #783  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 919
Hi Dogway, I've hit a bit of a "wall", and I don't know when what causes my problems, happened.

I have been "religiously" updating my scripts, whenever you release new ones (that I use), and I have been testing them to see if they at least pass a AVSMeter "test", but haven't been encoding with them until recently, and to my despair, I am getting errors when using certain scripts, mainly when using BM3D.

I know that SMDegrain is pretty complex, and maybe I'm missing something, especially when it comes to get BM3D to use CPU or CUDA commands.

The error is :-
Code:
 y4m [error]: bad sequence header magic
x264 [error]: could not open input file `-'
I have noticed in the past that this has been associated with an OpenCL problem, BUT the KNLMeansCL scripts DO work.

I have been backdating my scripts to see if I can find a point where the error doesn't occur, with no luck, yet.

I'm not sure if there's been a change with BM3D, or some of your updates have affected how BM3D is used, maybe there's more or different dependencies for recent builds.

So any suggestions would be greatly appreciated.
__________________
Long term RipBot264 user.

RB 1.27.1 (std)
RB 1.27.1 (fully loaded, OOD)
PD 7.6 (new)
TDS is offline   Reply With Quote
Old 1st February 2022, 12:30   #784  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,708
It would be much easier to debug if you open the script in VirtualDub, or any other tool that would give the exact error message and hopefully the position as well.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 1st February 2022, 12:39   #785  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
@TDS, with yesterday updates all the packs are now in sync and should work without issues. Normally errors will come from bugs from AVS+ x86, the plugins or the client (AVSmeter, AvsPmod...), and occasionally my fault. Just attack one at a time.
To be honest when testing BM3D a week ago I saw artifacts I didn't like, I'm not sure if it was always the case but it seemed to denoise only vertically. With that said I don't get that error, try to open in another client software to reproduce the issue. Also that hyphen error message(?) do you have strange unicode characters in your x246 output path or filename?

I will try to have a closer look later today and prepare a simple BM3D call for you to test.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 1st February 2022, 13:08   #786  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 919
@ Boulder & Dogway, thanks for your prompt replies & suggestions...I need to remind you that I use RipBot264, and it's taken me quite a while (with help from this Forum) to incorporate these scripts into the app, and have them work successfully....I'm pretty sure I had it all running sweet several weeks ago (haven't done much for a long time, except update my scripts), so like I said earlier, I ONLY test them within RipBot264, and it uses AVSMeter to "check" the scripts, and it a little vague in "pinpointing" the actual error

RipBot264 uses a Distributed Encoding feature, and my problem is mainly showing up on the "remote" encoding PC's, not on the Main PC, but it's not on ALL of them, despite them all being set up the same, and there's where it becomes even more frustrating.

It's quite a time consuming process, and I'm sure you're more than aware.

Thanks
__________________
Long term RipBot264 user.

RB 1.27.1 (std)
RB 1.27.1 (fully loaded, OOD)
PD 7.6 (new)
TDS is offline   Reply With Quote
Old 1st February 2022, 13:47   #787  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,308
Quote:
Originally Posted by Dogway View Post
anton_foy was having issues before with x86. I think there's something odd in x86 AVS+ but pinterf has to confirm, simply by testing the above Expr() call.



Ok, so 'scale_inputs' is broken on x86, I will try to ask pinterf when he's around.
Pls report bugs, I accidentally found this, so yes, there was a bug (an unaligned internal buffer) which affected LUT operation on AVX2 machines. I'm gonna make a test build later.
pinterf is offline   Reply With Quote
Old 1st February 2022, 14:03   #788  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
Yes sorry had a few busy days. Aside from scale_inputs there might also an issue with 'lut' arg in Expr (ah yes what you said) and frameprops calls outside runtime environment but I can't tell because I don't use AVS x86, this is all from what users report to me. In x64 everything seems to work fine on my side.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 1st February 2022 at 14:06.
Dogway is offline   Reply With Quote
Old 2nd February 2022, 19:46   #789  |  Link
Fjord
Registered User
 
Join Date: Dec 2005
Location: Denmark
Posts: 52
What does EX and MIX mean? Such as for LSFmod?

@Dogway, I'm very appreciative of your work on optimizing plugins with ExTools etc. And thanks also to all the other great contributors here!
I need some help understanding the mod versions. "EX" mods seems obvious, but what is a "MIX" mod? And why are there both EX and MIX versions of LSFmod? I don't know which I should be downloading. Or are they the same?
Fjord is offline   Reply With Quote
Old 2nd February 2022, 20:10   #790  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
EX mods is kind of what I would like it to be, everything run from internal functions, low dependencies.
MIX mods in turn has more dependencies as it uses RgTools, masktools2, etc

The reason I had to make MIX mods is because some operations are slower with internal code like Expr() pixel addressing being only SSSE3 accelerated vs masktools2 which uses AVX2.
ex_merge() is also generally slower than mt_merge(), and removegrain(12) or removegrain(4) are insanely faster than than ex_boxblur(mode="weighted") or ex_median("median") respectively.

In general terms MIX mods are slightly faster but output should be the same.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 2nd February 2022, 21:31   #791  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,967
Maybe a Doggy glossary is in order.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 2nd February 2022, 21:54   #792  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
hahaha and I thought it was clear in OP.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 4th February 2022, 21:34   #793  |  Link
gispos
Registered User
 
Join Date: Oct 2018
Location: Germany
Posts: 983
I like to use this function (found here somewhere), but it's not the fastest. Does anyone see potential to make it faster or improve it. HBD > Dogway?
Code:
function Highlighter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", int "method")
{
gradient = default (gradient,true)
gblur = (gradient==true) ? default (gblur,100) : default (gblur,5.0) 
threshold = default (threshold,100) #The lower the value, the more sensitive the filter will be.
twopass = default (twopass,false)
amount = default (amount,10)
method = default (method, 1) #0,1,2

amount = (amount<0) ? abs(amount) : amount

lighten= v.Tweak(sat=0).mt_lut("x "+string(threshold)+" > 0 x ?")
blurred= (gradient==true) ? lighten.gaussianblur(gblur).gaussianblur(gblur+100).gaussianblur(gblur+200) : lighten.gaussianblur(gblur)


photoshop_overlay = (method==1) ? mt_lutxy(v,blurred.invert(),"x 127.5 > y 255 x - 127.5 / * x 255 x - - + y x 127.5 / * ? ") : \
                    (method==2) ? mt_lutxy(v,blurred.invert(),"255 255 x - 255 y - * 255 / -") : v.Tweak(bright=amount)

photoshop_overlay = (twopass==true) ? photoshop_overlay.SmoothLevels(gamma=2.0, Lmode=2) : photoshop_overlay
return mt_merge(v,photoshop_overlay,blurred)
}
__________________
Live and let live
gispos is offline   Reply With Quote
Old 4th February 2022, 21:35   #794  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
Yes, big potential, give me some minutes.

EDIT: Here it is. By the way, what is it for?

Code:
function Highlighter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", int "method") {

    bi  = BitsPerComponent(v)
    fs  = propNumElements (v,"_ColorRange")  > 0 ? \
          propGetInt      (v,"_ColorRange") == 0 : false

    grad    = Default (gradient,true)
    gblur   = Default (gblur, grad ? 100 : 5)
    thres   = Default (threshold,100)          #The lower the value, the more sensitive the filter will be.
    twopass = Default (twopass,false)
    amount  = Default (amount,10)
    method  = Default (method, 1) # 0, 1 or 2

    amount  = ex_bs(abs(amount), 8, bi, true)
    thres   = ex_bs(thres,       8, bi, true)

    lighten = v.ex_lut(Format("x {thres} > 0 x ?"), UV=128)
    blurred = grad ? lighten.ex_gaussianblur(0.073*gblur+17.78,UV=1) : \
                     lighten.ex_gaussianblur(0.064*gblur+ 3.75,UV=1)


    photoshop_overlay = (method==1) ? ex_lutxy(v,blurred,"x range_half > range_max y - Y@ range_max x - X@ range_half / * range_max + Y x range_half / * ? ",  fulls=fs) : \
                        (method==2) ? ex_lutxy(v,blurred,"range_max range_max x - y * range_max / -",fulls=fs) : v.ex_lut(Format("x {amount} + ")+(!fs?"ymax min":""), fulls=fs)

    photoshop_overlay = twopass ? photoshop_overlay.ex_contrast(-0.8,1,167,tv_range=!fs) : photoshop_overlay
    return mt_merge(v,photoshop_overlay,blurred) }
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 5th February 2022 at 17:20.
Dogway is offline   Reply With Quote
Old 5th February 2022, 09:38   #795  |  Link
gispos
Registered User
 
Join Date: Oct 2018
Location: Germany
Posts: 983
Quote:
Originally Posted by Dogway View Post
Yes, big potential, give me some minutes.

EDIT: Here it is. By the way, what is it for?
To brighten up dark scenes (Maybe wrongly worded, 'dark areas' is probably more correct).
Will test your changes with a 'Preview Filter'.
Thank you!
__________________
Live and let live

Last edited by gispos; 5th February 2022 at 09:49. Reason: wrongly worded
gispos is offline   Reply With Quote
Old 5th February 2022, 11:13   #796  |  Link
gispos
Registered User
 
Join Date: Oct 2018
Location: Germany
Posts: 983
What am I missing?
Code:
Script error: There is no function named 'nmod'.
(D:\Tools\AviSynth\plugins64\DogwayFilters\ExTools.avsi, line 1691)
Found:
Located in ResizerPack.avsi
Hm, wouldn't it make more sense to put it in ExTools.avsi.

I don't really want to have to load all your packs... that's too much 'resources' for me if I only want to use one function 'ex_Highlighter'

Now I get:
Code:
Expr: Failed to convert '{amount}' to float
(D:\Tools\AviSynth\plugins64\DogwayFilters\ExTools.avsi, line 67)
__________________
Live and let live

Last edited by gispos; 5th February 2022 at 11:22.
gispos is offline   Reply With Quote
Old 5th February 2022, 17:36   #797  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,351
Yes sorry, I forgot to add Format() to the expression (edited), I got a phone call in between and got distracted.
Don't be like that, it's only 5 packs, I really don't like scripts that are like a junkyard where you put everything in it with no correlation.
ExTools is only for core filters, low level or build block functions. I only try to be conservative at big names like SMDegrain or QTGMC.
Give the functions a try, mmod(), PadBorders(), deep_resize(), you will surely get in the loop and won't come back :P
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 7th February 2022, 20:19   #798  |  Link
gispos
Registered User
 
Join Date: Oct 2018
Location: Germany
Posts: 983
Hi Dogway,
the 'Highlighter' filter is now called 'ex_Highlighter' and is 6 - 8 times faster with almost identical results. So full success.

And therefore (if you feel like it) you may try the opposite function. 'HighlightLimiter' and 'HighlightLimiterMod'
The filter reduces too bright areas. The mod version was changed by me (long time ago).
Code:
function HighlightLimiter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", int "method")
{
gradient = default (gradient,true) #True uses the gaussian blur to such an extent so as to create an effect similar to a gradient mask being applied to every area that exceeds our threshold.
gblur = (gradient==true) ? default (gblur,100) : default (gblur,5.0) #The strength of the gaussian blur to apply. 
threshold = default (threshold, 160) #The lower the value, the more sensitive the filter will be.
twopass = default (twopass, false) #Two passes means the area in question gets darkened twice. 
amount = default (amount, 20) #The amount of brightness to be reduced, only applied to method=2
method = default (method, 0) #0,1 Method 0 is multiply, the classic HDR-way. Any other method set triggers a brightness/gamma approach.

amount = (amount>0) ? -amount : amount

darken=v.Tweak(sat=0).mt_lut("x "+string(threshold)+" < 0 x ?")
blurred= (gradient==true) ? darken.gaussianblur(gblur).gaussianblur(gblur+100).gaussianblur(gblur+200) : darken.gaussianblur(gblur)
multiply = (method==0) ? mt_lut(v,"x x * 255 /") : v.Tweak(bright=amount)
multiply = (method==0) ? eval("""
(twopass==true) ? mt_lutxy(multiply,v,"x y * 255 /") : multiply""") : eval("""
(twopass==true) ? multiply.SmoothLevels(gamma=0.9, Lmode=2) : multiply""")

return mt_merge(v,multiply,blurred)
}

function HighlightLimiterMod(clip clp, Int "threshold", Int "dark", Int "rad", int "mode", float "gamma1_2"){
threshold=default(threshold, 160)
dark=default(dark, 255) 
rad=default(rad, 64)
mode=default(mode, 0) # 0,1,2
gamma1_2=default(gamma1_2, 0.6) # for mode 1,2

m = clp.Width() >= clp.Height() ? clp.Height() : clp.Width()
rad = round(m/rad/4) < 1 ? max(round(m/4.0),16): rad
darken=clp.Tweak(sat=0).mt_lut("x "+string(threshold)+" < 0 x ?")
blurred=darken.bicubicresize(round(clp.width()/rad/4)*4, round(clp.height()/rad/4)*4).bicubicresize(clp.width(),clp.height(),1,0)
multiply = (mode==0) ? mt_lut(clp,"x x * " + string(dark) + " /") : (mode==1) ? SmoothLevels(clp, gamma=gamma1_2) : mt_lut(clp,"x x * " + string(dark) + " /").SmoothLevels(gamma=gamma1_2)
return mt_merge(clp,multiply,blurred)
}
Thanks already
__________________
Live and let live
gispos is offline   Reply With Quote
Old 8th February 2022, 01:41   #799  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 919
Hey Dogway, and the updates just keep on coming...

So I just noticed in the latest iteration of SMDegrain, that KNLMeansCL (pre filter=5) has "disappeared" !!!!

In it's place, has appeared DGDecNV !!!! How does that work.

Can "we" have KNLMeansCL back ???

I haven't done anymore testing BM3D since my post a few days ago (been busy with other things), I will post my script & call's soon, and I would appreciate your help with that.

Have the dependencies for BM3D changed recently ??
__________________
Long term RipBot264 user.

RB 1.27.1 (std)
RB 1.27.1 (fully loaded, OOD)
PD 7.6 (new)
TDS is offline   Reply With Quote
Old 8th February 2022, 04:36   #800  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,134
You can try TemporalDegrain2 (postFFT=4), that option have KNLMEANSCL
kedautinh12 is offline   Reply With Quote
Reply

Tags
avisynth, dogway, filters, hbd, packs

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:20.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.