Vapoursynth

[Pat357]

Thanks to all who contributed to finding and solving my crash problem with fmtconv. I could not use something like mawen1250's "nnedi3_resample.py" script because of the frequent crashes.
This script is also used in HAvsFunc.
For somewhat around 6 months I have been pulling my hair because of the frequent/random crashes with such scripts.
I've always thougt it must be my system because no-one else seemed to have this problem -;)

With the recompiled version, all crashes all gone !!

I've tried to recompile fmtc myself on MINGW64/MSYS2, but I got an error in Interlocked.hpp :

config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands
CXX ../../src/main.lo
CXX ../../src/fmtc/Bitdepth.lo
In file included from ./../../src/conc/Interlocked.h:132:0,
from ./../../src/conc/AtomicPtr.hpp:27,
from ./../../src/conc/AtomicPtr.h:117,
from ./../../src/conc/CellPool.h:30,
from ./../../src/conc/ObjPool.h:44,
from ../../src/fmtc/Bitdepth.h:30,
from ../../src/main.cpp:18:
./../../src/conc/Interlocked.hpp: In static member function 'static void conc::Interlocked::cas(conc::Interlocked::Data128&, volatile Data128&, const Data128&, const Data128&)':
./../../src/conc/Interlocked.hpp:349:2: error: '::InterlockedCompareExchange128' has not been declared
::InterlockedCompareExchange128 (
^~
In file included from ./../../src/conc/Interlocked.h:132:0,
from ./../../src/conc/AtomicPtr.hpp:27,
from ./../../src/conc/AtomicPtr.h:117,
from ./../../src/conc/CellPool.h:30,
from ./../../src/conc/ObjPool.h:44,
from ./../../src/fmtc/Bitdepth.h:30,
from ../../src/fmtc/Bitdepth.cpp:28:
./../../src/conc/Interlocked.hpp: In static member function 'static void conc::Interlocked::cas(conc::Interlocked::Data128&, volatile Data128&, const Data128&, const Data128&)':
./../../src/conc/Interlocked.hpp:349:2: error: '::InterlockedCompareExchange128' has not been declared
::InterlockedCompareExchange128 (
^~
./../../src/conc/Interlocked.hpp:351:4: error: 'excg_hi' was not declared in this scope
excg_hi,
^~~~~~~
./../../src/conc/Interlocked.hpp:351:4: error: 'excg_hi' was not declared in this scope
excg_hi,
^~~~~~~
./../../src/conc/Interlocked.hpp:352:4: error: 'excg_lo' was not declared in this scope
excg_lo,
^~~~~~~
./../../src/conc/Interlocked.hpp:352:4: error: 'excg_lo' was not declared in this scope
excg_lo,
^~~~~~~


I used :
cd build/unix
./autogen.sh
./configure --prefix=/local64 --{build,host,target}=x86_64-w64-mingw32
make -j4

Anyone an idea to fix this ?

[jackoneill]

Probably delete https://github.com/EleonoreMizo/fmtconv/blob/394a360eace1a26d8c43c8e68a772e0018bab612/src/conc/Interlocked.hpp#L349-L354
and replace with https://github.com/EleonoreMizo/fmtconv/blob/394a360eace1a26d8c43c8e68a772e0018bab612/src/conc/Interlocked.hpp#L360

Also open an issue.

[Pat357]

Thanks a lot !
Yup, that was the first stopper...
Now I got :

CXX ../../src/fmtcl/TransOpErimm.lo
CXX ../../src/fmtcl/TransOpFilmStream.lo
CXX ../../src/fmtcl/TransOpLinPow.lo
CXX ../../src/fmtcl/TransOpLogC.lo
CXX ../../src/fmtcl/TransOpLogTrunc.lo
CXX ../../src/fmtcl/TransOpPow.lo
CXX ../../src/fmtcl/TransOpSLog.lo
CXX ../../src/fmtcl/TransOpSLog3.lo
CXX ../../src/fmtcl/VoidAndCluster.lo
CXX ../../src/fstb/CpuId.lo
CXX ../../src/fstb/fnc.lo
{standard input}: Assembler messages:
{standard input}:46: Error: unsupported instruction `mov'
{standard input}:96: Error: unsupported instruction `mov'
{standard input}:113: Error: unsupported instruction `mov'
{standard input}:125: Error: unsupported instruction `mov'
{standard input}:161: Error: unsupported instruction `mov'
make: *** [Makefile:1084: ../../src/fstb/CpuId.lo] Error 1
make: *** Waiting for unfinished jobs..
Further ideas ? (probably has to do with '' around the mov instruction)

[jackoneill]

Maybe this works: https://dpaste.de/sVR3

[Pat357]

Yep, compiling is now without errors.
But : the generated .DLL is unfortunately not working :

- as long as I use "VSPIPE -i ", everything seems to be ok. No errors and the clip properties are correct (I'm using fmtc for scaling & bitdepth conversion).
- if I try to feed the stream to ffplay, mplayer or MPV, it looks like no frames are produced. FFplay sits there but no errors or no window for playing is opened.
- VSPIPE -p behaviours the same as the players : no errors and the progress counter doesn't even start. No crashes neither.

Did you ever compile it yourself ?

[jackoneill]

Yep, compiling is now without errors.
But : the generated .DLL is unfortunately not working :

- as long as I use "VSPIPE -i ", everything seems to be ok. No errors and the clip properties are correct (I'm using fmtc for scaling).
- if I try to feed the stream to ffplay, mplayer or MPV, it looks like no frames are produced. FFplay sits there but no errors or no window for playing is opened.
- VSPIPE -p behaviours the same as the players : no errors and the progress counter doesn't even start. No crashes neither.

Did you ever compile it yourself ?

I did, but only in Linux.

It sounds like there is a deadlock. Do you have avstp.dll someplace where it might be found, like in System32 or the same folder as libfmtconv.dll?

[Pat357]

I did, but only in Linux.

It sounds like there is a deadlock. Do you have avstp.dll someplace where it might be found, like in System32 or the same folder as libfmtconv.dll?
No avstp.dll anywhere in my path or in VS-plugin directories.

Would it also work in VS ?
I used it before in AVS but I found increasing #threads in Set's AVS 2.6 MT more stable then this plug-in.

[jackoneill]

No avstp.dll anywhere in my path or in VS-plugin directories.

Would it also work in VS ?
I used it before in AVS but I found increasing #threads in Set's AVS 2.6 MT more stable then this plug-in.

Maybe it would work. I think the code for it is there, at least. And you really should open a new issue or two at https://github.com/EleonoreMizo/fmtconv/issues.

[Pat357]

Maybe it would work. I think the code for it is there, at least. And you really should open a new issue or two at https://github.com/EleonoreMizo/fmtconv/issues.

I'll open 3 (2 + found another today) issues, but since development slowed down or even stopped, there is no hurry doing so; I'll do it tomorrow or the day after.

clip = core.std.BlankClip(width=1920, height=1080, fpsnum=24000 ,fpsden=1001, color=[255,128,128], format=vs.YUV420P8)
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=960, h=720, interlaced=False, interlacedd=False, cpuopt=0)
clip=core.resize.Point(clip=clip, format=vs.YUV420P8)
clip.set_output()
This will return a green clip if cpuopt=0 and a white clip if cpuopt=1 or anything but 0.

If I add this right before the fmtc line :
clip=core.resize.Point(clip=clip, format=vs.YUV420P16)
you get a white clip in both cases. It seems if cpuopt=0, the 8bit data is not converted to 16 bit first, maybe it's even the way it's supposed to work and you need 16b (using fmtc.bitdepth) before using fmtc.resample !

BTW, how do I create a Blank or colored clip with >8 bits data ?
I've tried
clip = core.std.BlankClip(width=1920, height=1080, fpsnum=24000 ,fpsden=1001, color=[255,128,128], format=vs.YUV420P16)
clip = core.std.BlankClip(width=1920, height=1080, fpsnum=24000 ,fpsden=1001, color=[1,0.5,0.5], format=vs.YUV420P16)
the documentation points to using float, like I did in the second case..like Luma is ~0-1 U and V are ~ -0.5-+0.5

I hope it is not implemented in a way that you need to double the values for each extra bit >8 .... This would be confusing and non-practical.

[Myrsloik]

I'll open 3 (2 + found another today) issues, but since development slowed down or even stopped, there is no hurry doing so; I'll do it tomorrow or the day after.

clip = core.std.BlankClip(width=1920, height=1080, fpsnum=24000 ,fpsden=1001, color=[255,128,128], format=vs.YUV420P8)
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=960, h=720, interlaced=False, interlacedd=False, cpuopt=0)
clip=core.resize.Point(clip=clip, format=vs.YUV420P8)
clip.set_output()
This will return a green clip if cpuopt=0 and a white clip if cpuopt=1 or anything but 0.

If I add this right before the fmtc line :
clip=core.resize.Point(clip=clip, format=vs.YUV420P16)
you get a white clip in both cases. It seems if cpuopt=0, the 8bit data is not converted to 16 bit first, maybe it's even the way it's supposed to work and you need 16b (using fmtc.bitdepth) before using fmtc.resample !

BTW, how do I create a Blank or colored clip with >8 bits data ?
I've tried
clip = core.std.BlankClip(width=1920, height=1080, fpsnum=24000 ,fpsden=1001, color=[255,128,128], format=vs.YUV420P16)
clip = core.std.BlankClip(width=1920, height=1080, fpsnum=24000 ,fpsden=1001, color=[1,0.5,0.5], format=vs.YUV420P16)
the documentation points to using float, like I did in the second case..like Luma is ~0-1 U and V are ~ -0.5-+0.5

I hope it is not implemented in a way that you need to double the values for each extra bit >8 .... This would be confusing and non-practical.

It uses the supplied value directly. Floats are rounded to int for int formats. It's implemented that way so you can easily have exact values for testing.

[dipje]

Something else where I'm thinking "let's just ask, who knows....":

Is there a way to _separate_ high frequency content from the mid to low frequency content?

Commercial noise-reduction programs/plugins have (sometimes.. a lot?) a preview-mode where they display the luminance channel at the same time, 3 times: high freq, mid freq, low freq. It helps (me at least) a lot in dialing in noise reduction or sharpening plugins by better seeing what is happening in certain areas. Is it possible to do this splitting so I can tweak around with it in Vapoursynth Editor to take a look at just the high-freq or low-freq stuff. Splitting the luminance and chroma channels is easy enough.

Also, on a related note, can someone give me a small example how I would go about to blending two clips with a certain opacity? Like, I want the original image, but overlay 30% of a cleaned-image on top of that. Can that be done with the generic Vapoursynth stuff?

[Myrsloik]

Something else where I'm thinking "let's just ask, who knows....":

Is there a way to _separate_ high frequency content from the mid to low frequency content?

Commercial noise-reduction programs/plugins have (sometimes.. a lot?) a preview-mode where they display the luminance channel at the same time, 3 times: high freq, mid freq, low freq. It helps (me at least) a lot in dialing in noise reduction or sharpening plugins by better seeing what is happening in certain areas. Is it possible to do this splitting so I can tweak around with it in Vapoursynth Editor to take a look at just the high-freq or low-freq stuff. Splitting the luminance and chroma channels is easy enough.

Also, on a related note, can someone give me a small example how I would go about to blending two clips with a certain opacity? Like, I want the original image, but overlay 30% of a cleaned-image on top of that. Can that be done with the generic Vapoursynth stuff?

misc.AverageFrames([clipa, clipb], [10, 3], scale=1/10) (or scale=1/13 depending on how you want it)

For the frequency stuff you can simply use normal convolutions if you don't need speed or something spectacular. [-1 -1 -1 -1 8 -1 -1 -1 -1] <- highpass filter, make it all ones and you have a lowpass filter.

Or describe more exactly what you want.

[dipje]

I guess a highpass and/or lowpass is what I want yeah.. maybe some Levels tweaking to get the stuff visible but that is something 'done to taste'.

Only way for me to describe it better is by a screenshot: https://snag.gy/ym2UzE.jpg. But highpass / lowpass can help a lot already.

[dipje]

Found docs of AverageFrames in the github. Not on your site yet :P.

Hmm.. 'scale must be a positive number' as an error message, I get whenever I have a scale of less than 0.5. scale = 1/10 or cale = 1/13 both give less than 0.5 :).

Can I just keep 'scale = 1.0' and then change the weights around between 0.0 and 1.0... as long as both of my weights are a total of 1.0?

weight a = 0.0, weight b = 1.0 means get only 'clip b'.
weight a = 1.0, weight b = 0.0 means get only 'clip a'.
weight a = 0.5, weight b = 0.5 means get perfect blend between the two.

So then I can start shifting the weights to '0.4 and 0.6' or '0.7 and 0.3' depending on how much I want of which clip.. am I right in thinking like this?

edit: Took a look at the source of AverageFrames.
It seems 'weights' are integers and 'scale' as well, while the documentation lists them both as floats.
Also, the docs say that after multiplying by the weights + summing, the results is multiplied by the scale. Looking at the source it seems it's divided by the scale (which would make the integers much better to understand and work with :P).

So weight a = 70, weight b = 30, scale = 100 seems to give me 'a little bit of b' what I'm looking for.

[Myrsloik]

Found docs of AverageFrames in the github. Not on your site yet :P.

Hmm.. 'scale must be a positive number' as an error message, I get whenever I have a scale of less than 0.5. scale = 1/10 or cale = 1/13 both give less than 0.5 :).

Can I just keep 'scale = 1.0' and then change the weights around between 0.0 and 1.0... as long as both of my weights are a total of 1.0?

weight a = 0.0, weight b = 1.0 means get only 'clip b'.
weight a = 1.0, weight b = 0.0 means get only 'clip a'.
weight a = 0.5, weight b = 0.5 means get perfect blend between the two.

So then I can start shifting the weights to '0.4 and 0.6' or '0.7 and 0.3' depending on how much I want of which clip.. am I right in thinking like this?

No, you can't. Integer weights for integer clips. Floats get rounded.


Ooops, scale should be 10 or 13. It's the integer to divide by. I should double check this stuff and fix the documentation...

[Joachim Buambeki]

I can't comment on how to do frequency separation in VS but I can vouch for it to be extremely useful for (very) high resolution video denoising and am wondering why this hasn't been implemented in any noise removal scripts (or has it)?

[dipje]

I just use it to finetune the settings. I split my yuv into 3 separate channels, then one by one I use this convolution trick to really show the fine detail noise. Things like compression artifacts, blockiness, lines, etc.. In playing with the convolution stuff (and to enhance it's effect a bit or make it better to view) I switched to the 5x5 convolution and then play with the divisor value (values >= 0 <= 1.0) to get something that highlights the artifacts.
Then I place knlmeanscl before it and tweak the strength parameter there till the artifacts are gone. I do that separately for all 3 channels (although most of the time my U + V channels end up being pretty close / the same). I merge the 3 clips back into a single YUV clip and view it. If I think it looks 'too strongly denoised' I use the averageframeS() thing to blend it with the original till I get something that doesn't loose (too much) detail.
Deband everything slightly with f3kdb to 10bits / 12bits (depending on the project) which will add some dithering noise back in but I lost a lot of banding and compression artifacts of 'weaker codecs'. Render the stuff to a DPX sequence or something else used by colour programs and the clip can go into grading.

[age]

I guess a highpass and/or lowpass is what I want yeah.. maybe some Levels tweaking to get the stuff visible but that is something 'done to taste'.

Only way for me to describe it better is by a screenshot: https://snag.gy/ym2UzE.jpg. But highpass / lowpass can help a lot already.

It seems the same Frequency separation described in this article for gimp
https://pixls.us/articles/skin-retouching-with-wavelet-decompose/

We need "grain extract" and "grain merge"
https://docs.gimp.org/2.9/en/gimp-concepts-layer-modes.html

grain_extract=core.std.Expr(clips=[lower layer,upper layer], expr=[" x y - 128 + "])

grain_merge=core.std.Expr(clips=[lower layer,upper layer], expr=[" x y + 128 - "])



c
low_frequency=core.std.Convolution(c,matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1])
high_frequency=core.std.Expr(clips=[c,low_frequency], expr=[" x y - 128 + "])
f=core.std.Expr(clips=[low_frequency,high_frequency], expr=[" x y + 128 - "])

[Joachim Buambeki]

What would be an approximation of the mid frequency? According to my experience, you can be increasingly more aggressive with the (temporal) denoising, the lower your frequency is (with the odd exceptions).

[dipje]

@Age, Thanks for the input!

I tweaked it a bit though. Well.. I'm working in 16bit so the '128' in your example I changed to 32768. And the simple 3x3 convolution I replaced with a 5x5 gaussian blur kernel.

After splitting the image into 'hfreq' and 'lfreq' this way, I split 'hfreq' into separate y, u and v. I apply KNLMeansCL do on of the planes, afterwards I boost the contrast quite a bit (on the hfreq ones at least) by using std.Limiter / std.Levels. This way I can really see the high-freq filth in the image. I increase KNLMeansCL's strength bit by bit until it look good. I repeat the process for the u and v planes, which normally can do with less strength on knlmeanscl. I combine the y, u and v clips back into 'hfreq', then to the exact same with the lfreq clip (split into y,u,v, tweak knlmeans with a contrast boost, turn off the contrast boost and combine them back into one 'lfreq' clip). I actually finalized on less strength and less of a search window on the low frequency content (Since I'm more working on compression artifacts and banding issues and such, not the noise of the image itself).

Then use your last 'f' line to combine my lfreq and hfreq back into a single clip, use ContraSharpening against the original, f3kdb with slight settings to dither down to 10bit and presto.

[WolframRhodium]

If you are using something like
Expr([clipa, clipb], ['x y + 128 -'])
why don't you simply use MergeDiff:
MergeDiff(clipa, clipb)
?

[dipje]

On the same note, why note change
high_frequency=core.std.Expr(clips=[c,low_frequency], expr=[" x y - 128 + "])
into
high_frequency = core.std.MakeDiff(c, low_frequency)

Is std.Expr faster or something?

On the same note, is there a reason to use the whole misc.AverageFrames thing instead of core.std.Merge if I only have two clips?

[age]

Yes it's the same as

lfreq=core.std.Convolution(c,matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1])
hfreq=core.std.MakeDiff(clipa=c, clipb=lfreq)
c=core.std.MergeDiff(clipa=lfreq,clipb=hfreq)


:-)

[jackoneill]

Is std.Expr faster or something?


For 8 bit input Expr should be slower than MakeDiff and MergeDiff. Expr always calculates everything with 32 bit floats. MakeDiff and MergeDiff have code optimised specifically for 8 bit input.

For other bit depths, Expr could be faster than MakeDiff and MergeDiff, because it processes four pixels at a time.

[dipje]

I think I would use std.Expr a lot more if I could wrap my head around the notation :).

[age]

@Age, Thanks for the input!

I tweaked it a bit though. Well.. I'm working in 16bit so the '128' in your example I changed to 32768. And the simple 3x3 convolution I replaced with a 5x5 gaussian blur kernel.

After splitting the image into 'hfreq' and 'lfreq' this way, I split 'hfreq' into separate y, u and v. I apply KNLMeansCL do on of the planes, afterwards I boost the contrast quite a bit (on the hfreq ones at least) by using std.Limiter / std.Levels. This way I can really see the high-freq filth in the image. I increase KNLMeansCL's strength bit by bit until it look good. I repeat the process for the u and v planes, which normally can do with less strength on knlmeanscl. I combine the y, u and v clips back into 'hfreq', then to the exact same with the lfreq clip (split into y,u,v, tweak knlmeans with a contrast boost, turn off the contrast boost and combine them back into one 'lfreq' clip). I actually finalized on less strength and less of a search window on the low frequency content (Since I'm more working on compression artifacts and banding issues and such, not the noise of the image itself).

Then use your last 'f' line to combine my lfreq and hfreq back into a single clip, use ContraSharpening against the original, f3kdb with slight settings to dither down to 10bit and presto.

You can first split the image in grayscale and color in the same way but i don't now if it is better than yuv


#input is rgb48
c=clip
r=core.std.ShufflePlanes(clips=c, planes=[0], colorfamily=vs.GRAY)
g=core.std.ShufflePlanes(clips=c, planes=[0], colorfamily=vs.GRAY)
b=core.std.ShufflePlanes(clips=c, planes=[0], colorfamily=vs.GRAY)

gray=core.std.Expr(clips=[r,g,b], expr=[" x y + z + 3 / "])
#gray=denoise or frequency separation
gray=core.std.ShufflePlanes(clips=c, planes=[0,0,0], colorfamily=vs.RGB)
color=core.std.Expr(clips=[c,gray], expr=[" x y - 32896 + "])
#color=denoise or frequency separation
c=core.std.Expr(clips=[gray,color], expr=[" x y + 32896 - "])

gray
http://i.imgur.com/WSli60E.jpg
color
http://i.imgur.com/6MnrusC.jpg
recombine
http://i.imgur.com/eyhgBHb.jpg

[dipje]

you there do 'grayscale' by taking the average of rgb.
Since I'm almost always starting from YUV videos, I think I have something closer to 'luminance' any time than the average of RGB, which I don't call true luminance data, but quick-n-dirty grayscale.
And I actually do tweak the u and v channels separately. On of my older camera's produces way more chroma noise in the reds/greens for instance.

[Mystery Keeper]

On of my older camera's produces way more chroma noise in the reds/greens for instance.Thats strange. Camera matrices tend to have two green sensors per pixel, so usually they produce more red and blue noise and less green.

[dipje]

it's video mode, so it depends more on whatever the engine in the camera is doing, not what the actual photo cells measure :). The older sony engines just don't dare to touch any reds in noise reduction , maybe to leave some color in the skins or to prevent waxy looking faces or something. In the end they don't remove any red chroma noise at all it seems. In the shadows (if the shot is high iso) it can be pretty ugly, specially with the older AVCHD +/ 24mbps encoders in those files.

Aanyway, if the file is YUV in and I encode yuv-out I prefer to keep it yuv all the way if I can.
I'll have a try splitting it through RGB though, I believe Topaz Denoise 6 plugin does something like that (you can preview luma and you can preview 'color', and there you get a blue/red mixture without any green. Kinda weird, but helps in tweaking chroma noise.. other trick I know of is setting really high luma-only noise reduction to make the chroma noise stand out). Although I don't think I'll use it often it's nice to experiment and add it to the back of tricks.

In a related note, I don't think KNLMeansCL can run on two planes at once? I believe it's either Y, YUV or RGB, but I can't let it run on 'UV' in once pass. So maybe doing the gray/color thing in RGB means I can get away with one KNLMeansc all for gray and one for color, while otherwise it'll be 3 calls for Y, U and V. It all depends on how much slower the RGB-pass is for the 'color' clip vs two separate calls on a grayscale clip (u and v processed separately)

[hydra3333]

Hello. Re new very fast CUDA based filters :-

1. Sure, I'll add them.

2. You don't, write native stuff. Or recompile only avscompat.dll on your own. All you need is vs2017 community which is free.
Thanks!

Adding to post https://forum.doom9.org/showthread.php?p=1801677#post1801677 seeking to update the filter list into Vapoursynth so that VS can successfully seek and return the right number of frames forward/back, I gather that this is the latest list

filter name, frames (forward and back, or current frame) they need as input to produce one frame of output
DGSource(): current only
DGDenoise(): current only
DGSharpen(): current only
DGTelecide(): always previous + current
DGDecimate(): up to (cycle size + 2) number of previous + current
DGBob(): always previous + current + next (or, n-2,n-1,n,n+1,n+2 for double framerate ?)

( DGbob is a new gpu accelerated deinterlacer based on yadifmod2 http://rationalqm.us/board/viewtopic.php?f=14&t=559&start=180#p6701 )

A question, if I got it wrong (eg say dgbob was = current frame only) does it matter if previous,current,next is specified ?

Also, given
DGDecimate(clip c, int cycle, int device, bool show)
DGDecimate() is a fast stand-alone CUDA filter that implements 1-in-M frame decimation. The input clip must be in the YV12 format.
cycle: int value (default: 5)
Decimation cycle size M. For every set of M frames, the frame that differs least from its preceding frame is removed.
what are the "right" numbers to specify for DGDecimate ? an arbitrary 10 + 10 ?

[hydra3333]

It has been suggested that a script at run time cannot tell the Vapoursynth core at its compile time what the cycle size is going to be, and if Vapoursynth always assumes some maximum it is going to over-request frames.

I suppose an arbitrary large default number of frames may have to do for DGDecimate and it will need to over-request frames.

May I make an incremental improvement suggestion for a future release ?

Not sure if this would be possible or not ... wouldn't it be nice if values were configurable at script startup from a vapoursynth-provided .ini text file of defaults in the vapoursynth folder (with a default .ini file provided with vapoursynth) ... and then a new vapoursynth function available to over-ride at run-time with parameters along the lines of itll_needs_frames("function_name", number_before, number after).

Into the future, one may imagine all sorts of fantastic new filters popping up which may need more than current-frame and indeed may need an arbitrary number of past and future frames. There's some talk of a gpu accelerated QTGMC equivalent (thus fast by comparison to cpu based QTGMC) ... wouldn't that be amazing ?

[hydra3333]

2. You don't, write native stuff. Or recompile only avscompat.dll on your own. All you need is vs2017 community which is free.
Beaut, thanks.

OK "vs2017 community" to recompile only a "portable" avscompat.dll on my own ... now I'm off to look up how to do that for x64 :) and then plonk a replacement AvsCompat.dll under
Vapoursynth\vapoursynth64\coreplugins


OK, I did download an install vs2017 ... and didn't like what it did to my PC, it wouldn't hibernate any more and there appeared to be lots of new bloat junk running in the background - so I rolled back to a system image taken just before the vs2017 install. I wonder if I can find an old win8 licensed VM and try it in there. Having said that, wouldn't it seem preferable to have some functionality per the suggestion above :) ?

[Myrsloik]

No, lrn2vm

[hydra3333]

No, lrn2vm
I suppose it is time to dredge up some 40 year old coding skills and take a look.

Now, where's that paper tape reader and the newfangled golf ball printer ?

[vcmohan]

Something else where I'm thinking "let's just ask, who knows....":

Is there a way to _separate_ high frequency content from the mid to low frequency content?
I just got this post in my mail alerts. Looks that much water has flown since.
However if interested have a look of my plugin vcfreq which has functions to filter various freq bands.

[dipje]

I just got this post in my mail alerts. Looks that much water has flown since.
However if interested have a look of my plugin vcfreq which has functions to filter various freq bands.

I knew of that one, sort of, but always rejected it from reading the manual. And now I'm thinking the same: It doesn't look to separate a clip into different clips and then able to merge them together again, right?

It seems more to be able to do a whole lot of predefined filters on separate frequencies, or to nudge out some peaks on a 'FFT Diagram' (don't know the real word for it).

If I want to separate the frequencies into 3 clips + a leftover clip, do some random stuff to it with other plugins then combine them back.. I don't see how vcfreq can do that or I'm completely missing the point somewhere.


BTW, I'm pretty happy now with what I do got. I'm using the gaussian blur filter from the bilateral plugin, and I go a few times through the process of 'blur away some fine detail with small radius gblur, compare original image to blurred, extract differences to frequency clip, then repeat the process with the blurred image and a bigger radius gblur, etc...'. That way I get a few clips with increasingly more 'detail' in it and at the end one image which is sort of the 'base' image. Filter on certain detail-levels only with knlmeanscl / smdegrain / f3kdb then merge them back together

[TheFluff]

May I make an incremental improvement suggestion for a future release ?

Not sure if this would be possible or not ... wouldn't it be nice if values were configurable at script startup from a vapoursynth-provided .ini text file of defaults in the vapoursynth folder (with a default .ini file provided with vapoursynth) ... and then a new vapoursynth function available to over-ride at run-time with parameters along the lines of itll_needs_frames("function_name", number_before, number after).

No, please stop. The actual VS plugin API supports plugins requesting arbitrary frames asynchronously on a per-frame basis. If you want your plugin to behave nicer under the VS plugin API, write an actual VS interface for it instead of adding these braindamaged hacks. It's pretty trivial, or at least it should be (disregarding typical Avisynth plugin code quality).

The entire AVS plugin compatibility layer is already bad enough - I suspect it's mostly there to prove a point, but one that sadly seems to have flown over the head of the intended recipients (I think it was supposed to point out to the Avisynth developers that their policy of piling hack on hack on top of their ancient plugin API for compatibility reasons instead of adding a new one better suited to modern reality was fundamentally mistaken and unnecessary, but now the hacked up but not quite compatible garbage is so well established that they can't break compatibility with that either). Is there something in Avisynth itself that drives people to software architectural madness, or what?

[hydra3333]

No, please stop. The actual VS plugin API supports plugins requesting arbitrary frames asynchronously on a per-frame basis. If you want your plugin to behave nicer under the VS plugin API, write an actual VS interface for it instead of adding these braindamaged hacks. It's pretty trivial, or at least it should be (disregarding typical Avisynth plugin code quality).

I'm sorry, sometimes we don't get to modify others' code even if it were available and then there's the gaps in underpinning knowledge about how to do it :)

I was just trying to suggest how to make VS handy and simple to use from an end-users perspective (that'd be me), given that great new plugins pop up all the time and not every home scripter (that'd be me) knows how to recompile DLLs.

[Joachim Buambeki]

However if interested have a look of my plugin vcfreq which has functions to filter various freq bands.Incidentally, I just stumbled over your plugin yesterday while searching for ways to separate the frequencies.

Could you please give us an example script that separates the planes into high, mid and low frequencies (with the corresponding parameters/radii), denoises them and merges them back together?
Would you use different radii for HD vs ≥4K footage? If not, how about very low und ultra low frequencies for very high resolution? Commercial software denoises them as well (if I had to guess I'd assume it is a lowpass with a high three digit radius but it is hard to tell).
Also, if you use something like MDegrain, do you have to calculate vectors for each separated plane or can you use the vectors from the unseparated plane for each MDegrain call to save CPU cycles?

Thanks!

[AzraelNewtype]

No, please stop. The actual VS plugin API supports plugins requesting arbitrary frames asynchronously on a per-frame basis. If you want your plugin to behave nicer under the VS plugin API, write an actual VS interface for it instead of adding these braindamaged hacks. It's pretty trivial, or at least it should be (disregarding typical Avisynth plugin code quality).

The entire AVS plugin compatibility layer is already bad enough - I suspect it's mostly there to prove a point, but one that sadly seems to have flown over the head of the intended recipients (I think it was supposed to point out to the Avisynth developers that their policy of piling hack on hack on top of their ancient plugin API for compatibility reasons instead of adding a new one better suited to modern reality was fundamentally mistaken and unnecessary, but now the hacked up but not quite compatible garbage is so well established that they can't break compatibility with that either). Is there something in Avisynth itself that drives people to software architectural madness, or what?

This is about a series of n2 plugins, and n2 has declared Holy War against VapourSynth because he didn't want to invoke a function to reduce rational fps. Your outrage is understandable, but impossible in this instance.

[TheFluff]

I'm sorry, sometimes we don't get to modify others' code even if it were available and then there's the gaps in underpinning knowledge about how to do it :)

I was just trying to suggest how to make VS handy and simple to use from an end-users perspective (that'd be me), given that great new plugins pop up all the time and not every home scripter (that'd be me) knows how to recompile DLLs.

I went and checked the discussion on neuron2's forum and I think at least one of you is missing the point. You wrote:

In order to add to thread https://forum.doom9.org/showthread.php? ... ost1801677 seeking to update the filter list into Vapoursynth so that VS can successfully seek and return the right number of frames forward/back, is this correct (in particular dgbob which I'm not sure is temporal) ?
The prefetch list for Avisynth filters has nothing to do with seeking. VS will work perfectly fine even without the prefetch info. All providing the prefetch info does is to make the VS threadpool arrange so that by the time it calls getFrame(n) on the wrapped Avisynth filter, all frames that that filter will request once getFrame(n) is called have already been fetched from the upstream filter. That's why it's called prefetch. If you don't have prefetch info, all that happens is that VS reverts to standard Avisynth behavior and the thread blocks while waiting on the upstream filter(s) to produce input for the wrapped Avisynth filter.

Providing the prefetch info just makes the Avisynth filter behave better than it does in (single-threaded) Avisynth since upstream filters can be run in parallel. Without it, performance is (almost) exactly like Avisynth.

N2 then wrote:
Isn't something seriously wrong with Vapoursynth if this has to be hard-coded in the core? DGDecimate() at script run time cannot tell the Vapoursynth core at its compile time what the cycle size is going to be, and if Vapoursynth always assumes some maximum it is going to over-request frames.
And, well, yes, of course it's seriously wrong. It's just a dumb hack that only exists in the Avisynth wrapper to work around the fact that Avisynth's frame requests are synchronous and blocking. The actual VS plugin interface doesn't work like that, instead plugins request what frames they want asynchronously (as I've previously explained at some length (https://forum.doom9.org/showthread.php?t=174437)) and they can decide on a per-frame basis which frames they need at runtime.

If you want to load legacy plugins with no source the Avisynth wrapper may be an acceptable solution, but for new(-ish) plugins in active development I really don't think it's reasonable to demand that VS fix the concurrency bottlenecks caused by Avisynth's refusal to adopt a reasonable plugin interface that actually supports multithreading. There really isn't much to the VS interface, the data structures are quite similar to Avisynth's and supporting both interfaces in a single plugin really isn't that hard.

e:
Here's another stupidity with the VS prefetch nonsense. DGBob (and yadifmod2 and probably others) can specify either single-rate or double-rate deinterlacing by a parameter. The PREFETCH parameters would need to be different for these two and I don't see how to set that at VS compile time.
The hardcoded prefetch stuff actually does support adjusting the prefetch range based on one or more parameters to the filter and that's one of the reasons it'd be annoying to put it in an .ini file, so, well. See for example https://github.com/vapoursynth/vapoursynth/blob/master/src/avisynth/avisynth_compat.cpp#L248

[hydra3333]

OK and thanks for clarifying.

I went and checked the discussion on neuron2's forum and I think at least one of you is missing the point. OK, no doubt that'd be me.

If you don't have prefetch info, all that happens is that VS reverts to standard Avisynth behavior and the thread blocks while waiting on the upstream filter(s) to produce input for the wrapped Avisynth filter.

Providing the prefetch info just makes the Avisynth filter behave better than it does in (single-threaded) Avisynth since upstream filters can be run in parallel. Without it, performance is (almost) exactly like Avisynth.
Unless I misunderstood again, from an end-user perspective specifying useful prefetch values maximizes the probability of increased performance, which is a desirable outcome in a playing field where one can experience *few* fps processing from time to time.

If you want to load legacy plugins with no source the Avisynth wrapper may be an acceptable solution, but for new(-ish) plugins in active development I really don't think it's reasonable to demand that VS fix the concurrency bottlenecks caused by Avisynth's refusal to adopt a reasonable plugin interface that actually supports multithreading. There really isn't much to the VS interface, the data structures are quite similar to Avisynth's and supporting both interfaces in a single plugin really isn't that hard.Yes ... although Authors are free do do as they wish with the (sometimes semi commercial) art they create, and sometimes the code won't be publicly available.

The hardcoded prefetch stuff actually does support adjusting the prefetch range based on one or more parameters to the filter and that's one of the reasons it'd be annoying to put it in an .ini file, so, well. See for example https://github.com/vapoursynth/vapoursynth/blob/master/src/avisynth/avisynth_compat.cpp#L248OK, thank you, nice. Yes, that's in code which is not effectively changeable by end users unless I am mistaken. In a range of future cases though (separate from this instance) it seems possible to conjecture with near certainty that "filter" code changes may not occur for different reasons ... and the next best thing may be for end-users to have some flexibility (Vapoursynth would likely have design objectives of being better and faster, after all) by, if possible, either/or/and (i) create a way for end-users to help themselves, eg an .ini file (ii) create a script interface to set it at runtime independently of the filter.

I accept that the suggestions are being viewed with significant disfavour, but it was still worth the risk of a belt by asking.

[Myrsloik]

...
Yes ... although Authors are free do do as they wish with the (sometimes semi commercial) art they create, and sometimes the code won't be publicly available.
...

Don't you mean donationware? (that you have to donate to access but totally isn't commercial)

I'm also free to ignore all bullshit. HAHAHAHAHAHA

[hydra3333]

And so you are indeed free without hindrance to deal with matters how you please, kind sir.
Yes, I did indeed mean donationware as well as other variations including closed source.
Oh well.

[Myrsloik]

R38 RC1 (https://dl.dropboxusercontent.com/u/73468194/VapourSynth-R38-RC1.exe)

Go test it. BoxBlur: arguments"clip:clip;planes:int[]:opt;hradius:int:opt;hpasses:int:opt;vradius:int:opt;vpasses:int:opt;" h/v is for horizontal/vertical of course, passes is how many times the blur should be applied

r38:
updated to zimg v2.5.1
added boxblur filter
various fixes to averageframes
averageframes can now use up to 31 weights
increased the number of allowed constants in h and v convolution modes to 25
added new constants to clipinfo
fixed several subtext corruption and crash bugs (jackoneill)
added y410 and r210 output to avfs and vsvfw
added constants for 12 and 14 bit yuv formats
fixed a one pixel offset error only happening in vertical convolution with a 3 element vector
switched to vs2017 in windows projects

[hydra3333]

Seeking advice.

The visual studio integration is broken. (I think that comment is about VSYASM integration into Visual Studio 2017)

I am attempting to try to build a part of Vapoursynth using Visual Studio Community 2017 per advice in this thread, and started to look at getting dependencies together one by one.

The https://github.com/vapoursynth/vapoursynth GIT file "build instructions windows.txt" says this
Default install paths are assumed in all projects and scripts, be prepared to adjust many things if you changed them

Needs Visual Studio 2017 and vsyasm (yasm for use with VS2010)
Visual Studio 2015 Update 3 is also (unfortunately) required to compile the python module
The proper paths to put things in are:
<some random exe path>
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\BuildCustomizations

It also needs both 32 and 64 bit Python 3.6 series with recent cython, sphinx and sphinx-intl installed
Inno setup is needed to create the installer
7-zip is needed to compress the portable version

Clone VapourSynth
Clone zimg v2.5 branch into the VapourSynth dir (https://github.com/sekrit-twc/zimg)
Clone avs+ mt branch into the VapourSynth dir (https://github.com/AviSynth/AviSynthPlus)
FIX LLVM STUFF IN THE ZIMG PROJECT
Compile 32 and 64 bit release using the VapourSynth solution
Run cython_build.bat to compile the Python modules

In order to build the installer you need to download
and place isxdl.dll in "installer\scripts\isxdl".
It can be downloaded from:
http://www.codeproject.com/Articles/20868/NET-Framework-Installer-for-InnoSetup

There are also a few plugins that aren't included
which are easier to simply retrieve from an existing
VapourSynth installation.

Note that the Avisynth side of AVFS won't work properly in debug builds (memory allocation and exceptions across module boundaries trolololol)
Likewise AviSource can only be compiled as a release build (or debug versions of functions that aren't present are referenced)

Just checking, I read somewhere that there's updates to YASM git https://github.com/vapoursynth/vapoursynth which are required for it to work, but which aren't in the 1.3.0 release ?

Anyway, tried to build VSYASM using VS2017 and its build process crashes. Tweaked it a bit along these lines and jagged an error free build.
To build VSYASM with Visual Studio 2017 Community,
for use with Visual Studio 2017 Community,
in building parts of Vapoursynth.

Tested in a VM under Windows 8.1

Clue: parts of the build proces "fail to build", so
- ensure file/folder permissions are not read-only
- ensure "everyone" has full control of the folder tree
- edit a couple of .bat files which crash but don't really
- add compilation flag /FS to every project in the solution

1. Install VS2017 and every c++ bit that comes with it. Install everything.
2. Don't bother installing python just for this, it only gets in the way, but you'll need it later.
3. download YASM from GIT which does actually have all of the REQUIRED latest changes after 1.3.0 was released
- from https://github.com/yasm/yasm
- save the .zip which will be yasm-master.zip
- then extract it to a folder
4. change properties on the top folder .\yasm-master to not be read-only any more
5. change security permissions on the top folder .\yasm-master so "everyone" has full control
6. Locate file in the root called YASM-VERSION-GEN.BAT
6.1 Add this line as a new line 3
GOTO :notfound
6.2 Locate the line under label :output which says
if "%_ver_%" == "%_oldver_%" exit /b
and add REM in front of it so it becomes an inactive line
7. Locate file .\yasm-master\Mkfiles\vc15\genperf\run.bat
Add this as a new line 3
goto try21
Under the label :update locate and REM out this line
rem del tf
7. Open the solution in VS1017: .\vsyasm\yasm-master\Mkfiles\vc15\yasm.sln
7.1 One at a time, FOR EVERY PROJECT in the solution
- right-click and choose "Properties"
- Expand "Configuration Properties"
- Expand "C/C++"
- Click on "Command Line" and notice "Additional Options" textbox on the right
- Add /FS to the "Additional Options" textbox
8. in the top of VS for this solution, select
"Release"
"x64"
9. Right-click on the solution then choose "Rebuild Solution" then watch it build
10. See the text ========== Rebuild All: 11 succeeded, 0 failed, 0 skipped ==========
11. Open the folder .vsyasm\yasm-master\Mkfiles\vc15\x64\Release
and copy all of the .exe files to a temporary folder somewhere handy eg under your desktop
this whould include the file "vsyasm.exe"
12. Locate these three files
vsyasm.props
vsyasm.targets
vsyasm.xml
in folder .\yasm-master\Mkfiles\vc15 and copy them to the same temporary folder as the .exe files above.

13. Under Visual Studio 2017, the VC\bin folders mentioned below
are really bad to find and can be subversion-dependent
which is a real pain of Microsoft to have done.
So evey time you login ...
to tell Visual Studio where to find vsyasm.exe, the environment
variable YASMPATH can be set to the absolute path of the
directory in which vsyasm.exe is located
(this path should include the final backslash). eg
set YASMPATH=C:\Users\u\Desktop\vsyasm\yasm-master\exe\
To set the environment variable
- right click on this pc, choose System Properties, click the button Environment Variables
Under System Variables Click New then add the new variable described above pointing to your .exe folder

Use one of 14. OR.... 15.

14. Add the path for VS2017 to find the stuff. In VS2017 use menu
Tools
Options
Projects & Solutions
VC++ Project Settings
then notice on the right "Build Customizations Search path"
Paste in the full folder path (excluding the final slash) containing the .xml files etc above, and save it

15. pop the 3 files in this folder
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\BuildCustomizations

Now we've enabled YASM for use in compiling Vapoursynth. Hopefully.

Am I wasting my time or will vsyasm now integrate into vs2017 ? How to others do it ?

Re this
Visual Studio 2015 Update 3 is also (unfortunately) required to compile the python module
Does that mean I need to find and install vs2015 as well ?


edit 1: for other uninitiated newbies like me coming across this, for other dependencies like cython, sphinx and sphinx-intl this seemed to not crash:
1. install 64 bit python (3.6 in this case) as I only need 64bit stuff
2. start an Admin CMD dos box (or it can't copy files into protected folders) then use the pip command to install the packages, which apparently gets installed with python:
3. pip install cython
4. pip install sphinx-intl
#4 seems to also locate and install sphinx
edit 2: I got this error building avscompat LINK : fatal error LNK1181: cannot open input file 'gdi32.lib' which is apparently a known error per http://stackoverflow.com/questions/33599723/fatal-error-lnk1104-cannot-open-file-gdi32-lib
If anyone has this problem on Visual Studio 2017, it is an issue with the Windows 10 Creators Update.
A workaround is to select the
"Windows 10 SDK (10.0.15063.0) for Desktop C++ x86 and x64"
under the "Desktop development with C++" workload in the Visual Studio Installer.
edit 3: Last comment, for newbies.
1. For vapoursynth to build, the avs+ mt branch placed in the root of vapoursynth folder tree must be called "AviSynthPlus"
2. and the zimg v2.5 branch placed in the root of vapoursynth folder tree must be called "zimg".

[Myrsloik]

You only need vs2017, nothing else. You only need to compile the avscompat subproject. Ignore the rest. And vsyasm is derp

[hydra3333]

Thank you for this:
https://github.com/vapoursynth/vapoursynth/commit/c6c7efd1cd18529ae33a96ff1a42b1aa51409a10

If I read the code right, for DGBOB and DGDecimate it also does dynamic setting based on parameters provided to those plugins and serves as a template for doing such for future plugins whenever I need to rebuild avscompat. Beaut !
SOURCE(DGSource)
+ PREFETCHR0(DGDenoise)
+ PREFETCHR0(DGSharpen)
+ temp = int64ToIntS(vsapi->propGetInt(in, "mode", 0, &err));
+ PREFETCH(DGBob, (temp > 0) ? 2 : 1, 1, -2, 2) // close enough?

+ PREFETCH(DGTelecide, 1, 1, -2, 10) // also not good
+ temp = int64ToIntS(vsapi->propGetInt(in, "cycle", 0, &err));
+ PREFETCH(DGDecimate, temp - 1, temp, -(temp + 3), temp + 3) // probably suboptimal

Cheers !


I had to lookup derp ... and now will be using the noun at work to denote a fair portion of what I hear.

edit 1: I'll ask a question and minor suggestion over at the GIT about "(temp > 0) ? 2 :"

edit 2: Thanks. I ended up going with this for readability, and added the new standalone gpu based nvidia purevideo deinterlacer as well.
// DGBob yadif based deinterlacer http://rationalqm.us/board/viewtopic.php?f=0&t=463&p=6712&hilit=dgbob#p6712
// http://rationalqm.us/board/viewtopic.php?f=14&t=559&p=6732&hilit=dgbob#p6732
temp = int64ToIntS(vsapi->propGetInt(in, "mode", 0, &err));
//PREFETCH(DGBob, (temp > 0) ? 2 : 1, 1, -2, 2) // close enough?
// I dont think so. DG said always previous + current + next ... fudge it a bit for doublerate, what the heck
switch (temp) {
case 0:
PREFETCH(DGBob, 1, 1, -1, 1); break; // single framerate deinterlacing -1,current,+1
case 1:
PREFETCH(DGBob, 1, 1, -2, 2); break; // double framerate deinterlacing -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1 ... -2,2 can't hurt too badly I guess
case 2:
PREFETCH(DGBob, 1, 1, -2, 2); break; // double framerate deinterlacing to single rate (makes slow motion) -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1 ... -2,2 can't hurt too badly I guess
}
// not sure what to do about PVBob nvidia PureVideo based deinterlacer http://rationalqm.us/board/viewtopic.php?f=14&t=559&start=240#p6786 ... act like DGBob for the time being
temp = int64ToIntS(vsapi->propGetInt(in, "mode", 0, &err));
switch (temp) {
case 0:
PREFETCH(PVBob, 1, 1, -1, 1); break; // single framerate deinterlacing -1,current,+1
case 1:
PREFETCH(PVBob, 1, 1, -2, 2); break; // double framerate deinterlacing -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1 ... -2,2 can't hurt too badly I guess
case 2:
PREFETCH(PVBob, 1, 1, -2, 2); break; // double framerate deinterlacing to single rate (makes slow motion) -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1 ... -2,2 can't hurt too badly I guess
}

[feisty2]

I'm thinking of another way around to make pointers returned by getReadPtr/getWritePtr 2D without using alloca()
like the following,,, (overload "[]")

#include <iostream>
#include <cstdlib>

template<typename T>
struct FramePointer final {
T *Pointer = nullptr;
decltype(0ull) Width = 0;
FramePointer(T *Pointer, unsigned long long Width) {
this->Pointer = Pointer;
this->Width = Width;
}
FramePointer(FramePointer &&) = default;
FramePointer(const FramePointer &) = default;
auto operator=(FramePointer &&)->FramePointer & = default;
auto operator=(const FramePointer &)->FramePointer & = default;
~FramePointer() = default;
auto operator[](unsigned long long Row) {
return Pointer + Row * Width;
}
};

auto main()->int {
auto Width = 1920ull, Height = 1080ull;
auto RawFramePointer = new double[Width * Height];
auto ActualPointer = FramePointer<double>{ RawFramePointer, Width };

ActualPointer[2][34] = 2.718;
std::cout << RawFramePointer[2 * Width + 34] << std::endl;

delete[] RawFramePointer;
system("pause");
}


do you think it's any way better than the previous "alloca" approach?

[jackoneill]

The question is: better for what purpose?

Readability? No, especially with your style of C++.

Not using a bunch of stack space for no reason? Yes.