Vapoursynth

[Mystery Keeper]

Thank you. Installed R21 RC, and it has become fast.
This plugin interpolates between frames within (radius) using linear approximation/regression.

update: Compared unfiltered and filtered single planes. Filtered look like they have something like valid output, but much brighter (twice as bright?). That shouldn't be happening though, since all I'm doing is finding the linear approximation between the same pixels of several frames. Then, of course, I'm doing the bit depth conversion. But it shouldn't be different between planes, right?

update2: Checked the Y plane, and it is actually getting brighter too. So, there's a general mistake somewhere in my calculations.

update3: Silly silly me. I forgot to calculate xsum. Fixed. Now need to test with different bit depths and maybe optimize a little.

[Myrsloik]

Here's R21 RC2 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r21-RC2.exe).

It will probably be released with no further changes. This final RC is just to see so I didn't break anything when moving some code around. It also has a fix for an issue no one ever noticed.

[Mystery Keeper]

Ok. Now something unrelated seems to have popped up.
ret = core.fmtc.bitdepth(ret, bits=16)
ret = core.tla.TempLinearApproximate(ret, radius=10, outBits=8)
Gives me an error in AvsPmod: AVISource: couldn't locate a decompressor for fourcc P016.
Both RC1 and RC2.

[Myrsloik]

Thank you. Installed R21 RC, and it has become fast.
This plugin interpolates between frames within (radius) using linear approximation/regression.

update: Compared unfiltered and filtered single planes. Filtered look like they have something like valid output, but much brighter (twice as bright?). That shouldn't be happening though, since all I'm doing is finding the linear approximation between the same pixels of several frames. Then, of course, I'm doing the bit depth conversion. But it shouldn't be different between planes, right?

update2: Checked the Y plane, and it is actually getting brighter too. So, there's a general mistake somewhere in my calculations.

update3: Silly silly me. I forgot to calculate xsum. Fixed. Now need to test with different bit depths and maybe optimize a little.

I noticed some things in your code:

1. VS doesn't allow less than 8 bits per sample to keep things simple so if((outBitsPerSample < 1) || (outBitsPerSample > 16))
;
is wrong.

2. You are NEVER supposed to created your own VSFormat struct copies.
WRONG!
internalData.format = *internalData.videoInfo.format;
internalData.videoInfo.format = &internalData.format;
You have to keep the pointer around. If you need to get a VSFormat describing a certain format either use getFormatPreset or registerFormat.

Your format error is because the output is 16 bit and you have nothing that can play it. Convert it to 8 bit to preview.

[Mystery Keeper]

ret = core.tla.TempLinearApproximate(ret, radius=10, outBits=8)
It does convert to 8bit.
Well, I'll try to register format properly. Haven't found an example of how to do that properly so far.

[Myrsloik]

http://i.imgur.com/V05gviR.png

Python get crashed when using the argument prop_src?

I found the typo. Will put up RC3 soon.

[Myrsloik]

ret = core.tla.TempLinearApproximate(ret, radius=10, outBits=8)
It does convert to 8bit.
Well, I'll try to register format properly. Haven't found an example of how to do that properly so far.

The relevant lines from ShufflePlanes (https://github.com/vapoursynth/vapoursynth/blob/master/src/core/simplefilters.c#L527). It's very simple. You enter all the parameters for the format you want and then you get a VSFormat * back you can use in VSVideoInfo (or anywhere else, really).

[Mystery Keeper]

Fixed the format. Works well so far. Still there are things to do, but hopefully shall release soon.
http://paste.org.ru/?vaevta

[Myrsloik]

Another problem, I am unable to get result from both assvapour.AssRender and assvapour.Subtitle. In the shell, it just stays there. No any error message. No crash.


It's not stuck. AssVapour uses fontconfig which needs to create a font index the first run. It's very slow. Give it up to 5 minutes. (or possibly more)

[Myrsloik]

Here's R21 RC3 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r21-RC3.exe). Fixes the prop_src crash and checks so a proper VSFormat is passed to setVideoInfo().

Fixed the format. Works well so far. Still there are things to do, but hopefully shall release soon.
http://paste.org.ru/?vaevta

1. I've decided that all argument names should be lowercase and with underscores. So outBits => out_bits.

2. I also specify the planes to process as an int array in all internal filters. See Lut (http://www.vapoursynth.com/doc/functions/lut.html) which is called like this: Lut(clip, planes=[1,2], long_lut_array).

3. You probably don't want to have this condition uint32_t outValue;
if(inBytesPS == 2)
outValue = ((uint16_t *)srcp)[w];
else
outValue = srcp[w]; in the inner loop. Accept that you may need to write the outer loop several times instead. Again see how MaskedMerge works.

There's also the obivious question of why this filter is doing bitdepth conversion at all. Especially downconversion by truncation probably isn't what most people really want. Maybe you should leave that to fmtconv?

Upconversion from 8 to 16 bit is also extremely cheap to do in a separate filter placed in front of TLA so maybe drop that complexity?

[Mystery Keeper]

Yup, you're right. I did normalization of the source values, and I shouldn't. Double precision should be enough for 16-bit calculations on any reasonable radius. If it isn't - normalization wouldn't help anyway. So I'll drop normalization, bit depth conversion and format change altogether.

[Mystery Keeper]

Here you go. As fast as it gets. Still a draft, but usable.
http://paste.org.ru/?h6sf34

[Myrsloik]

Here you go. As fast as it gets. Still a draft, but usable.
http://paste.org.ru/?h6sf34

1. You can drop the (internalData.videoInfo->numFrames != 0) check. Requesting frames beyond the end is allowed (you'll get the last actual frame back)

2. You're calling vsapi->getReadPtr() in the inner loop, you probably want to store the pointers in a local array before the loop

3. You can automatically copy planes using newVideoFrame2() (which is free since it uses some reference counting behind the scenes). See ShufflePlanes as an example of how it's used. Basically you pass an array of 3 frame and another of the planes that should be copied from them. If you pass 0 as the frame pointer you get a normal blank plane.

[Mystery Keeper]

1. You can drop the (internalData.videoInfo->numFrames != 0) check. Requesting frames beyond the end is allowed (you'll get the last actual frame back)
I'm afraid I can't. This plugin needs to know the exact number of frames so it would take the proper range near the end.

[IanB]

I assume zero length clips are valid in VapourSynth as they are in Avisynth.

Zero length clips need to be handled gracefully.

[Tima]

I assume zero length clips are valid in VapourSynth as they are in Avisynth.

Zero length clips need to be handled gracefully.

Another corner case could be nonzero-length clips with both (or just one!) dimensions being zero :)

[Myrsloik]

I assume zero length clips are valid in VapourSynth as they are in Avisynth.

Zero length clips need to be handled gracefully.

If the length is zero it means that it is unknown. The actual number has to be determined some other way. Call it an esoteric feature. It can be useful for streaming and such. If you request a frame beyond the end all filters should return the last valid frame.

Clips that actually have 0 frames aren't allowed and can't be created for obvious reasons.

As for clips either both or no dimensions are known. You will trigger a fatal error if you try to set only one of width and height to zero.

I have plenty of extra argument checks going on to stop some common mistakes from happening. If VapourSynth ever "just quits" run it from the commandline to see the stderr output. It usually prints the reason and offending filter (if any).

[Myrsloik]

I finally released R21. Changelog in the first post. Download link on the website.

Blog post here (http://www.vapoursynth.com/2013/10/r21-big-improvements-again/).

Same procedure as every time... Now for some plugin writing.

Download link now fixed.

[Mystery Keeper]

Great work, Myrsloik! Is there MVTools among the priority plugins?

[Myrsloik]

Great work, Myrsloik! Is there MVTools among the priority plugins?

Of course it is. I actually do write these things down on the bug tracker and blog...

I don't know if there is any other really popular plugin left to port. I think I got all the extremely useful ones but correct me if I'm wrong.

[sl1pkn07]

Sangnom for example?

Greetings

[Myrsloik]

Sangnom for example?

Greetings

I suppose I could have a go at it now that tp7 has rewritten it. Should be fairly easy. The other thing is adding the remaining missing modes to removegrain which tp7 also reverse engineered.

Or someone could help by contributing a port. There's no shortage of small things to do.

[aegisofrime]

Of course it is. I actually do write these things down on the bug tracker and blog...

I don't know if there is any other really popular plugin left to port. I think I got all the extremely useful ones but correct me if I'm wrong.

Maybe it's just me using QTGMC a lot, but I recall you mentioned that you would like having all of QTGMC's plugins on native code?

[Reel.Deel]

While I doubt that all of these will come into fruition, I think these plugins make a nice addition to VapourSynth:

AddGrain (http://forum.doom9.org/showthread.php?t=111849) - Used by QTGMC and other scripts like GrainFactory3 (http://forum.doom9.org/showpost.php?p=1191292&postcount=30).

aWarpSharp2 (http://forum.doom9.org/showthread.php?t=147285) - A rewrite of aWarpSharp.

dfttest (http://forum.doom9.org/showpost.php?p=1386559&postcount=3) - Excellent spatial/temporal denoiser. Included in the Dither package.

ExpLabo (http://expsat.sourceforge.net/) - Creates neat looking color effects.

MedianBlur (http://forum.doom9.org/showthread.php?t=84636) - Kinda popular plugin that I've seen used in denoising, sharpening, and film restoration scripts.

RemoveGrainHD (http://chaosking.de/wp-content/uploads/avsfilters/Denoisers/Spatial_Denoisers/RemoveGrainHD___(0.5_-_2011-08-11).7z) - While not nearly as popular as RemoveGrain, it's still used in scripts here and there.

RemoveDirt (http://home.arcor.de/kassandro/prerelease/RemoveDirt.rar) - Very useful for film restoration.

VerticalCleaner (http://videoprocessing.fr.yuku.com/sreply/651/Can-use-quantile-like-vertical-median-filter#.UnED91PpdvY) - Also used by QTGMC and do other neat things like this (http://forum.doom9.org/showthread.php?p=1514235).


Closed source plugins that hopefully get ported or get a VS equivalent.


AutoGain (http://forum.doom9.org/showthread.php?t=167573) - High quality auto-leveling plugin.

FrFun7 (http://forum.doom9.org/showthread.php?t=110200) - Seems to be effective against dot crawl (http://forum.doom9.org/showpost.php?p=1584186&postcount=62). I know I'm probably wasting my time here. :)

SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971) - High quality color correction plugin.

I can probably think of a few more but this is all I have time for at the moment. If I don't get completely shot down I can add some more potential plugins later. ;)

I finally released R21.
Hey there Myrsloik, thank you for the updates!

[Myrsloik]

While I doubt that all of these will come into fruition, I think these plugins make a nice addition to VapourSynth:

AddGrain (http://forum.doom9.org/showthread.php?t=111849) - Used by QTGMC and other scripts like GrainFactory3 (http://forum.doom9.org/showpost.php?p=1191292&postcount=30).

aWarpSharp2 (http://forum.doom9.org/showthread.php?t=147285) - A rewrite of aWarpSharp.

dfttest (http://forum.doom9.org/showpost.php?p=1386559&postcount=3) - Excellent spatial/temporal denoiser. Included in the Dither package.

ExpLabo (http://expsat.sourceforge.net/) - Creates neat looking color effects.

MedianBlur (http://forum.doom9.org/showthread.php?t=84636) - Kinda popular plugin that I've seen used in denoising, sharpening, and film restoration scripts.

RemoveGrainHD (http://chaosking.de/wp-content/uploads/avsfilters/Denoisers/Spatial_Denoisers/RemoveGrainHD___(0.5_-_2011-08-11).7z) - While not nearly as popular as RemoveGrain, it's still used in scripts here and there.

RemoveDirt (http://home.arcor.de/kassandro/prerelease/RemoveDirt.rar) - Very useful for film restoration.

VerticalCleaner (http://videoprocessing.fr.yuku.com/sreply/651/Can-use-quantile-like-vertical-median-filter#.UnED91PpdvY) - Also used by QTGMC and do other neat things like this (http://forum.doom9.org/showthread.php?p=1514235).


Closed source plugins that hopefully get ported or get a VS equivalent.


AutoGain (http://forum.doom9.org/showthread.php?t=167573) - High quality auto-leveling plugin.

FrFun7 (http://forum.doom9.org/showthread.php?t=110200) - Seems to be effective against dot crawl (http://forum.doom9.org/showpost.php?p=1584186&postcount=62). I know I'm probably wasting my time here. :)

SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971) - High quality color correction plugin.

I can probably think of a few more but this is all I have time for at the moment. If I don't get completely shot down I can add some more potential plugins later. ;)


Hey there Myrsloik, thank you for the updates!

I did a quick survey of the source code for the filters you listed.

The only ones that can be salvaged for a somewhat easy port are:

AddGrainC - not too horrible once you delete the asm
dfttest - also has a C version so it's doable
ExpLab - seemed very straightforward, the biggest challenge would be to structure the pile of arguments in a nicer way

The rest are INLINE ASM MACRO PREPROCESSOR HELL. It's easier to start over and my duty as a mediocre coder to do so. Most of them don't have C versions of most functions.

It's simply faster to read the filter description and then guess what should be done. Avisynth filter writers love to implement different kinds of median filters badly... in asm. With macros.

Anyway, AddGrainC is a decent first porting project for anyone who's interested.

[Mystery Keeper]

By the way, VapourSynth keeps track of scene changes, right? Or does something need to be done for that? Is it possible to process a frame with different filter if it has scene change on both sides (flicker) within the script? Example please?

[jackoneill]

By the way, VapourSynth keeps track of scene changes, right? Or does something need to be done for that? Is it possible to process a frame with different filter if it has scene change on both sides (flicker) within the script? Example please?

There are two reserved properties (http://www.vapoursynth.com/doc/apireference.html#reserved-frame-properties). I'm not sure exactly how they're supposed to be used. I guess "_SceneChangePrev" should be set on the first frame of a scene, and "_SceneChangeNext" should be set on the last frame. Wouldn't "_SceneStart" and "_SceneEnd" make more sense?

Obviously VapourSynth doesn't magically add these properties to the right frames for you. Some filter needs to find the scene changes and set the properties accordingly.

[sl1pkn07]

hi, is possible port this script (the famous insertsign.avsi)


function insertsign(clip mainclip, clip overlayclip, int startframe, int "endframe") {
endframe = default(endframe,startframe+overlayclip.framecount()-1)
endframe = (endframe == 0) ? startframe+overlayclip.framecount()-1 : endframe
endframe = (endframe >= mainclip.framecount()-1) ? mainclip.framecount()-1 : endframe

begin= (startframe == 1) ? mainclip.trim(0,-1) : mainclip.trim(0,startframe-1)
middle= mainclip.trim(startframe,endframe)
end= (endframe == mainclip.framecount()-1) ? blankclip(mainclip,length=0) : mainclip.trim(endframe+1,0)

middleoverlay = Overlay(middle, overlayclip, mask=overlayclip.showalpha())

final = (startframe == 0) ? middleoverlay ++ end : begin ++ middleoverlay ++ end
return final
}


to vapoursynth?

greetings

[Mystery Keeper]

I request a function which loads all plugins in specified (not predefined) folder.

[sneaker_ger]

Can't you do that in Python?

[Myrsloik]

By the way, VapourSynth keeps track of scene changes, right? Or does something need to be done for that? Is it possible to process a frame with different filter if it has scene change on both sides (flicker) within the script? Example please?

My head has finally recovered. Here's a simple example that uses Chikuzen's scenechange plugin:
import vapoursynth as vs
import functools

def select_processing(n, f, original_clip, extra_processed_clip, core):
if f.props._SceneChangeNext and f.props._SceneChangePrev:
return extra_processed_clip
else:
return original_clip

core = vs.get_core()
source = core.ffms2.Source(source='D:/dl/Super Size Me/Super Size Me.avi')
source = core.scd.Detect(source)
source = core.text.FrameProps(source)
extra_processed_clip = core.std.BlankClip(source, color=[255,0,255]) # set the special frames to a distinct color

final_clip = core.std.FrameEval(source, functools.partial(select_processing, original_clip=source, extra_processed_clip=extra_processed_clip, core=core), prop_src=source)
final_clip.set_output()

[Mystery Keeper]

Thank you very much.

[Reel.Deel]

I did a quick survey of the source code for the filters you listed.

The only ones that can be salvaged for a somewhat easy port are:

AddGrainC - not too horrible once you delete the asm
dfttest - also has a C version so it's doable
ExpLab - seemed very straightforward, the biggest challenge would be to structure the pile of arguments in a nicer way.

Glad to hear that at least some are portable. Thanks for taking a look.
Another one I was going to mention was Vinverse, but I see that lachs0r already ported it (https://github.com/vapoursynth/vapoursynth/tree/master/src/filters/vinverse). :)

The rest are INLINE ASM MACRO PREPROCESSOR HELL. It's easier to start over and my duty as a mediocre coder to do so. Most of them don't have C versions of most functions.

It's simply faster to read the filter description and then guess what should be done. Avisynth filter writers love to implement different kinds of median filters badly... in asm. With macros

Maybe I'm missing something here but RemoveGrainHD's documentation says that it's written in "ordinary C/C++".
While RemoveGrain uses a very high level of parallelism - the SSE2/SSE3 version processes 16 pixels simultaneously - this is unfortunately no more possible for RemoveGrainHD. It is a rather ordinary C/C++ program without any inline assembler code.

If that documentation is wrong, please excuse my ignorance.

And finally, someone started (https://github.com/handaimaoh/vsremovedirt) to port RemoveDirt to VS. :)

------
Wouldn't "_SceneStart" and "_SceneEnd" make more sense?
I agree, _SceneStart/End is definitely more straightforward.

[sl1pkn07]

I found other plugin for vapoursynth

https://github.com/gnaggnoyil/VAutoDeint

greetings

[Myrsloik]

I found other plugin for vapoursynth

https://github.com/gnaggnoyil/VAutoDeint

greetings

Another secret plugin. Interesting.

Anyway, about the scenechange naming. I probably won't change it now since it's being used. I also think it makes more sense since in my world a scenechange happens between two frames. Not on a frame.

I'm busy converting the whole project to C++11 now and I've made quite a bit of progress. Qt has already been 80% removed from the codebase since C++11 finally gets a somewhat complete standard library.

[Are_]

I found other plugin for vapoursynth

https://github.com/gnaggnoyil/VAutoDeint

greetings

Unfortunately it's so Windows centric. It also performs plugin loading within the python module using filesystem paths (too bad, because the plugin itself compiles under Linux).

[sl1pkn07]

others (?)

https://github.com/gnaggnoyil/tc2cfr
https://github.com/4re/vapoursynth-modules

[lansing]

I just brought an i7 4470k and ran some speed test comparison between avisynth mt on the d2v source filter and tivtc/vivtc.

source was a 720x480 anime, all benchmark were measured by avsmeter.


avisynth-mt(fps)/cpu% vapoursynth(fps)/cpu%

d2v (mode 5)308/12% 912/12%
tfm (mode 2)215/17% 330/12%
tfm+tdecimate (mode 5)157/12% 255/12%
vfm 253/12%
vfm+vdecimate 31/13%




With vapoursynth, cpu was never fully utilized, not even 30%. And there's definitely something wrong with vdecimate, as running it alone also gives me 60fps.

[Myrsloik]

I just brought an i7 4470k and ran some speed test comparison between avisynth mt on the d2v source filter and tivtc/vivtc.

source was a 720x480 anime, all benchmark were measured by avsmeter.


avisynth-mt(fps)/cpu% vapoursynth(fps)/cpu%

d2v (mode 5)308/12% 912/12%
tfm (mode 2)215/17% 330/12%
tfm+tdecimate (mode 5)157/12% 255/12%
vfm 253/12%
vfm+vdecimate 31/13%




With vapoursynth, cpu was never fully utilized, not even 30%. And there's definitely something wrong with vdecimate, as running it alone also gives me 60fps.

That's quite odd indeed. I'll try to figure out why vdecimate makes it so slow.

[Myrsloik]

Here are two alternative vivtc dlls you can test:
Test1 (https://dl.dropboxusercontent.com/u/73468194/vivtc_test1.dll)
Test2 (https://dl.dropboxusercontent.com/u/73468194/vivtc_test2.dll)

Both are completely untested but should perform slightly better.

[lansing]

I did the test with the vpy script through using the VSimport wrapper (http://forum.doom9.org/showthread.php?t=168339), seems that it slows down the speed quite a lot, compared to simply threw the vpy script into virtualdub and ran analysis pass.

with wrapper:
R21 --> 30
test1 --> 35
test2 --> crash


without wrapper:
R21 --> 40
test1 --> 65
test2 --> crash

[Myrsloik]

I did the test with the vpy script through using the VSimport wrapper (http://forum.doom9.org/showthread.php?t=168339), seems that it slows down the speed quite a lot, compared to simply threw the vpy script into virtualdub and ran analysis pass.

with wrapper:
R21 --> 30
test1 --> 35
test2 --> crash


without wrapper:
R21 --> 40
test1 --> 65
test2 --> crash

If you want to test speed I really suggest you use vspipe as it will almost always have better throughput. It also shows the fps at the end now.

I'll take a serious look at it later tonight. It should be almost as fast as tdecimate at least.

[lansing]

If you want to test speed I really suggest you use vspipe as it will almost always have better throughput. It also shows the fps at the end now.

I'll take a serious look at it later tonight. It should be almost as fast as tdecimate at least.

I couldn't find any documentation on using vspipe on your site. I only know the most basic:


vspipe "clip.vpy" - -y4m | x264 --preset fast --demuxer y4m --output "clip.mkv" -

[jackoneill]

I couldn't find any documentation on using vspipe on your site.


It prints usage instructions if you run it with no parameters. Use NUL as output file.

[lansing]

It prints usage instructions if you run it with no parameters. Use NUL as output file.

Thanks, I got it working now

vspipe "clip.vpy" NUL


It's not so user friendly though, as it only showed the fps after the whole process was finished. not during the run.

[Mystery Keeper]

https://bitbucket.org/mystery_keeper/templinearapproximate-vapoursynth
Mature enough, or still needs improvements?

[sl1pkn07]

Hi Mystery Keeper

what is stdbool.h?

http://sl1pkn07.no-ip.com/paste/view/97611430

howto build? "gcc -o templinearapproximate.so main.c -I/usr/include/vapoursynth -I"put here stdbool.h path" ?

greetings

[LoRd_MuldeR]

what is stdbool.h?

Support for the "bool" type as well as the "true" and "false" constants in C language:
http://pubs.opengroup.org/onlinepubs/009696699/basedefs/stdbool.h.html

It's required, because C did not have a "bool" type before C99. Legacy code might still use "bool", "true" and "false" for other purposes, since they were not reserved before C99.

So if you want C++-style booleans in C, you have to include <stdbool.h> or just define those three macros yourself. MSVC still doesn't support C99 ;)

[Myrsloik]

https://bitbucket.org/mystery_keeper/templinearapproximate-vapoursynth
Mature enough, or still needs improvements?

https://bitbucket.org/mystery_keeper/templinearapproximate-vapoursynth/src/9f90c3faa3af9446c16a70cc7789357b591174e8/src/main.c?at=master#cl-196

The variable frames and planes can be allocated on the stack as an array of size 3. There will never be more than 3 planes in a format.

You should be able to eliminate most other allocations as well using variable length arrays since you went with C99.


https://bitbucket.org/mystery_keeper/templinearapproximate-vapoursynth/src/9f90c3faa3af9446c16a70cc7789357b591174e8/src/main.c?at=master#cl-208

You should pass input frame n instead of NULL. Otherwise all the existing properties such as colorimetry and times won't be copied over to your output frame.


"planes:int[]: opt:empty;" <- you probably don't want empty there, it's most likely an error if a user wants to filter no planes


https://bitbucket.org/mystery_keeper/templinearapproximate-vapoursynth/src/9f90c3faa3af9446c16a70cc7789357b591174e8/src/main.c?at=master#cl-114

"tla" should be "TempLinearApproximate". It makes more sense if the name reported when an error happens also matches the function used to create the filter.

That's all I could see. The only other small comment I have that should probably be ignored is that this would be a good function to implement with templates in C++ to avoid repeating it.

[sl1pkn07]

oks, build with

gcc -shared -fPIC -std=c99 -o templinearapproximate.so main.c -I/usr/include/vapoursynth

missing -shared and -fPIC

sorry. i'm not coder

greetings and thanks