VapourSynth - AUDIO SUPPORT AND NEW API BETA

[Myrsloik]

It's finally happened!

Biggest changes:

• Audio support
• Cleaner API
• More performance

I'll provide a longer post later but things have now reached a state I'm happy with in general. Basically full backwards compatibility is provided with only a few minor caveats:

• YCoCg is no longer a separate format so any scripts that reference it will break (now fixed in all known script, simply update your things).
• Filters that output multiple clips (old api FFMS2 in alpha mode) will simply have outputs beyond the first discarded.
• Compat formats have been removed so applications that rely on them for output are broken.
• Avisynth filters now all have a secret argument called compatpack added, set it to True to pack YV16 and RGB24 before passing to the filter.
• get_read/write_array() functions have been removed in Python and replaced by a better implementation that can't accidentally cause access violations
• Histogram filter no longer bundled in windows installer since it's not a part of the VS source tree
  

Note that none of the previously compiled plugins for api4 will work so use the provided versions in this release. If you want to do speed comparisons I'd recommend using the linked FFMS2 binary which supports both old and new VS APIs. The alpha plane is now attached to the main frame and has to be extracted with PropToClip().

https://github.com/vapoursynth/vapou...g/R55-API4-RC2
https://github.com/vapoursynth/vapou...R54-API4-test1 <- get filter binaries from this release

[DJATOM]

Apparently removing COMPAT stuff broke existing script previewers
Failed to convert to RGB:
The VSVideoInfo structure passed by Spline16 is invalid.
Couldn't create preview node for output number 0.

[Myrsloik]

Originally Posted by DJATOM

Apparently removing COMPAT stuff broke existing script previewers
Failed to convert to RGB:
The VSVideoInfo structure passed by Spline16 is invalid.
Couldn't create preview node for output number 0.

Anything VFW/AVFS based still works. Also, compat formats needed to die sooner or later

[ChaosKing]

Can't test ffms2 v4 in vdub. I always get this error if click on "run video analysis pass":

An out-of-bounds memory access (access violation) occurred in module 'VirtualDub64'...
...reading address 0000000000000290.

Scrolling through the video can also lead to a crash.

Old ffms2 is ok.

Tested on Win 10 x64, Ryzen 3600 CPU. Vdub2 44282
Video source mpeg2 dvd muxxed to mkv.

EDIT
And it seems that old ffms2 is very slow in this build. The dvd clip went from 220fps to like 6fps after 2000 frames. 1080p avc clip starts with 8fps. Something seems not right.

[Myrsloik]

Originally Posted by ChaosKing

Can't test ffms2 v4 in vdub. I always get this error if click on "run video analysis pass":

An out-of-bounds memory access (access violation) occurred in module 'VirtualDub64'...
...reading address 0000000000000290.

Scrolling through the video can also lead to a crash.

Old ffms2 is ok.

Tested on Win 10 x64, Ryzen 3600 CPU. Vdub2 44282
Video source mpeg2 dvd muxxed to mkv.

EDIT
And it seems that old ffms2 is very slow in this build. The dvd clip went from 220fps to like 6fps after 2000 frames. 1080p avc clip starts with 8fps. Something seems not right.

Read the initial post: "Old api source filters that used nfMakeLinear (most of them since seeking is slooow) will perform worse until ported to the new API."

I'll see if I can figure out the crash

[Myrsloik]

Originally Posted by ChaosKing

Can't test ffms2 v4 in vdub. I always get this error if click on "run video analysis pass":

An out-of-bounds memory access (access violation) occurred in module 'VirtualDub64'...
...reading address 0000000000000290.

Scrolling through the video can also lead to a crash.

Old ffms2 is ok.

Tested on Win 10 x64, Ryzen 3600 CPU. Vdub2 44282
Video source mpeg2 dvd muxxed to mkv.

I can't reproduce the crash and I unfortunately don't have dvds muxed to mkv specifically.

[ChaosKing]

Originally Posted by Myrsloik

I can't reproduce the crash and I unfortunately don't have dvds muxed to mkv specifically.

It happens with other videos as well.

[Myrsloik]

Originally Posted by ChaosKing

It happens with other videos as well.

Found the bug. The binaries have been sneakily updated.

[ChaosKing]

No more crashes now
But there's still something wrong when using v3 api ffms2. It is painfully slow. 190 fps (api v4) vs 6fps (api v3)
I tried multiple ffms2 versions, same low fps. CPU usage seems similar to v4.

EDIT

will perform MUCH WORSE until ported to the new API.

But will it be that slow?

[Myrsloik]

Originally Posted by ChaosKing

No more crashes now
But there's still something wrong when using v3 api ffms2. It is painfully slow. 190 fps (api v4) vs 6fps (api v3)
I tried multiple ffms2 versions, same low fps. CPU usage seems similar to v4.

EDIT

But will it be that slow?

Yes, I suppose that's possible. I got ~1/15 speed with a quick qtgmc test. Basically all the logic to try to make requests linear are gone and the new (and much better) system requires additional filter support.

If relevant source filters aren't updated by the time this branch is stable and tested maybe I'll add some kind of additional workaround.

[ChaosKing]

My results (vdub - run analysis pass)

R54_test1: ffms2 v4 + qtgmc() = 170fps
R54: ffms2 v3 + qtgmc() = 185fps

EDIT:
More reliable results with vspipe (run 3 times each):

R54 ffms2 v3 1080p source - 264.8 fps
R54 ffms2 v3 + qtgmc  480p source - 182.05 fps

R54_test1 ffms2 v4 1080p source - 264.3 fps
R54_test1 ffms2 v4 + qtgmc  480p source - 204.12 fps

Now test1 is faster with qtgmc xD

[ChaosKing]

Very heavy filtering test:
haf.MCTemporalDenoise + knlm.KNLMeansCL + haf.ContraSharpening + haf.FineDehalo + nnedi3_rpow2 + MfTurd + haf.FineDehalo + haf.LSFmod + f3kdb.Deband

R54: 11.3 fps
R54: 12.11 fps (max_cache_size = 15000) (I have 64gb ram)
R54_test1: 12.5 fps

I noticed that test1 does not show the "Script exceeded memory limit. Consider raising cache size." message. So caching is now smarter?

[Myrsloik]

Originally Posted by ChaosKing

Very heavy filtering test:
haf.MCTemporalDenoise + knlm.KNLMeansCL + haf.ContraSharpening + haf.FineDehalo + nnedi3_rpow2 + MfTurd + haf.FineDehalo + haf.LSFmod + f3kdb.Deband

R54: 11.3 fps
R54: 12.11 fps (max_cache_size = 15000) (I have 64gb ram)
R54_test1: 12.5 fps

I noticed that test1 does not show the "Script exceeded memory limit. Consider raising cache size." message. So caching is now smarter?

Yes, the cache insertion is completely reworked and it wouldn't surprise me if there're only 1/4 as many caches inserted (and in better places) for scripts like qtgmc.

[feisty2]

when will the documentation for API v4 be ready?

[Myrsloik]

Originally Posted by feisty2

when will the documentation for API v4 be ready?

You can find mostly complete notes here: https://github.com/vapoursynth/vapou...%20changes.txt

For plain video filters it's mostly a question of renamed things and you should be able to convert just by looking at the filter samples.

[_Al_]

vsedit would need to be updated, it uses COMPATBGR32 for RGB conversion then using QT's QImage.Format_RGB32 to convert it to QPixmap. In Python (not C ) it uses something along like this:

img = QImage(compatbgr.get_frame(f).get_read_array(0), w, h, stride, QImage.Format_RGB32).mirrored()
pix = QPixmap.fromImage(img).scaled(scale_w, scale_h, **modes)

[Myrsloik]

Originally Posted by _Al_

vsedit would need to be updated, it uses COMPATBGR32 for RGB conversion then using QT's QImage.Format_RGB32 to convert it to QPixmap. In Python (not C ) it uses something along like this:

img = QImage(compatbgr.get_frame(f).get_read_array(0), w, h, stride, QImage.Format_RGB32).mirrored()
pix = QPixmap.fromImage(img).scaled(scale_w, scale_h, **modes)

Fortunately packing planar RGB24 is trivial so applications should get updated quickly and would still work with both. If someone wants to compile a list of all applications that break because of my changes that'd be useful. I already have a good idea about which source plugins need to be updated and I'll get to work on d2vsource tomorrow.

[DJATOM]

Afaik, https://github.com/Endilll/vapoursynth-preview and https://bitbucket.org/mystery_keeper...or/src/master/ broken, both relies on Qt and Format_RGB32 (0xffRRGGBB).

[Myrsloik]

Test2 is released. Has fixes for crashes that could occur with certain filters but otherwise no real changes.

[lansing]

Is RGB24 compatible with Virtualdub filter if I'm to load it in the script?