TFM() and DGSource() for VapourSynth?

[CarlEdman]

After having used AviSynth for a decade for my video edits, I just discovered VapourSynth and it looks fantastic. Much more modern and compatible and without all the pain and suffering that the various MT/64-bit AviSynth versions involved. Kudos to everybody involved!

However, while a ton of my filters appear to have been converted, there seem to be two filters I really need that don't exist for VapourSynth yet.

(1) For teleciding I use tfm.tdecimate(hybrid=1). Back when I tested all the various teleciders for AviSynth it seemed to be the best and has performed well for all these years. It does the right thing even if the cadence is off and a creditable approximation when there are--as occasionally happens--some genuine interlaced sequences intermixed with the telecined ones.

Is VIVTC an adequate replacement?

(2) My source filter is always DGSource(). I saw that there was a d2vsource() filter which is great, but for modern codecs like h264, you need DGIndexNV/DGSource().

Any work on this or alternatives? FFMS2 could work, I guess, but I do not trust its accuracy as I do DGSource().

[mawen1250]

VFM works much the same as TFM. But compared to TDecimate, VDecimate lacks some important functions (such as vfr mode).

As for source filter:
1. For ts/m2ts input, lsmas.LWLibavSource is a good choice, it's safer than ffms2 when non-linear access occurs.
2. For mp4 input, lsmas.LibavSMASHSource definitely is the best choice.
3. For mkv input, both lsmas and ffms2 are OK.

If you really want to use those AviSynth plugins, it's possible to load them with 32bit VS.
Alternatively, you can try using them in an AviSynth script and loading it with vsavsreader (http://forum.doom9.org/showthread.php?t=165957).

[videoh]

If someone can point me to a guide or documentation for porting filters to VapourSynth, I would consider porting DGDecNV/IM.

[jackoneill]

If someone can point me to a guide or documentation for porting filters to VapourSynth, I would consider porting DGDecNV/IM.

The API documentation (http://www.vapoursynth.com/doc/api/vapoursynth.h.html#writing-plugins) contains some words about writing plugins.

There are lots of filters you can look at for inspiration:
https://github.com/vapoursynth/vapoursynth/tree/master/src/filters
https://github.com/vapoursynth/vapoursynth/tree/master/src/core
In particular, BlankClip (https://github.com/vapoursynth/vapoursynth/blob/master/src/core/simplefilters.c#L1030), because it's a simple source filter.

If you have any questions, feel free to ask.

[Myrsloik]

Don't forget the invert example (https://github.com/vapoursynth/vapoursynth/blob/master/sdk/invert_example.c) which has comments explaining many things.

Otherwise I think blankclip is the best place to start looking since it's a source filter.

[videoh]

Thanks, gents, I'll get cracking on it.

[sl1pkn07]

please. make a miltiplatform plugin. if not, we gain nothing

[CarlEdman]

Thanks, that would be great, videoh!

[videoh]

please. make a miltiplatform plugin. if not, we gain nothing What do you mean by multiplatform?

[sl1pkn07]

can build on linux/macos. no only windows-centric

[Myrsloik]

...
(1) For teleciding I use tfm.tdecimate(hybrid=1). Back when I tested all the various teleciders for AviSynth it seemed to be the best and has performed well for all these years. It does the right thing even if the cadence is off and a creditable approximation when there are--as occasionally happens--some genuine interlaced sequences intermixed with the telecined ones.

Is VIVTC an adequate replacement?


There's a simple reason it doesn't do hybrid things the way you want. It's OBSOLETE. For the past 10+ years (http://forum.doom9.org/showthread.php?t=64440) I've tried to get everyone to embrace VFR. It's really the right thing to do even if it requires a little bit of extra work.

But since you pointed it out I'll make vdecimate export more information so you could create something similar with a script, if that's what you want.

[CarlEdman]

@Myrsloik You are right in principle--VFR really is the way to go with material that genuinely is a mixture between interlaced and telecined material.

Fortunately, these days almost all the material I work on is already either constant 24 frames-per-second progressive or 60 fields-per-second interlaced. Almost all than remains is almost purely 24 frames-per-second telecined to 60 fields-per-second, with no or almost no interlaced material. All of those convert well to constant 24 or 30 progressive frames-per-second. The tiny fraction (about 0% for me in recent years) that does not fit into these categories might convert better to VFR, but then I'd start to worry about compatibility with all my devices and players.

[videoh]

can build on linux/macos. no only windows-centric Ouch, CUVID is Windows only AFAIK. So I would have to re-engineer for VDPAU. And who knows about Mac?

So is there any interest in a Windows-only port?

[sl1pkn07]

In my knowledge, CUVID It exists for linux


/usr/lib/libnvcuvid.so
/usr/lib/libnvcuvid.so.1
/usr/lib/libnvcuvid.so.352.09

/opt/cuda/include/cuviddec.h
/opt/cuda/include/nvcuvid.h
/opt/cuda/samples/3_Imaging/cudaDecodeGL/doc/nvcuvid.pdf


if build '3_Imaging/cudaDecodeGL' sample in CUDA SDK 7.0.28


└───╼ sudo GLPATH=/usr/lib make
[sudo] password for sl1pkn07:
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o FrameQueue.o -c FrameQueue.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o ImageGL.o -c ImageGL.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o VideoDecoder.o -c VideoDecoder.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o VideoParser.o -c VideoParser.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o VideoSource.o -c VideoSource.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o cudaModuleMgr.o -c cudaModuleMgr.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o cudaProcessFrame.o -c cudaProcessFrame.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o videoDecodeGL.o -c videoDecodeGL.cpp
"/opt/cuda"/bin/nvcc -ccbin g++ -m64 -gencode arch=compute_20,code=compute_20 -o cudaDecodeGL FrameQueue.o ImageGL.o VideoDecoder.o VideoParser.o VideoSource.o cudaModuleMgr.o cudaProcessFrame.o videoDecodeGL.o -L../../common/lib/linux/x86_64 -lGL -lGLU -lX11 -lXi -lXmu -lglut -lGLEW -lcuda -lcudart -lnvcuvid
mkdir -p ../../bin/x86_64/linux/release
cp cudaDecodeGL ../../bin/x86_64/linux/release
"/opt/cuda"/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=compute_20 -o NV12ToARGB_drvapi64.ptx -ptx NV12ToARGB_drvapi.cu
mkdir -p data
cp -f NV12ToARGB_drvapi64.ptx ./data
mkdir -p ../../bin/x86_64/linux/release
cp -f NV12ToARGB_drvapi64.ptx ../../bin/x86_64/linux/release

see the -lnvcuvid

if run the sample:

└───╼ ./cudaDecodeGL
[CUDA/OpenGL Video Decode]
Command Line Arguments:
argv[0] = ./cudaDecodeGL
[cudaDecodeGL]: input file: <./data/plush1_720p_10s.m2v>
VideoCodec : MPEG-2
Frame rate : 30000/1001fps ~ 29.97fps
Sequence format : Progressive
Coded frame size: [1280, 720]
Display area : [0, 0, 1280, 720]
Chroma format : 4:2:0
Bitrate : 14116kBit/s
Aspect ratio : 16:9


argv[0] = ./cudaDecodeGL

> Device 0: < GeForce GTX 770 >, Compute SM 3.0 detected
>> initGL() creating window [1280 x 720]
> Using CUDA/GL Device [0]: GeForce GTX 770
> Using GPU Device: GeForce GTX 770 has SM 3.0 compute capability
Total amount of global memory: 2039.7773 MB
>> modInitCTX<NV12ToARGB_drvapi64.ptx > initialized OK
>> modGetCudaFunction< CUDA file: NV12ToARGB_drvapi64.ptx >
CUDA Kernel Function (0x01fca330) = < NV12ToARGB_drvapi >
>> modGetCudaFunction< CUDA file: NV12ToARGB_drvapi64.ptx >
CUDA Kernel Function (0x01fd11c0) = < Passthru_drvapi >
Free memory: 1469.9062 MB
> VideoDecoder::cudaVideoCreateFlags = <1>Use CUDA decoder

setTextureFilterMode(GL_NEAREST,GL_NEAREST)
ImageGL::CUcontext = 01d5b7c0
ImageGL::CUdevice = 00000000
reshape() glViewport(0, 0, 1280, 720)
[cudaDecodeGL] - [Frame: 0016, 00.0 fps, frame time: 89543860224.00 (ms) ]
[cudaDecodeGL] - [Frame: 0032, 804.6 fps, frame time: 1.24 (ms) ]
[cudaDecodeGL] - [Frame: 0048, 654.3 fps, frame time: 1.53 (ms) ]
[cudaDecodeGL] - [Frame: 0064, 683.3 fps, frame time: 1.46 (ms) ]
[cudaDecodeGL] - [Frame: 0080, 657.7 fps, frame time: 1.52 (ms) ]
[cudaDecodeGL] - [Frame: 0096, 637.4 fps, frame time: 1.57 (ms) ]
[cudaDecodeGL] - [Frame: 0112, 637.5 fps, frame time: 1.57 (ms) ]
[cudaDecodeGL] - [Frame: 0128, 599.1 fps, frame time: 1.67 (ms) ]
[cudaDecodeGL] - [Frame: 0144, 627.6 fps, frame time: 1.59 (ms) ]
[cudaDecodeGL] - [Frame: 0160, 686.3 fps, frame time: 1.46 (ms) ]
[cudaDecodeGL] - [Frame: 0176, 598.0 fps, frame time: 1.67 (ms) ]
[cudaDecodeGL] - [Frame: 0192, 548.1 fps, frame time: 1.82 (ms) ]
[cudaDecodeGL] - [Frame: 0208, 516.1 fps, frame time: 1.94 (ms) ]
[cudaDecodeGL] - [Frame: 0224, 603.4 fps, frame time: 1.66 (ms) ]
[cudaDecodeGL] - [Frame: 0240, 725.9 fps, frame time: 1.38 (ms) ]
[cudaDecodeGL] - [Frame: 0256, 747.2 fps, frame time: 1.34 (ms) ]
[cudaDecodeGL] - [Frame: 0272, 753.0 fps, frame time: 1.33 (ms) ]
[cudaDecodeGL] - [Frame: 0288, 733.6 fps, frame time: 1.36 (ms) ]
[cudaDecodeGL] - [Frame: 0304, 715.5 fps, frame time: 1.40 (ms) ]
[cudaDecodeGL] - [Frame: 0320, 656.2 fps, frame time: 1.52 (ms) ]

[cudaDecodeGL] statistics
Video Length (hh:mm:ss.msec) = 00:00:00.502
Frames Presented (inc repeats) = 329
Average Present Rate (fps) = 654.13
Frames Decoded (hardware) = 329
Average Rate of Decoding (fps) = 654.13


then i supposed is possible can use CUVID in linux

http://docs.nvidia.com/cuda/samples/3_Imaging/cudaDecodeGL/doc/nvcuvid.pdf

for mac users, i'm not completely sure if cuvid works

greeetings

[captainadamo]

For OS X you'd probably want to use Video Toolbox. Its benefit is also not being tied to any specific hardware vendor. But obviously that makes videoh's porting effort much more difficult.

I've been planning to create such a plugin using it already, but I've just not had the time.

[Asmodian]

Ouch, CUVID is Windows only AFAIK. So I would have to re-engineer for VDPAU. And who knows about Mac?

So is there any interest in a Windows-only port?

I would be quite interested in a Windows-only port, if it was compatible with 64-bit VapourSynth. :)

[stax76]

Is there any HW decoder for VapourSynth x64 on Win10? I'm sure many users are interested especially when there are more 4K sources in the future.

[CarlEdman]

So is there any interest in a Windows-only port?
Any progress on the "promised" DgSource filter for VapourSynth, even if it is Windows-only? As its lack is my only obstacle to switching to VapourSynth from avisynth, I am eagerly anticipating it!

[videoh]

Is there any kind of wrapper in Vaporsynth to use existing Avisynth source filters?

[sneaker_ger]

Write an AviSynth script and use VapourSynth's native AviSource or vsavsreader (http://forum.doom9.org/showthread.php?t=165957).

Or do you mean a wrapper to create a VapourSynth compatible plugin?

[jackoneill]

32 bit VapourSynth for Windows can load many 32 bit Avisynth 2.5 plugins. http://www.vapoursynth.com/doc/functions/loadpluginavs.html

[Reel.Deel]

Since DGSource cannot be loaded in 64-bit VapourSynth I use AVFS (http://www.turtlewar.org/avfs/) to create a fake avi and then load it with AviSource in VapourSynth. Is not as nice as having DGSource natively but this works rather well.

[videoh]

First mistake: I installed Python 3.5 and Vapoursynth barfed on it. Why is Vapoursynth so picky about it?

[Myrsloik]

First mistake: I installed Python 3.5 and Vapoursynth barfed on it. Why is Vapoursynth so picky about it?

Because python is horrible and has no binary compatibility between different versions for modules on windows. That's why. R27 only comes with 3.4 support and it'll take a few more days before my 3.5 support is completely ready.

[videoh]

Making progress (I think). I can type these lines in the interpreter and they all succeed.

core = vs.get_core()
core.std.LoadPlugin('.../ffms2.dll')
ret = core.ffms2.Source(source='file.avi')
ret.set_output()

If I put them into a .py file then how do I open the script in Vdub 64?

Sorry for my ignorance but if y'all can help me y'all will get DGDecNV for Vapoursynth much faster. Thank you.

[Myrsloik]

Making progress (I think). I can type these lines in the interpreter and they all succeed.

core = vs.get_core()
core.std.LoadPlugin('.../ffms2.dll')
ret = core.ffms2.Source(source='file.avi')
ret.set_output()

If I put them into a .py file then how do I open the script in Vdub 64?

Sorry for my ignorance but if y'all can help me you will get DGDecNV for Vapoursynth much faster. Thank you.

You use .vpy as the extension and it'll work in vdub.

[videoh]

Sweet, it works. :)

Off to look at BlankClip and how source filters work...

[videoh]

So, is that ffms2.dll that I got from Git as linked at the Vapoursynth page special for Vapoursynth or is it a standard Avisynth one? The GitHub doesn't look dedicated to Vapousynth and in fact says Avisynth! I am so confused. :confused:

[Myrsloik]

So, is that ffms2.dll that I got from Git as linked at the Vapoursynth page special for Vapoursynth or is it a standard Avisynth one? The GitHub doesn't look dedicated to Vapousynth and in fact says Avisynth! I am so confused. :confused:

It's a combined plugin so it works in both.

[videoh]

That's awesome sauce. I'll make mine that way too. :D

[Boulder]

Sorry for my ignorance but if y'all can help me y'all will get DGDecNV for Vapoursynth much faster. Thank you.You probably don't need to worry about that, I asked DG on his forum and he said that he's going to do it. I expect it to occur rather sooner than later if I read correctly between the lines. Of course, there might be unexpected obstacles but anyway.. :)

[videoh]

Pretty much common knowledge now:

videoh = DG

If that were not the case how could videoh have the source code for DGDecNV? ;)

It's your request over at my forum that got me moving on this here.

[Boulder]

Hehe, I kinda wondered why DGDecNV had suddenly become open source :)

Time for a new nick, perhaps? :D

[videoh]

Time for a new nick, perhaps? :D Yeah, I was thinking of something like 'neuron2'. :p

[Boulder]

I thought you always wanted to go forward, not backward.. :P

[videoh]

neuron3 ? :scared:

OK, let's stay on topic now.

[videoh]

Wow, Vapoursynth is highly cool and not at all hard to code for as I imagined it might be. I've already got the Vapoursynth interface in place and can open a .vpy script with DGDecodeNV.dll and DGSource() and play it in VirtualDub. It plays green frames, though, because I haven't hooked up getFrame to my decoder yet, but that should take only a day or two. Famous last words, I know, but things are progressing nicely. The only challenging thing is to avoid unnecessary code duplication for the Avisynth and Vapoursynth functionalities.

Congratulations to Myrsloik for this excellent frame server project and thanks to him also for his valuable support.

[videoh]

@Myrsloik: Can the per-frame properties be enhanced to contain arbitrary name-value pairs that an app could define? It seems to me that hard-coding the properties precludes future use cases that you may not have anticipated.

[Myrsloik]

Question for Myrsloik:

Is there some reason that the per-frame properties cannot contain arbitrary name-value pairs that an app could define?

You can set any name value pairs you want. The ones listed are only because I want to have them well defined and those names start with _. Anything goes really.

[videoh]

Got it. Thank you.

[Boulder]

While you're designing the native VS build, I have a question regarding Vapoursynth's multithreading and DGSource. Is it generally safe to allow multithreading for DGSource or is it better to use core = vs.get_core(threads=1) before it and core = vs.get_core(threads=-1) afterwards? I assume it's mostly up to the GPU driver but there's no general opinion on this matter.

[Myrsloik]

While you're designing the native VS build, I have a question regarding Vapoursynth's multithreading and DGSource. Is it generally safe to allow multithreading for DGSource or is it better to use core = vs.get_core(threads=1) before it and core = vs.get_core(threads=-1) afterwards? I assume it's mostly up to the GPU driver but there's no general opinion on this matter.

This isn't avisynth. All filters have to specify how they want to be threaded. Only the first get_core() call matters, all arguments are ignored for later calls (you should never need to use any of the arguments apart from when debugging or developing).

[Boulder]

This isn't avisynth. All filters have to specify how they want to be threaded. Only the first get_core() call matters, all arguments are ignored for later calls (you should never need to use any of the arguments apart from when debugging or developing).What about in the case of an Avisynth plugin loaded in Vapoursynth?

[Myrsloik]

What about in the case of an Avisynth plugin loaded in Vapoursynth?

It's still not avisynth.

[videoh]

I don't know enough about Vapoursynth yet to comment with respect to DGSource(). I'll let you know what I discover about it. ;)

[videoh]

Things are looking real good, guys and gals. I am playing video with random access from a .vpy script with 64-bit DGDecodeNV.dll.

The only stuff left is to change the bitblt's for pulldown to Vapoursynth ones (AVSenv->BitBlt is obviously not available) and to implement the debug overlay. Then it's done! :eek:

[Reel.Deel]

Awesome! :thanks:

[MeteorRain]

OMG that's so awesome!

[videoh]

Here is some tasty DGDecodeNV cooked Vapoursynth-style. Your test results will be appreciated. The standard DGIndexNV should be used for indexing. Note that hinting is gone for Vapoursynth as it will be converted to use Vapoursynth per-frame properties.

http://rationalqm.us/misc/DGDecodeNV_Vapoursynth.zip

This is a dual-function DLL for both Avisynth and Vapoursynth. Sample script:

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin('dgdecodenv_vs64.dll')
ret = core.dgdecodenv.DGSource('alba.dgi',debug=1)
ret.set_output()

[MeteorRain]

Here is some tasty DGDecodeNV cooked Vapoursynth-style.

Did a quick test to load a random ts file and a bluray file and it works like a charm. MPEG2 and H.264 source.