Vapoursynth

[kolak]

AVIsource does not seams to have any RGBA color space implemented yet.

[Myrsloik]

How can I read AVI file containing alpha with avisource?
I tried
...

sample (http://www.mediafire.com/download.php?9c4d6x05d908dxp) and information (http://pastebin.com/0kBM2PY7)

I'll investigate it. The avisource stuff still is still alpha quality, I just wanted to get it out so people would send me samples of stuff that doesn't work.

[Myrsloik]

After upgrading to r15, MDegrain crashed. :scared:
http://img7.ressim.net/out.php/i6363580_mdegrain.png

Found it. The avisynth compat stuff is broken for everything except yv12 due to some stricter checks I added. I'll release R16 soon, just want to poke avisource a bit since it also has issues.

[osgZach]

Wow, neat thread.. Neat tool (I hope)..

There are 26 pages I truly can't be bothered to read, so I'm sorry if this has been asked before (linking to answers in a previous post is quite fine)

Is this intended to replace AVIsynth (eventually?) Are there long-term development plans? (i.e its not going to become vaporware because someone gets bored) Has serious planning already been done for development that would put this at the current level of Avisynth functionality / speed, but also be able to surpass it as future development continues?

I thought I saw cross-platform mentioned.. Is this the case? (Windows/Linux/Apple is a quite attractive trifecta)

Also, any plans to develop a front-end type program for this? ala Vdub, etc.. I know we had an active project for a Python based Avisynth front-end/editing application, so that could always prove a valuable base to start with..

[Myrsloik]

Wow, neat thread.. Neat tool (I hope)..

There are 26 pages I truly can't be bothered to read, so I'm sorry if this has been asked before (linking to answers in a previous post is quite fine)

Is this intended to replace AVIsynth (eventually?) Are there long-term development plans? (i.e its not going to become vaporware because someone gets bored) Has serious planning already been done for development that would put this at the current level of Avisynth functionality / speed, but also be able to surpass it as future development continues?

I thought I saw cross-platform mentioned.. Is this the case? (Windows/Linux/Apple is a quite attractive trifecta)

Welcome to my 26th page summary post!
Is it intended to replace avisynth?
YES! But it may take another 3 months before all really useful filters are ported (mvtools excluded since it's a huge and overly complicated mess)
Is it really cross platform?
Yes, many of the people who give me the most feedback are linux users. OSX supposedly also works but none of those users report their results to me.
Speed?
Since it can usually use all cores at once it surpasses plain avisynth for speed and gets close to avisynth-mt at its best, minus the crashes.
Serious planning?
Will some papers full of notes about what needs to be done do? there's also a suggest task list for contributors. Mostly it's bug fixing now and porting more useful plugins so the step to 64bit will be less painful.
Long term development plans?
Yes, I guess so, doesn't look like I'll find a job before Christmas at least.

Since it's binary compatible with most existing avisynth plugins you can use it for quite a bit already.

[osgZach]

Thanks for taking the time to respond to my question.

I really hope you have great success with this project, and we can reach a point where we blow past Avisynth in terms of functionality and other limiting factors (speed /technology etc).
I will be watching this software with great interest..

Maybe this is a bit of a stretch, but would it be right to assume, that since this appears to be built on Python, that all those neat libraries for mathematical and scientific calculations would be very useful for developers writing new plugins for this system? It would be really awesome to get the benefits of such libraries, I am thinking from a speed / complex operations standpoint.. Perhaps we will end up with new filters that not only replicate previous Avisynth filters, but do it better, faster, and more accurate / go beyond what the originals would be capable of in a reasonable amount of processing time...

It would be really nice to see some GPU agnostic assisted plugins come out of this, that prove to be useful and offer significant advantages.

[Myrsloik]

GPU stuff is overhyped. Proper coding and common sense will get you almost as far for most filters. I'll probably add support for it in some form one day anyway just to prove that I'm right.
A few facts about python. It's really slow and in practice single threaded. The c libraries behind the calculation modules will run much faster if used directly in a c plugin.
Maybe it could be usable for filter prototyping for some people but I predict a huge yawn there. Video filters also aren't really "scientific calculations", they're more of an exercise in stuffing as many pixels into an xmm register as possible.

[Myrsloik]

merge is slow, uses less than 40% CPU.

Post the whole script and define slow.

[Myrsloik]

38% CPU usage
1~ FPS output

And now the script.

[ryrynz]

Are there any plans for a DirectShow filter that can serve as a replacement for ffdshow's AviSynth processing?


Not at this time. It'd be pretty easy to implement for someone who is familiar with dshow though (and that's usually the bottleneck for dshow-related stuff anyway).

Anyone able to poke around for someone capable of doing this?

[Chikuzen]

Could std.AddBorders add a 'color' argument to be able to specify the color(like std.BlankClip does)?

I think it should be added.
addborders2.dll (http://www.mediafire.com/download.php?2nw4k3vtzg4b4nb)
http://pastebin.com/YnhJWvKu

[Myrsloik]

Could std.AddBorders add a 'color' argument to be able to specify the color(like std.BlankClip does)?

I'll add it in the next version.

[jackoneill]

I think it should be added.
addborders2.dll (http://www.mediafire.com/download.php?2nw4k3vtzg4b4nb)
http://pastebin.com/YnhJWvKu

I'm curious... how big is a wchar_t on your system?

#include <stdio.h>
#include <wchar.h>

int main() {
printf("%zu\n", sizeof(wchar_t));
return 0;
}

That prints "4" here.

Well, in any case, if I read your code right, it only works correctly if bytesPerSample is the same as sizeof(wchar_t).

[Chikuzen]

I'm curious... how big is a wchar_t on your system?


hmm, stddef.h(mingw) says "/* wchar_t is unsigned short for compatibility with MS runtime */"

my code may be have problem on 64bit Linux/Mac:(

[TheFluff]

wchar_t is two bytes on win32 (UTF-16, argh) but IIRC the spec only says it has to be "at least as wide as char" so it can really be anything from 1 byte and up. As jackoneill points out, it's 4 bytes on many Linux systems.

[Chikuzen]

wchar_t is two bytes on win32 (UTF-16, argh) but IIRC the spec only says it has to be "at least as wide as char" so it can really be anything from 1 byte and up. As jackoneill points out, it's 4 bytes on many Linux systems.

yeah, I confirmed it on Linux.
Fix version http://pastebin.com/aRFGAka3

[sneaker_ger]

Is there a way to prevent this mess when viewing the function list?
http://www.abload.de/thumb/function_list_mess5to2g.png (http://www.abload.de/img/function_list_mess5to2g.png)

[Chikuzen]

Is there a way to prevent this mess when viewing the function list?


#vsfunclist.py
import subprocess

def show(core):
fname = 'funclist.txt'
file = open(fname, 'w')
file.write(core.list_functions())
file.close()
subprocess.Popen(['notepad', fname])



>>> import vapoursynth as vs
>>> import vsfunclist as fl
>>> core = vs.Core()
>>> fi.show(core)


Python is very powerful language :D

[Myrsloik]

Is there a way to prevent this mess when viewing the function list?
http://www.abload.de/thumb/function_list_mess5to2g.png (http://www.abload.de/img/function_list_mess5to2g.png)

Ask for all functions, get all functions.

Doing something like core.avs.list_functions() also works if you only want to view one plugin. The interactive python window you can open from the start menu has nicer line breaks too.

[Myrsloik]

import vapoursynth as vs
core = vs.Core()
core.std.LoadPlugin(path_to_ffms2)
clip1 = core.ffms2.Source(path_to_file)#720p24
clip2 = core.std.Reverse(clip1)
last = core.std.Merge([clip1,clip2],0.5)

#output is from 1 to 5 FPS depending on content, CPU usage mostly at 45%
#import os
#with open(os.devnull,"wb") as null:
# last.output(null)


It's not merge that makes your script slow. It's what you're doing to the poor source filter.

1. Reverse() will always be slow since decoding a file backwards is slow.
2. You're pulling both inputs from the same source, it provokes it to seek to first frame, last frame, first frame + 1, last frame - 1, this is horribly slow. Using two instances of the source filter, one for the reversed clip and one for the forward one will most likely be a lot faster.
3. Because of 1 this will never be a fast operation, reverse could be optimized slightly by caching chunks at a time I guess but that definitely makes it a bit more complicated to implement and quite ram hungry too.

[Myrsloik]

R16 release to fix the major breaking issues in R15.
R16 – The Slightly Less Broken Release (http://www.vapoursynth.com/2012/11/r16-the-slightly-less-broken-release/)

[qyot27]

I figured out why. Sse2 math is enabled in my compiles. There's an event every 100 frames or so when internal statistics are updated using floating point.
So what can I say, vapoursynth requires a cpu with sse2. This is to make things easier for plugin writers too. Nowadays sse2 support is about as widespread as pure mmx support was 10 years ago or so.
Granted r337 prevents using it at all now on my old Celeron Coppermine, but I hadn't experienced any crashing when using native Linux builds; no problem at all serving out >100 frames there. Same computer, so it was still only restricted to the instruction sets the Pentium III had.

Would it have eventually had a problem, though, given enough frames? Or if it was simply using MMX/SSE instead, was/is there something unstable about it on a basic level that requires SSE2 (or disabling asm entirely, if it came to that) to solve?

[jackoneill]

Granted r337 prevents using it at all now on my old Celeron Coppermine, but I hadn't experienced any crashing when using native Linux builds; no problem at all serving out >100 frames there. Same computer, so it was still only restricted to the instruction sets the Pentium III had.

Would it have eventually had a problem, though, given enough frames? Or if it was simply using MMX/SSE instead, was/is there something unstable about it on a basic level that requires SSE2 (or disabling asm entirely, if it came to that) to solve?

You compiled the Linux version on the machine in question, didn't you? I imagine the compiler wouldn't generate code that can't run there, would it?

[sneaker_ger]

Since upgrading from r15 to r16 I've been experiencing random crashes when loading in VirtualDub through VfW. (Either crashes directly when loading the script or sometimes when skipping to a random frame)
Anyone else experiencing this? What more info would you need to find the problem?

http://pastebin.com/08qKfLqB (VirtualDub crash report)

[Myrsloik]

We've had SSE2 since forever. First CPU with it released 12! YEARS! AGO! Since then most cpus have had it, a few poor early athlons excluded. SSE2 is also required on any x64 cpus which have dominated the sales for a loooooong time.

We've had x64 cpus for 9 years.

I don't know what else to say, just go read my minimum requirements blog post. I have no intention of making the lives of plugin writers more difficult just to support a few ancient computers out there. Because in the end this project depends more on being reasonable to develop for than pentium 3 support.

[Myrsloik]

Since upgrading from r15 to r16 I've been experiencing random crashes when loading in VirtualDub through VfW. (Either crashes directly when loading the script or sometimes when skipping to a random frame)
Anyone else experiencing this? What more info would you need to find the problem?

http://pastebin.com/08qKfLqB (VirtualDub crash report)

The crash report looks like it's just pointing at random memory and the stack trace is weird too.

I need something else to go on. I've seeked around in vfw a bit and it doesn't seem to crash here. Maybe someone else has the same issue?
Does it go away when you use a simpler script?

[Robert Martens]

I'm having what looks like the same issue, a std::bad_alloc exception thrown from KernelBase when my filter attempts to copy part of one frame to another, but only when I use VSHelper.h's vs_bitblt. When I loop through the pixels myself, copying things manually, everything works as expected.

The code is a bit sloppy at the moment while I add Vapoursynth support, but if it'd help you track down the problem I can pack something up and send it along.

[Myrsloik]

I'm having what looks like the same issue, a std::bad_alloc exception thrown from KernelBase when my filter attempts to copy part of one frame to another, but only when I use VSHelper.h's vs_bitblt. When I loop through the pixels myself, copying things manually, everything works as expected.

The code is a bit sloppy at the moment while I add Vapoursynth support, but if it'd help you track down the problem I can pack something up and send it along.

Yes, I definitely need to see some code to figure out why it's happening. It's also odd that it throws a std::bad_alloc exception every time, are you accidentally writing to the wrong destination?

[sneaker_ger]

Can't really make it any simpler, because I need to convert 16 PNG input to 8 bit YUV to even make it work with VfW output.
import vapoursynth as vs
import sys
import os

core = vs.Core()

core.std.LoadPlugin(path=r'C:\vsimagereader.dll')
ext = '.png'
dir = 'C:/sintel-4k-png16/'
srcs = [dir + src for src in os.listdir(dir) if src.endswith(ext)]
ret = core.imgr.Read(srcs, fpsnum=24, fpsden=1)

core.std.LoadPlugin(r'C:\fmtconv.dll')

ret = core.fmtc.matrix(clip=ret, mat="709", col_fam=vs.YUV)
ret = core.fmtc.bitdepth(clip=ret, bitdepth=8)

last = ret

Setting core threads to 1 greatly reduces the number of crashes. (From like 3 : 1 on opening the script to 15 : 1)

[sneaker_ger]

Also noticed that VirtualDub does not free up all the memory when closing the script, so maybe it just runs out of memory. (And earlier with more threads)

[Robert Martens]

Yes, I definitely need to see some code to figure out why it's happening. It's also odd that it throws a std::bad_alloc exception every time, are you accidentally writing to the wrong destination?

Probably something likewise idiotic, and I wouldn't be surprised or offended to find at least some portion of the code highlighted in a future Vapoursynth blog entry. I'd have kept my mouth shut and spent at least a few days combing over my work before saying anything, if not for sneaker_ger's crash report mentioning KernelBase and bad_alloc.

Give me a few minutes to pack things up for you and I'll upload the source to Mediafire.

[qyot27]

You compiled the Linux version on the machine in question, didn't you? I imagine the compiler wouldn't generate code that can't run there, would it?
Yes I did, and that's generally what I figured was happening as well. My attempts at trying to get a Windows build cross-compiled (or natively compiled under either GCC-MSys or MSVC-MSys) all failed at one point or another, or resulted in a borked .dll.


At this point I can live with it (especially since I do have access to a Core 2, Athlon64, and Core i5, just not constantly), I'm just more curious about the architectural design.

[Robert Martens]

All right, here goes nothing: http://www.mediafire.com/?vzs4p8fauvdaevb

The plugin is the next version of TurnsTile (http://forum.doom9.org/showthread.php?t=158695), highlighted here: http://www.youtube.com/watch?v=WFJreg-LW50

I've stripped out everything related to both the Avisynth/Avxsynth version of the plugin and my use of CMake, and made a new project with only Vapoursynth-relevant content. The zip file includes the source and headers, a Visual Studio solution, a ten frame test clip with associated test tilesheet, and a .vpy script that's already set up to load the clips and call my function (though you'll need to change the script's absolute paths to the plugin DLL and test footage, of course). I've tested the zip file myself, unpacking it to a new directory and building the plugin, and it reliably demonstrates the crash.

You shouldn't need to bother with any of the function's other parameters, as the test script provokes the problem as is. Set the VS debugger to run the latest experimental release of VirtualDub with the provided test script as the command argument, and you should be set. I've marked portions of the code with "Myrsloik" in four places, so just search for your username and you'll get to the sections I'm having trouble with. The package as is has the working loops commented out, immediately below the broken version, so switching back and forth should be easy enough.

Let me know if there's anything else I can do to help.

[Keiyakusha]

I need to convert 16 PNG input to 8 bit YUV to even make it work with VfW output.
You probably want to say "even make it work in VDub"
vfw supports P016, Vdub is not.

Edit: i mean not only p016, vfw module supports more than that

[Myrsloik]

All right, here goes nothing: http://www.mediafire.com/?vzs4p8fauvdaevb

The plugin is the next version of TurnsTile (http://forum.doom9.org/showthread.php?t=158695), highlighted here: http://www.youtube.com/watch?v=WFJreg-LW50

I've stripped out everything related to both the Avisynth/Avxsynth version of the plugin and my use of CMake, and made a new project with only Vapoursynth-relevant content. The zip file includes the source and headers, a Visual Studio solution, a ten frame test clip with associated test tilesheet, and a .vpy script that's already set up to load the clips and call my function (though you'll need to change the script's absolute paths to the plugin DLL and test footage, of course). I've tested the zip file myself, unpacking it to a new directory and building the plugin, and it reliably demonstrates the crash.

You shouldn't need to bother with any of the function's other parameters, as the test script provokes the problem as is. Set the VS debugger to run the latest experimental release of VirtualDub with the provided test script as the command argument, and you should be set. I've marked portions of the code with "Myrsloik" in four places, so just search for your username and you'll get to the sections I'm having trouble with. The package as is has the working loops commented out, immediately below the broken version, so switching back and forth should be easy enough.

Let me know if there's anything else I can do to help.

From a quick glance on my phone I'm wondering why you don't use tileW_UV in the bitblt. I'll compile and debug it properly tomorrow if that isn't the real issue.

[Robert Martens]

I didn't use tileW_UV for the blit because I was rushing, and didn't spend enough time looking over my code. I wrote the planar RGB version first, where I use the same numbers for each call to vs_bitblt, and then did the YUV version without thinking. Switching to use tileW_UV and tileH_UV solves the issue for me. Thanks for catching that, and sorry to sneaker_ger that my mistake isn't related to your crash.

While I'm here, I'd like to confirm something if I may: can we safely assume that the stride for red, green, and blue planes in planar RGB, and the U and V planes for planar YUV, will always be identical? Or are there circumstances where all the planes could have different pitches?

[Myrsloik]

A plane of the same width and bytesperpixel will always have the same stride.

[sneaker_ger]

You probably want to say "even make it work in VDub"
vfw supports P016

But VS does not support P016 + 4:4:4 VfW output.
Actually I would have preferred to use simple old RGB24 to make some screenshots, but that's not supported yet either.

[sneaker_ger]

"On purpose"? You mean "not yet", right?
Anyways, my goal was to make some screenshots of the 16 bit -> 8 bit dithering of fmtconv and I wanted to quickly use VirtualDub to chooe the frames and copy them to clipboard. My initial post was a bit misleading. Since RGB24 VfW output is not yet supported, 8 bit 4:4:4 YUV (BT709 PC) seemed like the next best choice to me.

[sneaker_ger]

Your post disappearing means you just successfully tested 4:4:4, I guess.

[Chikuzen]

I ported FineSharp (http://forum.doom9.org/showthread.php?p=1569035#post1569035) to VapourSynth.

https://gist.github.com/4111749

This may be an example to port avs scripts.


import vapoursynth as vs
import finesharp
core = vs.Core()
core.avs.LoadPlugin('removegrain.dll')
core.avs.LoadPlugin('repair.dll')
clip = some.source('file')
fs = finesharp.FineSharp(core)
clip = fs.sharpen(clip)


EDIT:
script updated.

[Myrsloik]

I ported FineSharp (http://forum.doom9.org/showthread.php?p=1569035#post1569035) to VapourSynth.

https://gist.github.com/4111749

This may be an example to port avs scripts.


import vapoursynth as vs
import finesharp
core = vs.Core()
clip = some.source('file')
fs = finesharp.FineSharp(core, '/path/to/RemoveGrain.dll', '/path/to/Repair.dll')
clip = fs.sharpen(clip)


Mostly looks ok. I think having filter paths in the script is a bad idea though. Just throw an error if the prerequisite plugins aren't loaded.

You should also be using std.Merge() for your mergeluma function. The goal is to use vs as much as possible, right?

[Chikuzen]

You should also be using std.Merge() for your mergeluma function.

std.Merge(clip[] clips[, int[] weight])
http://www.vapoursynth.com/doc/functions/merge.html

Since I thought that weight could use only int, I used lut2.

[Myrsloik]

std.Merge(clip[] clips[, int[] weight])
http://www.vapoursynth.com/doc/functions/merge.html

Since I thought that weight could use only int, I used lut2.

Documentation typo. It really is float.

[Chikuzen]

- .. function:: Merge(clip[] clips[, float[] weight])
+ .. function:: Merge(clip[] clips, float[] weight=[0.5])

[Revgen]

Since VapourSynth doesn't support audio yet, I've ported/rewritten TheFluff's split_aud script (http://mod16.org/hurfdurf/?p=33) as a Python module for use with VapourSynth.

Name: sam (for "split aud module")
Requirements: mkvtoolnix
Limitations: no non-linear trimming, no vfr support

Typical use case:

video_path=""
audio_path=""
DGDecode=""
VIVTC=""

import vapoursynth as vs, sys, sam
core = vs.Core()
core.avs.LoadPlugin(DGDecode)
core.std.LoadPlugin(VIVTC)
vid = core.avs.MPEG2Source(video_path)
vid = core.vivtc.VFM(vid,1)
vid = core.vivtc.VDecimate(vid)
vid = core.resize.Bicubic(vid,1280,720)
vid = sam.trim(vid,100,200)+sam.trim(vid,13000,13100)
sam.split_merge(audio_path)
last = vid
last.output(sys.stdout, y4m=True)

Source:
http://pastebin.com/q25ea2kN
Testing, thoughts, comments, suggestions and improvements(!) are appreciated.

Thx. I did a quick test on a 1920x1080i 29.97i MPEG-2 file using two trims. Encoded to h264, Muxed the .264 file and the .mka file in MKVToolnix. Audio was in perfect sync. I might try something more complicated next week after Thanksgiving is over.

[Chikuzen]

@HolyWu
I updated that script, and created new thread (http://forum.doom9.org/showthread.php?p=1601585#post1601585).

[Chikuzen]

import vapoursynth as vs
core = vs.Core()
clip1 = core.std.BlankClip(format=vs.YUV420P8, color=[0, 128, 128])
clip2 = core.std.BlankClip(format=vs.YUV420P8, color=[255, 128, 128])
last = core.std.SelectClip([clip1, clip2], clip1, lambda n, f: n % 2)


last.output(file) works as expected.
But VFW module does not.
VirtualDub/AVISource has frozen.

[Myrsloik]

import vapoursynth as vs
core = vs.Core()
clip1 = core.std.BlankClip(format=vs.YUV420P8, color=[0, 128, 128])
clip2 = core.std.BlankClip(format=vs.YUV420P8, color=[255, 128, 128])
last = core.std.SelectClip([clip1, clip2], clip1, lambda n, f: n % 2)


last.output(file) works as expected.
But VFW module does not.
VirtualDub/AVISource has frozen.

I've figured out why. The GIL deadlocks on itself. It's a bit of a mystery why it isn't properly released at the end of script opening and I really can't rule out a cython bug.

Here's a new vsvfw.dll (https://dl.dropbox.com/u/73468194/vsvfw.dll) that should work. I hope you enjoy my ugly hack.

[Revgen]

here's a replacement for bob, at least using avisynth code (i haven't had time to teach myself python yet):

interleave(separatefields().selecteven().bicubicresize(last.width,last.height,1/3.,1/3.,0,.25,last.width,last.height/2),separatefields().selectodd().bicubicresize(last.width,last.height,1/3.,1/3.,0,-.25,last.width,last.height/2))

i think vapoursynth should have enough internal filters to implement the above line, if the resizers are up to it.

It's now possible with fmtconv, but it's pretty slow. Runs at 49-50fps at 1920x1080 on my Q6850 quadcore in vdub. It crawls to 25fps if I use vs.Core(threads=1).

video_path=r'E:\myvideo.d2v'

import vapoursynth as vs
import sys

core = vs.Core()
core.std.LoadPlugin(r'C:\Program Files\VapourSynth\fmtconv.dll')
core.std.LoadPlugin(r'C:\Program Files\VapourSynth\d2vsource_beta3_mingw.dll')
vid = core.d2v.Source(video_path,0)
sep = core.std.SeparateFields(vid,tff=1)
evn = core.std.SelectEvery(sep, cycle=2, offsets=0)
odd = core.std.SelectEvery(sep, cycle=2, offsets=1)
evn = core.fmtc.resample(evn,w=1920,h=1080,sx=0,sy=0.25,sw=1920,sh=540,fulls=0,fulld=0)
odd = core.fmtc.resample(odd,w=1920,h=1080,sx=0,sy=-0.25,sw=1920,sh=540,fulls=0,fulld=0)
evn = core.fmtc.bitdepth(evn,bitdepth=8,fulls=0,fulld=0)
odd = core.fmtc.bitdepth(odd,bitdepth=8,fulls=0,fulld=0)
vid = core.std.Interleave(clips=[evn,odd])
last = vid
#last.output(sys.stdout, y4m=True)